Memory Unleashed on Threadripper: 128gb & 2933 & ECC tested | Level One Techs

Yep It is, WGet the deb from the site, dpkg -i, find and resolve dependency install tests.
It looked a little intimidating from the manual at first but I’ve managed to install some tests and run the first one all from the console.
https://openbenchmarking.org/result/1811235-RA-RUN1STREA45
They seem OK for 2933?
I’ve 15 VMs running so… I could find some more performance…

I’ve worked out how to get all the tests on one link
https://openbenchmarking.org/result/1811232-RA-4939X399D75

Is it possible to remove a result? I didn’t intend to upload “Jef_UKs_1950x-2933_ECC-CL16” and cancled it part way through to re run with a name that matched the rest better and it looks a mess sorry :confused:
https://openbenchmarking.org/result/1811235-RA-1810154FO56&obr_hgv=Jef_UKs_1950x-2933_ECC-CL16&obr_sor=y&obr_hgv=Jef_UKs_1950x-2933_ECC-CL16

Certainly not for any anonymous test results, that would be a bad idea.
I’m not sure what might be offered with an account though, that info seems to be a bit sparse.

You can clone any of the tests that are online, rename, sort, and remove any entries, and re upload it, though it will get a new name and link.

@wendell I was a little late getting to your last video update on Threadripper memory, thanks for the shout out :slight_smile:

@CrayZeApe have you ever seen this issue with pts/opencv-bench-1.0.0/opencv-benchmarks-0.1?

/home/…/.phoronix-test-suite/installed-tests/pts/opencv-bench-1.0.0/opencv-benchmarks-0.1/src/discrete_fourier_transform.cpp: In function ‘int main(int, char**)’:
/home/…/.phoronix-test-suite/installed-tests/pts/opencv-bench-1.0.0/opencv-benchmarks-0.1/src/discrete_fourier_transform.cpp:39:30: error: ‘CV_LOAD_IMAGE_GRAYSCALE’ was not declared in this scope
Mat I = imread(filename, CV_LOAD_IMAGE_GRAYSCALE);
^~~~~~~~~~~~~~~~~~~~~~~
/home/…/.phoronix-test-suite/installed-tests/pts/opencv-bench-1.0.0/opencv-benchmarks-0.1/src/discrete_fourier_transform.cpp:84:33: error: ‘CV_MINMAX’ was not declared in this scope
normalize(magI, magI, 0, 1, CV_MINMAX); // Transform the matrix with float values into a
^~~~~~~~~
/home/…/.phoronix-test-suite/installed-tests/pts/opencv-bench-1.0.0/opencv-benchmarks-0.1/src/discrete_fourier_transform.cpp:84:33: note: suggested alternative: ‘CV_MMX’
normalize(magI, magI, 0, 1, CV_MINMAX); // Transform the matrix with float values into a
^~~~~~~~~
CV_MMX

Phoronix then complains:

    pts/opencv-bench-1.0.0:
        Test Installation 1 of 1
        1 File Needed [0.04 MB / 1 Minute]
        File Found: opencv-benchmarks-1.0.tar.gz                    [0.04MB]
        Installing Test @ 17:48:28
            The installer exited with a non-zero exit status.
            ERROR: collect2: error: ld returned 1 exit status
            LOG: ~/.phoronix-test-suite/installed-tests/pts/opencv-bench-1.0.0/install-failed.log


    [PROBLEM] pts/opencv-bench-1.0.0 is not installed. 

Looks like it’s not seeing the OpenCL libs.

I’m not sure what options Arch gives you for OpenCL, but many tests require OpenCL 2.0.
Mesa can be build with OpenCL 1.1 support in the form of Clover, but for 2.0 support, you might be limited to what comes with the proprietry graphics drivers for now, in which case it becomes more of a GPU compute test than anything else.

You can install opencl-amd and see how you go.

Relative Info:
https://wiki.archlinux.org/index.php/AMDGPU
https://aur.archlinux.org/packages/amdgpu-pro-opencl/

Greetings! I recently tried to join the 128gb ram on Threadripper club, but I failed. My build log is Here

I got it booted up earlier today and only 4 of the 8 ram sticks are showing up in UEFI. I’ve read up a ton in preparation for this build and I suspect the CPU needs to be re-seated, but I would really appreciate your feedback.

Thanks in advance for your help!

Use your cell phone to take really close up pics. Loose dimm? Seated level both edges? Take pics to show ram seated right on both sides? Used the toruqe wrench well?

Good evening good sir!

This is my first Threadripper build, but I’ve installed at least 1000 if not 2000 ddr4 modules and I have not had this issue before.

I followed these two videos as my method of installation for the CPU:

An interesting aspect is that it is all of the sticks on the right side, i.e.
DDR4_A1
DDR4_A2
DDR4_B1
DDR4_B2

Thanks!

Any chance the memory is in backwards on that one side? The memory orientation rotates between the two sides? Remove all the memory from that side ?

Edit: yep in your pic the memory is in backwards on one side which means it’s not snapped in all the way on the one side.

Because TR, the memory sticks rotate around the socket. Memory on one side is rotated 180 degrees relative to the other side on TR. (Not so on Intel, sometimes)

Lol Wendell! You know that feeling you get when you’ve left the house and someone says “You locked the front door right? , You turned off the stove right? , You closed the garage door right?,” You know you did it, but that seed of doubt has been planted and it just grows and grows until you go back and check it.

I read your message but then wasn’t able to get back to the machine for hours. Needless to say, the doubt-tree grew to full size. You got me good on that one lol! :joy:

The memory is correctly installed on both sides, in 180 degree rotation like you mentioned! When I went to populate the left side, I did do a double take about the rotation before I installed the memory. I made a note of that Gotcha in my personal build notes to share with future builders as well.

I’m at the workbench for a bit now and hopefully I can get this resolved. It will be a shame if I have to take the cooler off as I was hoping to get some thermal performance benchmarks with the pre-applied TIM included with the Wraith Ripper to share with future builders.

Also, a forum member mentioned that they do not see images in my build log, only links. On my end I see images and captions. What do you see on your end?

Seriously, I appreciate you taking the time to share your feedback with me. I’m new to this community, but I’m extremely pleased with what I have seen so far.

Thanks

A follow up as I work on it.

I removed one stick from DDR4_B2 and booted and still had the remaining 3 sticks undetected.

I then removed the sticks from DDR4_B1 and DDR4_A2 and booted. Now the stick in DDR4_A1 shows in UEFI.

I’m going to add the sticks back in 1 at a time, but I’m sure they were well seated and installed correctly. :crossed_fingers:

Edit: Progress

That did the trick!

I drank WAY too much caffeine for all of this excitement this morning.

Do I get a membership card now? When are the weekly meetings for the 128GB RAM on Threadripper club? :grin:

Thanks for the support everyone!

I missed the fun, glad you got it sorted out.

TR+128GB is more of a secret society than a club at this point.
Don’t settle for a membership card, we need an Illuminati style ring on our fingers :slight_smile:

I’ve just been playing about at the other end of the memory spectrum, a C-Sky SOC with a whopping 64MB.

1 Like

Still working a little bit on getting tests running on Arch – upgraded the tests to the latest version and that seems to have helped a bit. Maybe its generating some more helpful error or I’m just getting more familiar with how its working.

One major issue with Phoronix and Arch is that python is defaulted to version 3.x and there are lots of scripts that assume version 2.x. I’m not sure how to work around this – I tried a few things that an internet search suggested, but directly modifying the files to explicitly call python2 is tho only thing that has worked for me so far.

Also ran into issues where the result parser wasn’t able to extract data from the test because (for instance) my machine is CST and the parser identifier string was expecting EST it seems. Manually updated that XML file and it works, but I haven’t been able to find a lot of documentation on how this is supposed to work. Ideally I would figure out how to express a more general case and make a pull request upstream…

As you found, getting Phoronix working on various distros is, ummm, interesting to say the least.

Did you encounter in your search, this answer/post from houbysoft:

Another thing you could try is putting a python->python2 symlink in some random directory, and then add that directory to the front of your path as you call the test suite.

Something like this:

mkdir /random/directory
cd /random/directory
ln -s /usr/bin/python2 python

Just call phoronix-test-suite instead of “python --version” in the following example:

$ python --version
Python 3.6.6

$ PATH=/random/directory:$PATH python --version
Python 2.7.15

I haven’t looked at the Phoronix source enough to suggest anything on the result parser. The whole project reminds of a giant bowl of spaghetti :expressionless:

I’m a little late to the thread but I have 6 of the following systems available for testing over the next few weeks, just built them for our animation/modeling employees and just haven’t deployed them yet. All RAM showed up with no issues at all.

AMD 2990WX
Gigabyte Aorus Xtreme
1TB Samsung 970 Pro
128GB of Vengeance Pro 3200 RGB
2 - Nvidia 2080ti FE
Enermax AIO Cooler
EVGA 1600P2
Windows 10 Pro

1 Like

Some new coverage from Phoronix. @wendell :slight_smile:

https://www.phoronix.com/scan.php?page=news_item&px=ECC-DRAM-Overclocking-TR

1 Like

Just wanted to know if you all experienced anything like this.

I have a 2990WX Threadripper build with 128 GB G.Skill 3200 15-15-35-50 RAM (3200C15Q2-128GTZR). I can only run it at 2933, which I’ve accepted as the highest mark. I still have slight instability issues, but I think I’m going to just accept the risk. I’m running 128 GB of this

But I have an interesting score for my Stream: Copy test.

It’s about 66% of what you all are receiving for your runs. But my other Stream tests are in the same ballpark as the tests you all are running. I’m getting ~ 40 GB/s and I’m seeing ~ 60 GB/s for you all at 2933.

Any ideas about this? Have you all seen this before? Is there something I’m missing?

Here is a link to the results.
https://openbenchmarking.org/result/1902162-SP-MEM68277305

@wendel You mentioned in your video that the Crucial Memory you used was Dual Rank; however, in a blog written by Max E, he mentions that you used Crucial single rank. I couldn’t find the serial number of the Crucial memory on your article. So, it is single or dual rank?

Okay. I found the model in another of your videos. https://www.amazon.com/dp/B078N7HC6L/?tag=level1techs-20

Thank you for your work!!!

Good Evening,

I am new the forums here. Very interested in this thread and learning a bit more about what is involved with getting 128GB 8x16GB dual rank Bdie dims running @ 2933 stable on my x399 taichi board.

From watching the level1 videos i have learned so much. Wish i could go back and get a 2nd gen board but alas i am here now with my Taichi. Curious if anyone has one running 128GB stable @ 2933 and what settings they recommend or tweaks to increase stability?

I also have one specific question: I currently have my 2080Tis in PCIE slots 2&4 (x8) not 1&3 (x16). I just liked the way it looked better and saw that the 2080Tis don’t saturate the x8 bandwidth. After watching one of the level1 videos however it was mentioned that keeping items close to the CPU is ideal… wondering if anyone knows if this can affect stability or cause issues? Moving them would be time consuming to say the least. Re-create back-plate and drain ect… so I figure i would ask before just doing this one.

I get random BSDs and memory errors @ 2933.

Things i have tried @ 2933:

  • Ran MemTest did a full pass then focused IMC test 6. No errors.
  • Ran Prime 95 FFt 512-4096 for 24 hours. No errors.
  • Re-seated Ram.
  • Dropping to 2800 seems to help. Still got one BSD …seems less frequent

What I am experiencing:

  • Random BSOD’s from doing anything from light gaming to Photoshop work.
    System_Service_Exception (usually). Event Viewer says little. Usually stable for most of the time. Crashes once a day on avg. Like a phantom.

What I am thinking of trying:

  • Testing the dims individually ?
  • Increasing voltage to 1.38 from 1.35?

What i use the PC for:

  • Docker / virtualization / Linux
  • Photoshop & Rendering / Windows
  • Light Gaming / Windows

Current Hardware:
TR2 2950X - Creator Mode / Stock Settings
x399 Taichi Board updated to Bios 3.5
128Gb TridentZ Royal 3200 14CAS Samsung B-Die Ram
2x2080Ti in SLI
Dual EK custom hardline / loop system. Usually sits at 28C.
1600watt EVGA 80 Gold PSU

I am very new to TR2 and just finished a build that took 4 months to complete. Have found using it frustrating to say the least with the random BSOD issues and lack of stability. Feels just sensitive all around compared to my old 930 i7 that ran for 8 years like a rock. Granted i know I have 128GB of ram running on a pretty old memory controller…

Any help or info would be appreciated! Also what is the best way to test chips individually?

Thanks very much for any time invested in this,
-V