DDR5 rdimm sugestion for threadripper build?

The biggest different in RDIMMs you’ll notice for the platform is going from 1Rx8> 2Rx8/1Rx4>2Rx4 DIMMs. It is highly likely that the very best binned/most expensive 2Rx4 RDIMMs won’t clock nearly as well as a bone stock, cheap non-EXPO, 1Rx8 RDIMMs.

I wouldn’t count on this indefinitely, the vcolor RDIMMs with the traditional heatspreaders aren’t being manufactured anymore and vcolor has moved on to using heatslugs instead because they claim they cool better. Once all the old stock is gone, it is likely that this kit will be manufactured with the new heatslug style solution.

I’m not saying this is impossible, but the highly tuned TR pros I’ve seen only manage about 90ns for memory latency if measured by aida64. I have however seen TR non-pro get into the low 60ns memory latencies with enough tuning due to their less congested architecture.

TR PRO 7975WX at DDR5-6400 with 1.3V VDD/VDDQ/VDDIO_S3 boosting to 5550MHz:

Like previously indicated, sub-60ns access latency.

3 Likes

dang, that’s the best I’ve seen yet on TR Pro

Sadly these findings aren’t applicable for higher capacity (48/64 GB) sticks. If they existed, I’d happily buy single rank 64GB sticks.

Interestingly, I use the exact Zen timings you posted in New Workstation | Buy Zen 5 EPYC Now or Wait for Zen 5 Threadripper? - Hardware Hub / Build a PC - Level1Techs Forums on a 7970X/Asus Sage, and am only getting 65ns at best in the same test.

Shimada Peak removed the MCLK=UCLK option, so latency tanked.
That’s why I’m waiting for the 8200/CL38 kits, you have to go as high as possible now.
Interestingly, Asus replaced the 8400MT/s presets in BIOS 1106 with 8000 ones in 1203 (see screenshots at Puget Systems).

GSkill posted a 8400 8x32 overclock last week, but that was from a 7600 kit that is not available for sale.

If your timings differ from the ones you posted before, I’d be interested in an update.

The V-Color kit from my post is Dual Rank (2R) of 16Gb SDRAM ICs, which is why it’s so good at scaling with tight timings. The performance impact trade-off at higher densities it not due to the number of ranks, but the DENSITY of the individual ICs.

Be sure to also DISABLE Security Memory Encryption (SMEE).

Also, set APBDIS to 1 and DFPState to 0 as well as DF CStates to 0 to avoid IF latency hits at the start of streaming loads.

Finally, I recommend NPS4 for NUMA.

That’s interesting, I thought with TR Pro you are in latency hell with around 90ns.

Thanks a lot, I’ll give that a try.

Ah, that must be why your latency is lower, NUMA>1 does that.
I use my machine for C++ compilation, and with only 32 cores NUMA1 is always faster, probably because of the shared cache.
But with my 9985WX it will probably be a contest between NUMA4 and NUMA8 (“ACPI SRAT L3 Cache As NUMA domain” in DF Common Options).

I just built a system with 9975wx and v-color 384g 6800 ram.

Thread: Threadripper 9975wx! <-- now a build log - #15 by aatchison

Ram:
https://www.newegg.com/v-color-wrx90-oc-r-dimm-384gb-ddr5-6800-cas-latency-cl34-memory/p/2SJ-004R-00056?Item=9SIAMCMK9U6426&_gl=1*jmduzz*_gcl_au*MzgyMDM4NTEyLjE3NTM3NTA5Mzk.*_ga*OTUyODU4ODE5LjE3NTM3NTA5NDA.*_ga_TR46GG8HLR*czE3NTQxODMyMTYkbzUkZzEkdDE3NTQxODMzODUkajYwJGwwJGg2ODYxNjkwMDI.

Other than temps, it’s looking good. Any information I could provide, I would gladly assist.

Well those settings worked, but NUMA4 is still making my builds slower (by about 5%).
I get about 66ns in NUMA1.


Boom.

Trade-offs as always.

ASUS still hasn’t updated the QVL for the WRX90, but the ASRock one shows V-Color memory that doesn’t exist (8200 MT/s only for 32GB/module, when V-Color has only announced that speed for 24GB modules).
Only 4 sticks though, the highest 8 socket support is 8000.
ASRock > WRX90 WS EVO

1 Like

There is no functional difference between 4-channel and 8-channel memory with TR vs. TR PRO. Each CCD is serviced by a 2-channel IMC in the I/O die. The load on the IMC is the same as a dual-channel AM5. The only difference is there are either 2 or 4 of those operating in parallel. This is why buying 2 kits of “TR50” memory is an effective approach for the WRX90 platform. Whatever you want, just buy 2 kits (of 4 sticks each) and proceed with confidence.

The TR I/O die IS the TR PRO I/O die, only with 2 of the 4 internal IMCs disabled. The TR I/O die is simply a TR PRO I/O die with manufacturing faults in either 1 or 2 IMCs or a number of the PCIe complexes which have been permanently disabled. Ergo, the TR PRO IMC is not an 8-channel memory controller, it’s a “quad” 2-channel memory controller, enabling high-speed low-latency memory performance just like its lesser Ryzen counterparts.

Yes, but when you hit bugs like “works with 7 sticks but not 8”, it’s nice to know that they’ve tested this exact same RAM with 8 sticks populated, and official QVL support can matter if you need a RMA.
For the 8200 RAM I mentioned, the fact that they list 2 and 4 sticks but not 8, while listing 8 for the next lower speed, would give me pause.

The fact that you and I get the exact same results with the exact same settings with 7970X/4 sticks and 7975WX/8 sticks pretty much confirms your last point :slight_smile:

I believe all those troubles were an artifact of less-than-fully-mature AGESA for TR/TR PRO, remedied since, for example, ASUS BIOS 1203 (really, 1106, which mysteriously disappeared from ASUS’ webpage after the release of 1203). It makes no sense why 3 of the 4 memory controller subunits would work but not the 4th, all else being equal.

You have to take an understanding of the modular approach to building CPUs to its logical conclusion. These are not all different designs - they are built to be scalable, and dies that don’t bin out (or have manufacturing defects) are not discarded… they are re-purposed for other products. Waste not, want not.