DDR5 4 dimms on am5 -- what's working, what's not?

I can’t confirm this combination is the best any more, sadly. With updates the performance I can reach has been reduced to 4800MT/s for two DIMMs per channel in ECC mode. The ECC still works just fine, and I do have 128GBs of it, but the performance is likely lower than what you’d be able to get with more modern 48GB DIMMs. If you care about performance and don’t strictly speaking need 128GB, consider doing one DIMM per channel 96GB total instead. It’ll work way better.

1 Like

That being said, impact might be barely non depending on workload.

1 Like

Thanks diizzy, I did read you had mentioned another brand though I thought you were using an MSI board (though I cannot find that post now searching) rather than the ProArt which seems to be often recommended for VFIO, though in my PC Build thread another recommendation has been made. (Black Friday 9950X3D ML, LLM, & gaming build). Perhaps it is the case that we can generally expect the same experience with ECC or DDR5 across x670e or AM5 generally?

I do not see that specific Micron model on the ProArt x670e, though the Kingstons are and rated at 4800, albeit only for 1,2 socket support.

Great share. My use case here fits into Machine Learning / AI, Software Development, 1440p gaming, and I guess Science and Research.

Sorry to hear that but glad to receive the update.

I do care about performance, and I would like to play games sometimes, and I see from the article diizzy linked is does impact some of my other use cases, but I am trying to balance that with my plans to backtest financial market trading algorithms, and economic modelling, which if my understanding is correct I should only use EEC for which will also have the bonus of a more stable system.

I am also considering the rule of thumb I have read for machine learning that the system RAM should be twice VRAM, and since the build will be with dual 3090s, primarily so I have the option to run LLMs >24GB, 96GB is exactly that. As I also want to be able to run 2 VMs at all times, and sometimes a third VM simultaneously Windows gaming I have been aiming for 128GB. I have queried if 96GB is adequate for my needs in the ML/LLM section: < 96GB RAM on 48GB VRAM?

1 Like

This thread has been an amazing resource for 4x32G, and one of the only written ones I’ve found.

I want to report a working configuration I’ve settled into in case anyone else wants to try, and to see if folks here can spot any obvious areas for improvement:

CPU: Ryzen 9 7900X
MB: MSI MAG B650 Tomahawk Wifi (MS-7D75)
Memory: Two 2x32G sets of GSkill F5-6000J3040G32G (rated at 6000MT, CL-30-40-40-96 at 1.40V)

These should be decent but not top-of-the-line parts, originally purchased only targeting 64GB (2x32) in mind. I decided to stick another 2x32 set in, and with some tweaking got it stable at a modest performance level.

Overall result:

  • 4x32GB Stable at 4800MT, CL34-40-40-96
  • Memory bandwidth 63GB/s and 86ns latency isn’t great, but substantially better than stock 3600MT
    • This is comparable to running a single 2x32 set without EXPO.
    • For reference, a single 2x32 set with the 6000MT EXPO profile reaches 77GB/s and 74ns latency - so there’s definitely a performance penalty.

Tweaks:

  • Timings set to 34-40-40-96.
    • Pretty sure I can squeeze down to tCL=32 with a bit more tuning, but I didn’t want to spend that effort.
    • Given the sticks themselves are rated at 30-40-40-96, how much room is there to squeeze? Why is the tRAS=96 so high?
  • VDD/VDDQ/VDDIO set to 1.3V
    • Sticks are themselves rated at 1.4V and this config works at 1.4V as well. I figured I’d undervolt a bit as I wasn’t too comfortable with 1.4V (for no particular reason).
  • VSOC set to 1.15V
  • Set the following. I think they were what ultimately pushed me over the edge to stability:
    • RttNomWr=40
    • RttNomRd=40
    • RttWr=129
    • RttPark=40
    • RttParkDqs=40

Questions:

  • Any thoughts/comments on this setup?
  • Anything obvious I should try to tighten up the timings or boost performance? Given the diminishing returns with memory scaling I don’t want to spend too much more time on this, but I’m happy to try any low-hanging-fruit.
  • My gut sense is that this probably won’t push to 5200MT - do folks here agree?

cachemem_128gb_v2

1 Like

Yeah. that’s interesting. I haven’t looked at Hynix A-die closely as the DIMMs I have to work with are all M-die. With M-die I’ve put tRAS into the 40s, though the BIOS is probably overriding to what would be tCL + tRCDRD = 74 in this case.

With M-die I was able to drop tRFC to ~400 and tRFCsb to ~280, (extrapolated from 5600 1.25 V to 4800 1.3 V with first order voltage and speed adjustments). It’s my impression A-die wants a bit looser timings but, if you haven’t already checked, there might be some room for improvement.

Also, if SPD temperatures aren’t over 70-75 °C putting tREFI up to 5-6 μs should be no problem. DDR tuning guides usually indicate considerably higher (20+ μs) but, in my AM5 benching, I’ve seen some perf gain and power reduction in the 3.9 → 5.5 μs range but no change past that.

Worth at least trying, IMO, though with the VSOC and termination changes for stability at 4800 I agree the odds are against 5200. However, some 4x32s are hitting 6000, 1.3V should be overkill for 5200, and since 7900X supports 5200 at 1DPC the IMC’s guaranteed to handle the clocks. So it’s just a question about pushing 2DPC loads a bit farther.

That said, when I benched 7950X I found some workloads ran slower at 5200 than 4800. So if 5200’s stable it may not be worth it.