Purpose: AM5 Linux Workstation for Proxmox VE and VFIO compatible. All parts are from one supplier and they will do the build.
BOM:
Processor | AMD Ryzen 9 7950X3D 16 Core Processor |
CPU Cooler | Deepcool Assassin IV CPU Air Cooler Black
Motherboard | ASUS PROART B650-CREATOR |
Queries: Hopefully the IOMMU groups are ok now. Please advise of any gotchas. The board does not have Wifi/BT, Please suggest a suitable card if possible.
Query:
Options 1 and 2 are the same price. Also would there be any issue with upgrading to DDR-5600. Not sure if the supplier knows of any constraints for not recommending Option 2. My questions are:
Would it be better downgrading to Option 3 for that 10% cost saving at the cost of lower performance.
Is it a must to buy one 4x32GB kits ?
Storage: Samsung 990 EVO 2TB M.2 Gen 4 NVMe PCIe [Supports PCIe 5.0x2 / PCIe 4.0x4 ].
Its double the price of standard PCIe 4.0 with Read Speed of 5000MB/s and write speed of 4200MB/s.
I initially did not consider PCIe 5.0 (this seems to be the only one with that supplier) but there are other options in similar price range such as
- T-FORCE A440 Lite, M.2-2280 Internal SSD, PCI-E, 2TB, Read 7400MB/s, Write 6400MB/s
- Kingston KC3000 PCIe 4.0 NVMe M.2 SSD 2048GB - Read 7000MB/s, 2048GB - Write 7000MB/s
These on paper seem to offer faster performance than Samsung. Please advise whats going on here.
One benchmark suggests Micron Crucial T500 2TB M.2 NVMe PCIe4 SSD is the faster drive than the Samsung 990 EVO 2TB M.2 Gen 4 NVMe PCIe as well.
Case | Cooler Master HAF500 ARGB E-ATX Case
PSU | FSP HYDRO G PRO 1000W 80+ Gold PCIe Gen 1
Any suggestions for better Cases and PSU aternatives or are these top tier? I am still doing my research.
CPU: The 7950X3D is probably a bit of a waste for this setup but if you want to throw money at it sure, go for fit.
CPU Cooler: Likely fine looking at be quiet! Dark Rock Elite Review - AMD Socket AM5 Results | TechPowerUp
Motherboard: Probably fine, Realtek NICs though…
Memory: Get something that follows JEDEC specs, some of the Fury series do otherwise https://www.idealo.de/preisvergleich/OffersOfProduct/203263217_-pro-64gb-kit-ddr5-5600-cl46-cp2k32g56c46u5-crucial.html
Case: Probably fine, I prefer the Fractal Design R-series though…
Storage: Get Crucial or Solidgm, P5 Plus (EOL), T500, P44 Pro are “decent” drives in general
PSU: Likely fine but way overkill, a decent 750W will do more than fine unless you plan to do something very excessive
Look to go with 2x 48GB RAM at higher speeds (up to DDR5-6400). DDR5-6000, I think, is considered the sweet spot on AM5.
Upgrade to workstation class 4 channel or higher platform.
In case of 1. consider saving on RAM by going with slower speed RAM sticks (e.g. 4x DDR5-4800), although I don’t think I have seen any reports on success with 4 sticks as most people want to take advantage of higher bandwidth here. Should be fine, though, because these are JEDEC certified speeds.
You don’t really want to go about of spec if you want something stable which seems to be the main goal here. Most of not all reports failure reports are using non JEDEC and/or manufacturer that focues on not meeting industry standards and going or out of spec specifications (overvoltage, SPD values being weird/broken/not targeting JEDEC speeds etc).
Here’s the completed build. No plans to overclock.
I got a deal on some Kingston RAM so thats what I got.
It seems the system is not stable for windows above 4200 MTs for windows. I have not tested personally; feedback from my IT supplier.
I think assumption is that the CPU speed is around 4200 Mhz so thats why they downgraded to 4200MTs for RAM…
I installed Proxmox and did bump up the RAM clock speed to 5000 MTs but ran into some issue which I cannot 100% attribute to the change in clock speed.
My main complaint is with the Asus BIOS its complicated to say the least since I am more used to HP and Dell bios from older systems which tend to me less complicated.
Still learning about XMP, CPU curve etc…
The load optimal factory defaults was my goto hack back in the day and even to this day.
But this does not do the job…
The memory training is a bit of a pain each time you make a small change it takes ages to training …
These are essentially minor issues…
For now planning to run rook/ceph on K8s… but eventually the plan is to setup dedicated Ceph deployment possibly on spinning rust…
I will go through the mega thread thanks for that. Just too much to consider when I was planning to purchase. So I when I found a decent MOBO I decided to bite the bullet.
128GB has limited speed on AM5, as does any ddr5 with 2 dimms per channel. It is very plausible that 4200 is the highest stable speed (possibly plus some margin by the OEM). Your issues are probably caused by the memory OC. Remember, for 2 dimms per channel anything over 3600 is an OC according to AMD spec…
No. Core clocks are decoupled from FCLK and FCLK’s decoupled from MCLK and UCLK. At DDR5-6200+ there seem to be benefits to lifting FCLK with MCLK but, as differences are often within test to test variability, this gets hotly debated at times.
Some builds get stuck at 4200 or even 3600 but 4800+ quad DIMM’s commonly achievable with Raphael. So the builder’s pretty safe committing to 4200 (presumably if they couldn’t hit it they’d move the parts to somebody’s two DIMM build) and could likely go higher. My main question here’s what’s the part number on the Kingston sticks.
Edit since I forgot this bit: yes, EXPO. EXPO and XMP are slightly different versions of much same thing and every recent AMD board I’ve used will apply either. However, a kit that doesn’t have EXPO is unlikely to formally support AMD. Some XMPs won’t post on Zen and then you have to pull two DIMMs out of a quad DIMM setup or reset to get back into the BIOS.
Also, consider a comprehensive initial assessment of a memory overlock’s stability takes around 24 hours and it can be optimistic to really call it before a month or two. Memory training’s trivial in comparison.
I’m not sure what you’re trying to achieve, you will almost for sure (like 99.98%) not be able to run 4 DIMMs at anything near 5600 which you’ve been told multiple times by now and it’s not something that’s wrong/broken etc.
Several others have mentioned your choices for memory. I am using a 96gb 5600 kit from GSkill and it has been working well for me. I have the same CPU and an MSI X670E MAG Tomahawk WiFi. I have not noticed any issues and had no issues with getting it setup. The timings are CL40-40-40-89 @ 1.25v. It is model F5-5600J4040D48GX2-TZ5NR. I spent several days looking over memory options for my system. I decided it would be best to not use 4 sticks and stick to 2 just to keep things simple. This kit was the best i found. I wanted 128gb as well but I changed my mind after reading the megathread, the one mentioned by @jode and reading some other thing on AM5 and memory.
I’m running a 7950X non-3D w/ 64GB DDR5-6000 on an MSI PRO X670 mobo. It’s working great. As others have noted, 128GB RAM w/ AM5 is not really recommended: 1DPC seems to be the sweet spot.
You’d be better off with an X870/E board AIUI, although plenty of reviewers say that other than USB4 it’s the same as X670/E.
Personally, I stick with 1DPC on AM5 for now, which means 96GB max, or alternatively don’t stray outside of JEDEC recommendations.
ETA: My RAM is G.Skill Trident Z5 NEO CL30-40-40-96.
Wendell for one. 128GB (32GB x 4) is OK on AM5 if you’re running at lower speeds and higher latency. Admittedly that info was from a year or so ago. AFAIA 128GB EXPO DDR5 on AM5 is still not as stable as 64GB e.g. (2DPC vs 1)
have some new information and thought I would share. I just put together a B650, 9700X system for a friend. We used a Micro Center combo, 9700X Combo. I had an identical memory kit left over after my upgrade to 96gb and decided to try 4 sticks. There have been no issues running at 6000. I had my doubts based on what I had seen and read, but I guess improvements have been made. My friend also gets the benefit of having 64gb of memory. If anything changes I will be sure to update.
