Hey folks, I am finally about to upgrade my old home server that I built in 2016. After a lot of thread sniffing and research, I decided on building around the Asrock rack B650D4U-2L2T, as I wanted a few things:
DDR5
ECC
AMD
mATX
10G+ LAN or free PCIe slot for an adapter
My use-cases is proxmox running web services, data hosting services, media streaming, AI experiments and ripping blurays. With plenty of storage for redundancy through RAIDZ1/2 configs.
I went with AMD 9950X as the CPU and a couple of compatible 48GB ECC UDIMM sticks. However, as it arrived in the mail, a couple caveats made me rethink, the mobo has a huge lack of PCIe slots (I was planning to put in my old GPU in it for lite AI experiments and transcoding) and only one m2 slot.
Now after a good amount of searching I did find that in the short term I can fit my needs with PCIe adapters (specifically adding 2 extra NVMes), but it feels like a disappointing upgrade with the lack of extensibility (e.g. no free PCIe slots for HBA/SATA expansion).
I threw a quick build on paper around the EPYC 8004 platform and saw that with only about $500 more I would get:
MOBO: ASRock Rack SIENAD8UD-2L2Q
CPU: EPYC 8124P
RAM: Same 2x48GB config as before but RDIMM instead
The motherboard upgrade alone brings:
An extra PCIe gen 5 x16 slot
access to RDIMM ECC (which people seem to report is better supported)
an extra m2 NVMe slot
loads more expandability (more RAM slots, 2x25Gig LAN, etc)
The latter upgrade feels good in every way, and I’m fortunate to have the means to afford it. But the issue is that my end game was always to have 2+ system and I’m worried if I’m missing out by settling for an mATX EPYC motherboard that costs $$$. Are there also caveats around going EPYC 8004 this build? I see people selling 8124P for $800 on ebay, and in comparison, my 9950X build was all with new parts.
Would love to get feedback on my build, sorry I kind of frantically wrote this up, happy to answer any questions or provide more details. Thanks!
EPYC CPUs have slower single core performance compared to Ryzen/Threadripper.
OTOH they do have superior IO and memory performance. To take advantage of the higher memory bandwidth you need to use more RAM sticks, ideally 6 to fully utilize the 6-channel memory configuration (or - at least use 4x32GB modules, which should be cheaper per stick than 2x48).
The wild card to watch out for is power consumption which is of importance to most home labbers.
Both configs use server class mobos with IPMI chips which consume power, the CPUs are probably pretty similar, RAM sticks consume ~5W each.
It’s a big compromise and only you can decide what config is better suited to support your workloads (which you have not shared in this thread).
Btw, what motivates you to upgrade your existing hw (other than age)?
I think the better question to ask yourself is: Do i need to have exactly one do it all server, or is possibly multiple smaller servers fill my needs just fine?
For instance to do ai experimentation assuming your not running ai on cpu, any old desktop with a fast gpu and nvme storage will do. Does it need to be stuffed into your home nas?
Same goes for your storage, does your nas need to be in your virtualizatoin platform?
If you got the Ryzen 9950X because you need its performance then an EPYC 8124P obviously won’t do it for you since it’s about half the performance, both single thread and multithread.
I don’t need the single core performance, frankly 9950x is overkill for what I need but I wanted at least 16 cores and this was the cheapest as far as I could tell to get AMD, ECC, and DDR5. 7950x was considered but the price difference was negligible as I got the 9950x on sale. I do like the potential of upgrading and reusing RAM modules, I like that about the EPYC route with ECC RDIMMs. I don’t mind potentially underutilizing the memory bandwidth in the meantime
After reading people’s two cents on this, IPMI feels like a good investment even with the additional power-draw. I dont have a crazy homelab setup so I’m okay with this compromise
Sorry for not sharing the expected workloads for this, I wasn’t sure how to frame it to draw up its relevance. I was trying to understand in what cases one would pick a similar core count between Ryzen 9000 and EPYC 8004 as i’m having a hard time understanding the trade offs. I would like for this to handle media streaming for 2-4 simulatneous streams, and NAS share for a couple different systems at a time. I expect between 10-20 virtual machines to run regularly, and would want to have headroom to run more for playing around. It’s hard (like it always is) to predict my workload even over the next year, but I dont mind spending extra money now than have to feel limited in the things I can do. My budget is more time-bound than money-bound.
I am running hot on memory usage and don’t want to continue investing in DDR4 Non-ECC. I have services running on bare metal right now and wanted to do the software (proxmox + ZFS) and hardware update in one go. After contemplating this for a couple years this is where I landed, since I want to land on this upgrade for a few years.
Great questions. My plan is to eventually add more servers, but I don’t live in a house so size and noise are both big factors. In a few years when it might not be, I plan to revisit my needs and fit them into a proper rack-mounted setup in a garage/closet where noise might not be as big of an issue. For now, I want to reuse my existing Node 804 case, and that’s why I wanted to go with AM5 now and proper servergrade motherboard and CPU down the line.
had some more time to think about this. This so far seems like my tradeoffs
Pros of SP6 route over AM5:
Additional M2 slot
Additional PCIe Gen 5 x16 slot
DDR5 ECC RDIMM is better supported
Future Pros (not immediately actionable)
25Gig LAN
Space for two GPUs OR two x8/x16 HBAs or one of each
RAM can be reused in other future EPYC builds
Cons of SP6 route:
$1830 price tag for chosen parts (vs $1540 for AM5)
Need to deal with 3rd party sellers for getting 8124P
Half the single core performance of 9950X (although don’t have any workloads planned right now that would require it on the daily)
I ended up holding off on the build and ordering the parts for the EPYC route since I was able to find better prices for both the CPU and RAM sticks (3x32GB refurbished). The added flexibility and expand-ability in the near future was too much to give up, and the price difference didn’t end up amounting to much. It really came down to the motherboard. The Asrock Rack B650D4U-2L2T has terrible layout, the ASrock rack SIENAD8UD-2L2Q feels very much worth the price premium.