Recently there’s a flood of QS EPYC 9654s available on ebay: ebay 9654 QS
I’m thinking of building an HPC workstation. I already have a Fractal Torrent case, nvme drives and a 1200W PSU.
Maybe start with 12x 32GB DDR5 RDIMM.
Would need to pick up a MB as well and thinking about Gigabyte MZ33-AR1 for Turin compatibility when the 9655’s hits ebay in 18 months.
My main use case is virtual Win 10 machines running under Ubuntu 24.04. I have a Looking Glass setup with a RTX4090 and a 7950X3D that works just fine, the only problem being I can’t remote into my individual Win10 machines this way. I also just need more cores.
Does this sound reasonable or should I stay away from the QS versions? I can’t justify paying USD10K for a CPU.
I went ahead and ordered the MZ33-AR1, EPYC 9654 QS and 12x 32GB 4800MHz DDR5 ECC. I requested a quote for everything from an ebay seller with good reputation.
I’ll read up on the cooling and make sure to get a TPM as well. @JayVenturi Did you find a more reliable source for coolers?
Absolutely
The Gigabyte board is great for compute nodes
Huge RAM, super fast networking, and or PCIe interface cards to GPU node.
The SuperMicro is the way to go for Full size GPU acceleration on the board.
You’ll lose alot of slots with the card overhang, but PCIe 5.0 risers are not advised.
On the gigabyte board, you cannot have a full length PCIe card without interference with the RAM DIMMs. EPYC requires 1 DIMM per memory channel for optimal performance, but cards will actually interfere with the latches of empty slots.
But that is only 4.0 Speed, and x8, plus you need a place to put the GPUs
or get a kit like this
Which is 5.0 but significantly more expensive
But like TryTwiceMedia said here
if you want a no faff approach, the Supermicro motherboard is pretty good, for up to 3 5090s ( assuming they can all get fresh air lmao)
The Gigabyte one is good if you want 24 Dimms and 12/14 NVMES straight away
IMO it is the superior motherboard, but it needs alot of adapters to configure it the way you want it (multiple GPUs), but with a bigger/custom enclousure they will be well cooled
e.g. something like this
Hopefully that helps,
Personally im still deciding between going
W880 and/or ZEN5 or Turin via H13SSL or MZ33-AR0
Im leaning more towards W880 for ST(especially because do i really need more than 256GB RAM , and its more of an AIO Home server build than a HPC build) but Genoa / Turin is especially the 9115 CPU looks really good.
I think it just ultimately comes down to RAM + CPU cost, DDR5 RDIMMS are expensive atm, but that will lower with time
For a single GPU I could have it in the third pcie slot, that would clear the RAM.
What is that huge cutout for though?
Edit: That would be an OCP slot which I gleaned from reading the manual. Is there a way to mount nvme drives there?
That’s a nice motherboard, it looks like it uses pretty much all of the 128 5.0 lanes genoa/Turin has
I think it depends on the revision of the motherboard you get, the supermicro one, you need to get the latest revision
I think it’s the same with the Gigabyte Rev 3.0 iirc
That’s what I was wondering/thinking
I haven’t found pics of the IO shield
but all it would do mechanically is retain the card in 1 axis
The OCP card itself is supported by the mounting rails on the board in the other 2 axis
To elaborate, I’ve built and deployed half a dozen EPYC 9004’s in the past year. Never seriously considered that board but have used ASRock Rack, Gigabyte, and Supermicro.
My preference is the reverse of alphabetical, but it’s all application dependent.