Hi,
I’m wrapping up a house build, which is a great excuse to finally upgrade my PC and perhaps do something with my old hardware 
I’ve been out of the PC building game for the past 6 years, so I thought I’d ask for tips before I blow a couple grand on new hardware, just in case I’m missing some important detail.
I have a “server room” that unfortunately does not have much space for a server rack. My compromise is to build a custom pull-out 42U 19in rack “cabinet” that can support up to 450mm deep devices with some room in the front / back for cables. This, unfortunately, means that nearly all of the OOTB rack solutions are too deep for this setup. All that gets connected to a HRV system etc. - I plan to build the whole thing myself, I’ve been really into both woodworking and automotive parts fabrication over the past few years and I enjoy the challenge I have already found a local rack case fabricator that can make me a 400mm deep 4U blank for cheap + lots of machines to do further fab work.
My current PC runs a i7 5960X (mildly OC’d right now) on an ASUS Rampage V Extreme (X99) with a GTX 1080, both running a custom watercooling loop. It’s a toaster with 40 PCIE3.0 lanes. While the heat recovery system will at least let me recover some of that wasted electricity, I’d rather rely on a power-efficient heat pump to do the heating instead. So while the GTX1080 might provide some value running ML models in a headless setup, the ROI on replacing the 2015 8-core CPU is really making me consider buying a brand new CPU for the home server. I’ll reuse the watercooling gear to keep everything compact.
I plan to run the following services there:
- Frigate NVR with ~14 cameras in the long run, running a ML object detection model on a Coral Edge Dual TPU probably mounted on a M.2 adapter, so the setup needs to support bifurcation quite well.
- a bunch of smart home adjacent software - I have a couple Pis running my “wired smart home” logic, so I’d imagine I would host stuff like Linphone, Grafana/Prometheus for monitoring etc. - overall pretty lightweight
- Plex / Jellyfin supporting 4K transcoding
- some NAS software (ZFS on Linux, Samba?)
- possibly some game servers for the kids
- some VMs to eg. offload 3D scanning compute to the server with USB over IP (lots of RAM), run sandboxed software with GPU passthrough etc.
I will probably roll it out on a custom OS with RAID1 for the main drive, possibly dedicated SSDs for some VMs. A couple large SATA HDDs for media storage (JBOD?).
So I was planning to optimize for the following things:
- Reasonable power usage under low to medium loads. 8+ reasonably fast cores.
- Probably maxed out RAM, 128GB? ECC would be ideal.
- Decent support for custom watercooling parts, though I can adapt.
- Good Linux support for all the hardware involved incl. decent IOMMU grouping
- A couple PCI-E slots for reasonable GPU passthrough to VMs (ideally 2 GPUs if I want to run non-TFLite ML models in the home automation stack?). Bifurcation on at least one M.2 slot.
- 10Gbit networking over either SFP+ or Ethernet. This gets complicated with the previous point, unless I get some sort of a M.2 to PCI-E converter. And even then, how many disks would I even get to use???
- Reasonable motherboard form factor. ATX preferred over E-ATX.
- Reasonable price HEDT is too expensive in this gen.
- No need for IPMI, I have a PiKVM ready to integrate into the build.
I’ve been eyeing AM4 and AM5. ASUS and ASRock seem to be the most reasonable for ECC support. I’ve had really good experiences with both. Going for the previous generation is always great for discounted watercooling blocks. Given that I’m planning to go with up to 3 GPUs (which can be thin, the GTX1080 is a single slot), the only AM5 option I could find is ROG STRIX X670E-E GAMING that costs $550 here in Central Europe. Not ideal. There’s also MSI MEG 670E ACE that’s even more expensive and, from what I’ve read, has dubious ECC support.
Should I instead look for EPYC? Or are there any AM5 products I’ve missed? Perhaps there’s something available for AM4 that would satisfy my requirements? What if Intel is the answer given the iGPU (dropping the need for a third GPU) and QuickSync support? Or should I build 2 mid-spec servers instead?
I’m willing to drop my requirements to shitty host GPU + passthrough GTX1080 and run the 3D scanning offloading on my workstation if there are indeed no better options. I’d rather have a dedicated machine, though, just in case someone else wants to do large automotive scans in my workshop while I’m working in my office.
Looking for hardware suggestions