After a failed attempt to get a second-hand HP DL380 server from an auction I decided to go fully overboard and build a new server from scratch.
This is for a home environment, so sound is a concern and don’t have a server rack (yet), so going for a tower build for now. I currently use an old PC (Intel Core i7-4790 with 32GB RAM), but that one is now struggling to keep up, so time for an upgrade.
Requirements:
- I’m running 15-25 VMs for various tasks where Elastic Stack and Zabbix eat most resources.
- Storage is on a local NAS, but that is not fast enough for the VMs, so local storage for the VMs is required (re-use the current NVMe SSD?)
- 10GbE to play around with and have room for growth.
- I will be switching from Hyper-V to Proxmox for virtualization.
I’ve investigated a few options now:
- AMD Threadripper - Best overall so far.
- AMD Epyc - Difficult to get 32 core Zen 2 for reasonable price here.
- AMD Threadripper Pro - Bit above budget and don’t need the extra PCIe lanes or performance.
- Intel Dual Xeon - Need 2 CPUs to make it viable. AMD seems better bang for buck.
Specs I’m looking at:
- 32 Core CPU
- 128 GB RAM
- Tower format
- 2 NVMe drives for storage?
So the build I am thinking about now is:
Processor: Threadripper 3970X
Motherboard: Asrock Rack TRX40D8-2N2T - Best fitting server board I could find.
Memory: 4x Kingston KSM32ED8/32ME 3200 DDR ECC - 4 32GB modules to use the quad channel and leave space to expand later if needed.
SSDs: XPG GAMMIX S11 Pro - Already have one, this has high TBW for cheap SSDs (1280TB on 2TB) which is why I bought it. Might get a second one later if I need more storage.
Case: Lian Li Lancool II Mesh Performance - Good airflow case with plenty of room
Cooler: Cooler Master MasterLiquid ML240 RGB TR4 Edition - Largest AIO that fits in the top (don’t want to obstruct the front for airflow if possible).
Considerations for this build:
- Threadripper Pro would be nice, but is around €700 more expensive for a similar build.
- Since noise is my enemy; I’d like to ensure sufficient cooling, therefore going for an AIO. (Also I’m moving soon and may need to ship the server, so this will be less fragile while moving).
- I don’t need a lot of PCIe lanes if I have onboard LAN and storage controllers.
- I don’t need a graphics card, so a BMC controller would be nice.
Overall this will set me back €3.411,32 here in the Netherlands (including taxes)
Questions I have:
- What do you think of the setup? Would you do anything differently?
- The Asrock Rack TRX40D8-2N2T connects the NVMe and LAN through the chipset instead of directly to the processor through PCIe. Is this a big issue? I can always add a PCIe 4.0 to M.2 card or 10GbE PCIe card later if needed, right? Or are there better alternatives to this motherboard?
- Is a 240mm AIO at the top (exhaust) with 3x 140mm fans in the front (intake) and 1x 120mm fan in the back (exhaust) sufficient cooling for this setup? Normally I think it should be, but all non-server motherboards have active chipset/VRM cooling, which worries me a bit.
- The memory is not explicitly on the Memory QVL list. There are very few on there and no 32Gb 3200 modules. I believe these modules should be ok since this is a dual rank module of a 16GB single rank module which is on the list. Is this the best chance I have to get a compatible module or are there other things to look at?