Homeserver hardware considerations

I currently dual boot proxmox on my desktop pc for tinkering and general homelab stuff, but I’m thinking about upgrading to a proper homeserver, with permanently deployed VMs and containers, that can double as a homelab environment.

I’m looking to run:
Proxmox on the baremetal homeserver on 2 SSDs in raid1, VMs and containers will also be stored on the SSDs.

VM running pfsense or vyos or another software router/fw. This will be the router and fw for my physical home network, as well as the virtualized VMs and containers, so a 10GBit port passed through to the VM will be connected to a switch for my physical devices, and a 1GBit port passed through to the VM will be connected to my ISPs router (running in bridge mode), or at least that’s the plan.

VM running truenas with 3-4x 8-10TB HDDs passed through to the VM for a VDEV in a ZFS pool (possibly expanded by another 4 HDD vdev in the future).

Container running jellyfin with access to the media files on truenas, hardware acceleration active for decoding/encoding/transcoding via the iGPU or a dedicated GPU.

Container running pi-hole, for network wide ad blocking.

Container running tailscale, to access the homeserver from outside the network.

Maybe 1 or 2 more VMs and maybe a few containers for tinkering/homelab stuff, nothing permanent, nothing running 24/7.

Backups will probably only be done for the environment itself (proxmox + VMs) and sensitive data like family media and documents via external HDDs once a week, I probably won’t bother backing up all media files.

Considerations:
Power is quite expensive over here in Europe, so low idle power consumption would be great.

Noise levels should be kept to a minimum, the server will either sit in my living room or in a closet/storage room, so ideally close to silent or at least quiet.

I could make room for a larger machine, but ideally less than a desktop big tower in size, smaller is better I suppose. Small pc (minisforum?) without any HDD space + external usb hard drive enclosure might also be an option.

“Enterprise grade” vs “consumer grade” hardware, where does it make sense to save money and where do I really want the good stuff? With a hypervisor and several VMs running 24/7 I’m worried about killing consumer grade m.2 SSDs in a few years. I’m assuming enterprise grade HDDs also make sense in this case, or will I be fine with “prosumer” nas drives? SATA vs SAS HDDs? Should I get a platform with ECC memory?

CPU and memory, 8 cores at a minimum, ideally 16+, 64GB, 96GB or 128GB?

Network switch, what are my options here for a decent, ideally not too expensive, VLAN capable managed switch? PoE is not required at the moment, but could very well be an advantage in the future with PoE access points. 10GBit uplink and ideally 5GBit for some or all ports, at least 2.5GBit, I don’t really want to be stuck with gigabit in the future.

Access points that are vlan capable, multi SSID (so VLANs can be mapped to specific SSIDs), wifi6 should be enough, but possibly 1 wifi 6e or wifi 7 AP for wireless VR.

Is a modern iGPU, like the Radeon 780m on APUs or mobile ryzen CPUs, good enough for jellyfin? Or do I want a dedicated GPU for that task? Ideally I want to be able to stream 4k HDR bluray rips from my server with possibly 2-4 more concurrent 1080p or 4k streams going at the same time.

What I’ve looked at so far:
Building my own system, either with ryzen or ryzen pro, miniITX or mATX motherboard, in a case like the Fractal Design Node 804 or Node 304, or Jonsbo N3 or N4.

Minisforum, or another company’s, minipc with a ryzen mobile CPU, afaik there are also pro models available for ECC support, + external drive enclosure for the harddrives.

Any help on the topic is appreciated, I think I have fairly well defined goals of what I want the server to be able to achieve, and I want to get something decent from the start and not start out with a cheap solution that I’ll have to upgrade part by part over the next years.
If any of my plans for this server and home network seem odd or seem like a bad idea in general, feel free to tell me as well.
And finally, if I’m missing a killer app that I could be running in a container or as a vm, that you would definitely recommend in any home environment, I’m also more than open for any suggestions. Maybe a central backup server or something like that?

You got a lot of boxes to tick, ngl. If I were to put myself in your shoes, I’d probably give aliexpress laptop cpu motherboards a try. Like this one. I have no idea how reliable they are, but it’s a laptop cpu, with a lot of storage, a laptop cpu but more like the ones they put in gaming laptops, so you get a bit of juice but also low power draw in idle.

Case and cooling might be questionable to source, but I’d start around aliexpress.

My overall advice is: Get the cheapest thing that can get the job done. If it breaks with normal usage patterns, you probably deserve the more expensive options.

just like previous user said, probably something based on mobo+cpu is good enough at least to get feet wet and see if you really need more

This should also be a good and cheaper option and it ships from france (within europe). it has fewer ethernet ports but you can get a cheap switch

I’ve seen those hacked together boards, but I honestly don’t trust them. At that point I’d rather just live with higher power consumption and build something new with standard desktop parts, or get a mini PC with laptop parts and an external HDD enclosure for the drives.
Even if this whole project will start out as a lot of tinkering, I do intend on ultimately using the thing at home and for remote family members as a backup target, so I don’t want to mess around with parts that are possibly more prone to failure for no real benefit.

that’s fair

if you go with older parts you can get most hardware for basically the same price (b650 matx mobo + ryzen 5700g + basic ethernet pcie card)

I did some digging, and apparently the ASRock DeskMeet X300 (AM4) or X600 (AM5) has very low idle power consumption, fits a Ryzen 5/7 APU and supports ECC memory (with the right CPU).
It also has 1 Pcie slot, so in the future I could add a network card (if I want to go the VM router/fw route) or an HBA card (possibly to add an external HDD cage with up to 4 HDDs for truenas).
Obviously it doesn’t tick all the boxes, but it might be a way to get started cheaper (as you guys suggested), expand to ~80% of my original requirements and then, maybe in a few years, build it new to meet 100% of my needs.

I’ll recommend going with ECC because it’s not big price difference. Consumer AMD platforms usually support ECC UDIMM quite well and cost difference is fairly insignificant. What you’ll trade off is RAM frequency because ECC sticks are not ultra-overclocked gaming rgb stuff, just JEDEC spec but typically server use cases don’t benefit much from ultra high clock RAM. And your ZFS will be happy about ECC.

You won’t kill consumer SSDs. I’m using consumer SSDs in database servers that have so far like 1200 TBw or more and it’s fine. Seagate FireCuda 530 are really good consumer drives for write intesive workstation/server use cases if that’s what you care about. But it may be challenging to find them as they’re no longer made afaik. We also use them in one server on production that is hosting automation software and uses elasticsearch database under the hood. It works just fine.

VERY IMPORTANT NOTE : get UPS. consumer drives don’t have BBU’s / power loss protection and ZFS will be unhappy if your machine goes down due to power loss. On the other hand - I also use enterprise Kingston DC600M that DO have power loss protection and I still lost data on them due to power loss so I call it bs. Any SSD based server needs UPS and enterprise SSDs won’t save you there. Just get UPS please…

And speaking of UPSes… Don’t cheap out on those. I use APC. It’s safe bet. I used to think hurr durr, overpriced crap and bought cheap UPS. 1 year later it literally EXPLODED - started smoking and I had to take it out of my apartment in an emergency. Luckily I was at home at that time. I don’t know what would happen if I wasn’t. UPS’es are very dangerous devices if you cheap out on them. Kinda like Chinese electric scooters. You don’t want cheap one. Get SMALLER one, but not cheaper one of the same capacity.

Another case where enterprise stuff makes sense is networking gear. I have bunch of used x710 Intel cards from ebay for pennies. They support SR-IOV so you won’t need to pass-through whole network card, just single virtual function and it will work well.

MikroTik. Cheap and they’re even cheap at 10G. I’ve got dozen of CRS317 16x10G switches but they also offer 8x10G or even 4x10G for really low price. They’re very advanced devices that will also be capable of simple routing and many other things. They’re unproportionally cheap to what they offer. They’re absolutely killing in value department. They also have 2.5G options.

If you want to go for cursed hardware - MikroTik CCR2004-pcie. It’s two-in-one router+25/10G network card. It has full router software stack, capable of firewall etc etc, yet it’s in pci-e form factor. You log-in to it over SSH like to regular router and it is in fact regular router that has 7 ports - 4 virtual that are exposed to host PC over pci-e and 3 physical ones. And then you set up real networking inside this card like it would be just normal router. In simplest config (that is default config) two first pci-e ports are bridged to hardware 10G ports and two latter pci-e ports are bridged to card management interface and ethernet 1G port on card. Really neat and one-of-a-kind stuff.

It’s actually quite cheap even comparing to “normal” dumb 10G network cards. I’m using it in VM host.

Thank you so much for the info in regards to consumer SSDs, managed switches, network cards, and I didn’t even think about getting a UPS, since the last power outage I can remember is from my childhood.
I just went down quite a rabbit hole when it comes to UPSes, line-interactive vs standby, how to automatically shutdown proxmox if the UPS is tripped and below a certain battery percentage, and the ever important low idle power consumption. Apparently CyberPower UPSes offers the lowest idle power consumption on their small models (3W-4W vs 15W-20W for the cheap ACP units). I read negative reviews about their “value pro” series, but the more expensive pfc sinewave series has much better reviews, the model I was looking at is: CP900EPFCLCD

I’ll still mull it over, but the cheapest way to get started would probably be:
190€ asrock deskmeet x300 barebones
140€ ryzen 5 pro 5655g
FREE 2x16GB DDR4 (non-ecc) that I have laying around
FREE 2TB samsung evo 970 plus that I can cannibalize from my desktop (this is where proxmox is currently installed)
FREE 2x2TB hard drives (old and probably dying, but I can still test the whole setup in truenas and play around)
FREE 2TB external HDD for backups

That should be close enough to the real thing to play around and experiment, I’ll also be able to see if I actually need 64GB of memory or if 32GB is enough.
Then I can upgrade to ECC RAM, bigger hard drives and a UPS down the line.
I’ll probably have to check again, once I upgrade to larger HDDs, but seagate exos x16 seem affordable right now at 270€ for a 16TB drive. Model number ST16000NM001G, it seems they also have good results with a large sample size in the backblaze failure rate reports.

1 Like