Currently I am running a Supermicro a1sam-2750f with 32 GB ram. It is very old now and starting to get long in the teeth. I am using it to run Xenserver hypervisor. Currently I am running several VM’s that I use all the time. I have a nextcloud server, and a couple of ubuntu VM’s that I use for various tasks, a backup to PCloud, xen orchestra and a locally hosted website. Everything has started to get sluggish and it is time for an upgrade.
Here are some of my constraints.
cost = $3,000
desktop not server case
VERY future proof
ECC ram
perhaps not a supermicro motherboard.
IPMI
I was actually considering the AMD threadripper pro 3955WX with the Asus motherboard. I haven’t decided on the ram but perhaps would start out with 128 GB but filling half the slots so I could double the number in the future. I realize this is vastly more powerful than what I have but I really enjoy the hobby. I suppose I am just thinking it might be way over kill. I am not up for simply wasting money, but the cost isn’t that big of a deal. I figure I will have the machine for quite a while. I currently really like Xen but I could easily be swayed to try something else in the future and want ECC ram incase TrueNAS or something else comes into play.
There are a few ways to go here. I’m thinking 16 cores/32 threads should be more than enough to bifurb into 4 virtual machines + hypervisor, YMMV.
Ryzen 5950X, 128 GB ECC RAM, X570 Motherboard - 16 cores, 32 threads. Should last you for a while, the question is if you can live with the limitations of the X570 chipset. Apart from that you can easily put in 128 GB RAM ECC memory, which should last you for a while. This will draw much less power than the other options, which is also something to consider. This would cost ~$1900 for RAM, Mobo and CPU.
Threadripper, yeah. It’s awesome, and expensive. $3000 is the bare minimum, and the question is do you really need that much power?
Xeon, Intel server CPUs are still competitive especially at a price/performance ratio. However, again this is starting to get really expensive.
Epyc, best of the best and top tier, but probably outside your price range.
It is possible to find deals, but I’d say even the Ryzen option would easily power all your stuff for at least 5 years. The only question is if you have a separate NAS, and if you have special requirements like an mITX or mATX case.
The Ryzen has one drawback, and that is that it is not made to be powered on 24/7. This may or may not be a problem in practice. If you have a small case, Xeons are your second best bet I think.
I don’t know if anything you can buy today can be said to be very future proof today. Alderlake’s teething issues, AM5 on the horizon, things happening with operating systems schedulers causing performance issues and what not. I’m nit picking at that one requirement but its so bloody hard to figure out what is future proof right now.
So, I’m going to suggest this. You change your future proofness requirement to least likely to make you frustrated with its peculiarities. I got a 5900x in July and its been so fast but such a little bastard to keep cool and quiet with a spiffy all in one water cooler that I’m regretting a little bit of my choices. Quiet I should add is a relative term, my idea of quiet is not your idea of quiet. I got this system to be a 5-10 year if I have to build, future proof I said to myself, but I don’t think I’ll last more than a year or so before I replace the whole lot with whatever is half the power and roughly equal in performance.
Also something to consider is power draw and/or noise. My server (Ryzen+128GB ECC+PCIe stuff+9 drives) is running at 90-100W at low load, 160W all guns blazing. Desktop NAS case.
If you go Xeons/Threadripper/EPYC and 19" stuff, expect a multitude of that. But yeah…(dual)EPYC Rome/Milan board with 32+ cores and shitton of PCIe 4.0 lanes, that’s probably as future proof as one could get right now. I just didn’t want to pay additional 2000€ in energy bills over its lifetime.
All great suggestions. I guess noise isn’t an issue since the machine will be in the basement. I have 2 x 3090s down there mining ethereum and they aren’t quiet. I guess the power usage isn’t small either. For the Threadripper Pro is an AIO cooler adequate. I would assume they are available. I haven’t looked.
Im running a TR PRO 3945 and NVIDIA A100 for my homelab and the cpus nice since it forfills all my needs and alot more, I will be upgrading to the 5xxx chips when i can but honestly if you need PCIe Lanes/RAM TR Pro is a pretty nice way togo.
Yeah, Ryzen has very limited PCIe lanes => limited in terms of PCIe extensions, especially compared to Threadripper (Ryzen: 24, Threadripper: 64, Threadripper Pro: 128).
I wouldn’t mind a board with at least six x4 m.2 ports and two x16 PCIe ports, driven by 64 lanes in higher end Ryzen products and bifurbed in lower end products, maybe even an AM5/AM5+ deal. Let all expansion ports such as SATA and extra fan headers become m.2 addons, which solves a lot of legacy cruft (Need SATA? m.2. Need USB 2.0 headers? m.2. Need A97 Audio header? m.2. etc) but now we’re in dreamland territory, although such a board would be great inside a standard console layout.
For your use case though, it sounds as if breakout boards and PCIe lanes aren’t really a concern, which is why I would suggest a 5950X paired with a B550 Aorus Master - cheap, excellent VRMs, silent but still has room for a 10 Gb Ethernet card, and just gets the job done. However, other options exist, too.
Do make sure to read this recent thread by @rehabilitated_it_guy, it contains a lot of good advice for why you probably will not need to go completely off the rails with your homelab, even though a NAS, a commercial grade firewall and a virtual server setup does make sense, it doesn’t have to be the best of the best of the best:
In what sense specifically, what would you expect would go wrong?
Re IPMI… there’s a thing called PiKVM that allows you to capture video, send keyboard events, emulate USB storage, toggle power reset buttons and so on.
It might be an interesting thing to look at, as it would open up your choice of motherboards… regardless of cost effectiveness.
As in, it is a consumer product designed to be used a few hours each day, not stand unattended day and night. This should not have any large impacts, but one of my friends had trouble with his 4700H PN51. That’s embedded Ryzen though, so a different beast.
Running any kind of machinery outside it’s intended purpose may have unintended side effects, but it’s like OC - Do it if you want to, just don’t expect the OC chip to behave according to spec.
https://pcpartpicker.com/list/9RdqsL
This … Threadripper 2950x - 128GB ECC - tower case plus an asrock TRX40D8-2N2T will fit into your budget.
Future proofing can have multiple meanings … we are at the end of the cycle for the TRX4/AM4/DDR4 platforms and waiting for the new generation of chips/sockets to be released. Given the way pricing from AMD is going I would expect this new generation to be even more pricier than usual so I would not count on being able to fit that platform into a 3K budget for the next couple of years at least.
If by future proofing you instead mean the ability to support 2x or 3x times your current workload then definitely, and without breaking a sweat. The threadripper platform provides a lot more expandability versus the Ryzen one, you pay that with more power consumed and no options to underclock at the same levels, so that is something that only you can evaluate …
Sorry, but have to call BS on this one … unless you can point us to a very detailed and statistiaclly acceptable analisys of this … any consumer processor platform since the early 2000s can work 24/7 with the only caveat of the cooling system being operational/properly maintained. It is in fact much more stressful for the system to be regularly powered on/off than to be kept powered and runnning 24/7 …
The challenge with that nowadays is power consumption, especially of discrete graphics cards but nonetheless …
