I have been researching this topic for weeks now, but I seem to have come full circle at this point. I need help.
My end goal:
To have a home lab that is capable of doing anything.
What I currently have:
16" MacBook Pro with 64GB RAM & Core i9.
What I want to add:
A Type 1 Hypervisor
Likely a NetGate SG-3100/5100 directly after my Verizon Fios ONT (Speeds average around 800 Mbps) for a permanent security appliance/sensor.
Why not a hosted hypervisor:
I don’t want to sell my MacBook, and with a hosted hypervisor, I don’t know that it would be used at home anymore.
Other typical reasons e.g. OS overhead.
So, where I am at now, and how I have come full circle:
Started looking at a type 1 3970x threadripper build
Decided against it due to compatibility worries/issues.
Started looking at just doing a hosted build
Came to the same realization that my MacBook would not see any use, and it’s really not needed.
A type 1 would be simpler for my wife to pull resources to her MacBook.
…I just want it.
Started looking at a white box build…then stopped looking at them
There are no decent motherboards for 2nd Gen EPYC processors yet
I don’t want a system that is always on / I don’t want a loud server in our office.
Now I’m back to a desktop type 1 system…maybe the 3970x?
Like I first said…I need help. I know what I want, and I know what I don’t want, but I can’t seem to find any solid compatibility info for stuff other than the commercial Dell/HP offerings.
As for what I need ( …want* ) in my system:
~32 Total Cores. I’m not against building multiple machines if it is more cost effective or easier to configure, I just figured a single threadripper would be simpler all around.
256GB RAM
10Gbps RJ45 x 2-3
The three things I am still clueless on are GPU requirements, I/O, and which flavor of hypervisor I should choose.
I want one VM to have a NVIDIA 2080 passed through, not for gaming, but for calculation stuff. Can I get by with one card, or will every VM need a basic GPU to function?
I assumed that I could run the VM’s from my MacBook, and it would link that I/O. I see a bunch of posts about USB passthrough and having to utilize I/O switches when setting things up…I would like to avoid all of that. Not sure if those are atypical situations I’ve read about or not.
Lastly, I initially was going to go with the standard ESXI, but I know there are a ton of choices out there. Can anyone shed some light on which choice may suit me best? Hyper-V is out because I want to be able to run any OS.
I’m not ashamed to say that I had to google what headless mode was…but yeah, it will be a mixture of the two. PfSense, lightweight Linux distros, Ubuntu, Windows 10, windows/Linux servers etc.
Most VM’s will be simple gui’s or CLI, but I’ll probably load up any open PCIe slots with some basic cards, and then have my one heavy hitter card.
Do you mean running an eGPU on my MacBook and passing it through to the VM? That’s not what I was eluding to, sorry. I just mean I want to be able to run the VMs from my MacBook and not have to have a separate keyboard and mouse dedicated to each VM. Thinking on it more, I do believe these were special cases, as that wouldn’t make much sense at all.
I’ll look into KVM, thanks. Anything particular you like about it over the others?
I’m trying to keep it around $5-6000.
The only issue I come back to with the threadripper is that everyone seems to be having problems getting things working properly, usually motherboard related.
I guess I was curious if I would be better off going with multiple tried and true setups and not having to fuss with anything. PCIE 4.0 would be nice though.
Alright, I did a ton of research tonight…and there are a few straggling questions I have.
I am absolutely going to use KVM, but likely Proxmox. It seems to be the go-to from the reading I’ve done.
I priced a 3970X threadripper machine, and two 3950X Ryzen 9 machines that would equal the same core count when combined. The pair of Ryzen’s are hilariously cheaper, and that’s with 4 GTX 1660 Super GPU’s (2 each).
That being said, here’s the straggling questions:
Will KVM/Proxmox be able to cluster those two machines together to create a seamless experience like ESXI would?
Do I need a “donor” card (one that will be used by the Ryzen host machines), or can I pass all 4 cards through to VM’s?
I’m going to research this endlessly, but if someone wants to save me the time, is there a quick wiring guide for how these will be setup? I’m trying to figure out the layer 1 stuff.