I’m not asking to be spoonfed, I’d just like a quick sanity check if it makes any sense to pursue my current plan.
I’d like to convert either my current system (8-core ryzen 7, 32GB of RAM, could be upgraded to 64GB if necessary), or a used PC on the cheap, into a baremetal hypervisor. Within that Hypervisor I’d like to setup virtual networking and a 3-node cluster. Within that cluster I’d setup my actual homelab environment, with Windows servers as Domain Controllers, a fileserver, a few clients, etc.
From what I’m reading, that should be possible with nested virtualization. I know fully well, that virtualizing a cluster makes zero sense, since hardware failure results in all nodes going down, defeating the point of HA. This is just for learning purposes.
For the baremetal hypervisor I’m planning on using proxmox, with vmware ESXI (and other vmware tools) for the hypervisors that make up the cluster.
The reason, why I’m thinking about doing this, is to be as cost and space efficient as possible. I don’t want to get 3 physical machines to make up a 3-node cluster, which I then have to network together as well physically, which would involve getting extra network cards or switches. And I’m hoping/assuming that the virtual network capabilities of proxmox are good enough to simulate physical networking for my environment.
As for my background, I have windows server administration and hyper-v experience, but my main focus is on Microsoft 365 and Azure administration, system integration and consulting at my current job. I’d like to become more well-rounded and learn more about linux and other alternatives to microsoft products and services, hence vmware and proxmox.
Hi, welcome to the forum!
For evaluating hypervisors and general lab/demo use, nested virtualization definitely makes sense. The one potential issue during setup that you might bump into is needing to set resources per VM a little higher during initial setup and adjusting them downwards afterwards (example that comes immediately to mind: Server 2012 wants 800mb RAM to install, even though the official minimum is 512mb). Will depend per OS/application that you’re setting up, of course.
I have a single machine doing most of my homelab bits, so a sinle machine should be fine (not HA, or failover, or redundant, but it’s homelab…)
I use Proxmox host, with linux vm switch, linux vm vpn concentrator, openwrt vm firewall / router, and linux vm pihole.
Storage is just on the proxmox host, using zfs.
Never really looked at windows servers, especailly as vm’s.
How’s the licensing on them?
Also, how wil ESXI’s Vsphere price changes factor in?
(EOL the free tier)
Evaluation edition is good for ~180 days (source: Windows Server 2022 | Microsoft Evaluation Center), but has to activate in the first 10. Usual caveat being they want you to sign up for emails for it and all that.
Please excuse the derailing of the thread OP…
Was that the thing where you can only re-activate it like 3 times?
Or just every 180 days?
Or, wipe it each 180 days, and start a fresh 180 day trial?
That I don’t know. I’d assume it’s wipe and start fresh. I tripped over it in the context of somebody else’s last-minute-panic-upgrade from 2007 version of an app to a 2016 version that had to step upgrade through 2010, 2013, and then to 2016, with all the attendant infrastructure upgrades underneath.
Thanks. (end of derail)
Thanks for the quick replies, sounds like all of what I’m trying to do is feasible. Guess I’ll get to it and start learning proxmox.
I’m actually not 100% sure how the licensing on windows server works out after the trial version, since I only know it from work and everything is licensed there.
From what I read you get a baked-in product key that is licensed for 180 days when you download the evaluation edition of windows server, so you can probably download a new ISO and extend the trial for another 180 days by reinstalling with a fresh ISO. Might even be able to just read the product key from the fresh ISO and transfer it over to the existing VMs.
I won’t worry about licensing my homelab, I highly doubt I’ll get a MS license audit on it and tearing it down after 6 months is no big deal, might be a good opportunity to automate some setup tasks.
As for licensing Windows VMs in general, you always license the physical host server (either per core, volume licensing, enterprise agreement, etc.) and after that you activate the VMs on that host with your product key. Windows Server Standard Edition is good for up to 2 VMs, more than that requires Windows Server Datacenter Edition.
It gets more convoluted as a Microsoft partner, you get free licenses (how many depends on your partner status) that you are allowed to use for your own organization, but not for customers. But since the licensing is per physical host you can’t use the same hypervisors for your internal VMs and VMs/environments you’re hosting for your customers. That’s how we ended up with two 3-node clusters at my old workplace, even though it would have made much more sense to build a single 6-node cluster.
nice.
I saw Morten recently go thru the re-arming of the trial version, and was vaguely aware it was a thing
He also runs Vphere / esxi as a hypervisor, and a bunch of stuff in VM’s.
But he sure doesn’t ascribe to a single box containing all the things 
There shouldn’t be any reason why you can’t virtuallize everything. I run Linux on all my computers and VMs so I can’t speak to the windows experience but for me it has very little overhead on proxmox. I even have a VM that I use for light gaming and as a desktop.
With that being said, I would go full Proxmox with a three node cluster. Proxmox can do everything you need and will avoid the sillyness of VMware. It also should have great support for vfio for setting things up like virtual NAS.
Yeah, I got proxmox up and running and it’s pretty much working as expected so far, including windows VMs and even GPU passthrough, which is awesome.
I think my networking needs are also taken care of within the virtualized environment, with a VyOs VM acting as a router and virtual bridges within proxmox acting as switches.
I understand that proxmox probably has all the functionality I need, including clustering, but I’m specifically trying to learn VMware, simply because you run into it a lot in many business/enterprise environments.
Nested virtualization totally makes sense.
Here’s a thought, you could try putting a proxmox cluster inside of proxmox. Try ceph, try FRRouting, try vm migration.
Yes, loads of sense. VMs or containers. Keep the host fast, clean and secure. Spin up VMs or containers as needed. Learn, test, break, fix, destroy, rebuild. Will make it easy for you to learn without messing up the whole system.
Have fun.