Help picking a virtualization OS

Software & Operating Systems Open Source & Web-Based
1

Hi all, first post here.

I could use some help picking an operating system that can host multiple virtual machines. I have had a look at proxmox and unread. While they are viable options for what I want to do. I wanted to ask the community for their input on this.

A bit of background- I am trying to run a medium sized active directory within a home lab to practise my AD offensive security skills. My hardware specs are i9 9900k 32gb ram and 3TB of overall SSD storage with a 120gb for the OS to boot off as a seperate drive.

Any help is greatly apperciated thank you.

2 Likes
2

When I moved from dedicated servers to virtual machines in my homelab I played around with several type-1 hypervisors. Though I did not do anything with Hyper-V as I refused to run windows in my homelab since I moved to Linux many years ago. In fact, my only two windows machines are a gaming system and a VM on my laptop for some software (HP iLo & Dell iDrac) that doesnā€™t like Linux or for doing driver updates (printers and other similar devices) on some of my devices. I could not ESXi to run on my hardware that was a few years old at the time and still is old by technology standards, though I did try out XCP-ng though ultimately settled on using Proxmox VE.

I have been using Linux for my servers for a long time now and decided on Ubuntu Server many years ago when I moved away from Windows XP for my host OS and on my workstations (laptop and another desktop) use Pop_OS so with Proxmox being based on Debian with a modified Ubuntu kernel it made it easier to learn and understand.

3

Thank you for your reply.

I agree with you not wanting to use Windows hyper V at all.
What made you go with Proxmox over say free nas? I have tried Proxmox and I do like it a lot but was wondering if there was anything better suited for my needs is all.

4

Welcome to the forum!

This community is very biased towards Proxmox. And for good reasons, it is good, stable, easy to use, gets the job done and you very rarely need to touch the terminal. There are also some guides on the forum for Proxmox, mostly in the form of questions, like how to set up pfsense in proxmox.

I only know one one individual who runs oVirt. Some people around run esxi. I run virt-manager after moving off of Proxmox (nothing wrong with it besides clustering on less than 4, or rather 5 nodes not being ideal, my reason for it was its LXC implementation and clustering, the later of which I donā€™t use anymore anyway, so I could theoretically use proxmox as a standalone).

Would not recommend unraid, unless you really need to mismatch HDDs. Iā€™d say just buy disks and run proper ZFS, which Proxmox supports.

On the forum, Wendell was the only one I know to have ran bhyve extensively on either FreeBSD or TrueNAS Core.

I would suggest sticking to what most people use, which is Proxmox.

1 Like
5

(post deleted by author)

1 Like
6

You didnā€™t really expand on your needs other than being able to run windows, possibly in a segregated network segment of your homelab, and given the intended use, the need to be able to re-set the core VMs after you are done ā€˜experimentingā€™

These are fairly basic needs nowadays that all the mentioned hypervisors can easily manage, so only you can decide what suits best your workflows

The main difference between Ovirt/Xcp/Esxi and Proxmox/Truenas/Unraid/Generic Linux with libvirt are that the former need dedicated level 0 hypervisor on top of which you run your VMs, where the others let you use the host for other tasks, if needed (I put Ovirt in the level 0 category improperly as it runs kvm, but youā€™re not supposed to touch the hosts, only manage them through the web based manager a la ESXi)

Proxmox and Truenas support ZFS, the others donā€™t (at least natively) but not a factor for your use case

Is it? Iā€™d say the community is biased toward open source products, so youā€™ll have to dig to find someone that doesnā€™t hate the proprietary parts of ESX and the associated cost, but thereā€™s a place for that in some contexts (hello VSAN/VMotion/live migrate to AWS/Azure/RVTools and such) just not a lot of sense to deploy it in a homelab unless your already familiar because of work/want to skill up in order to find work and it is very picky on the hardware

I use oVirt, just not at home, both in datacenter and even in some GCP deployments, go figure :slight_smile:
It is a very solid technology, geared towards the enterprise, so it really doesnā€™t make a lot of sense to run it on a single server, whereas it is a very solid option for SAN/Cluster deployments where specific functionality of VMWARE is not needed and licensing cost is important (also very important if youā€™re deploying Oracle Databases and want to save as much money as possible on licenses). You can even deploy it in a hyperconverged configuration using gluster on the host nodes, but I havenā€™t yet dared to propose it to a customer :slight_smile:

I wouldnā€™t either as I prefer ZFS based hosts, and I like to have as much control over my kernel modules/config/system packages as possible, but it has a lot of pre-baked plugins that appeal to a lot of less tech savy users

I have run a forbidden router on Bhyve for almost three years (pfsense), now running vyOs on scale

So, what is worong with trying out a couple of options and then reporting back on what you liked/disliked ?
One thing I can tell for sure is that most of the people on the forum like to tinker with their homelabs ā€¦

7

Iā€™ve run xen, proxmox and esxi.

For your use-case of not wanting the hypervisor to be the project but the environment built within to be the playground Iā€™d say try ESXi first.

8

One of our company internal labs (not my homelab, but I did set it up) runs oVirt on a repurposed Exadata X-4 eight rack, using two 512GB/28 core database nodes as hosts for the VM, one 42TB storage node for the storage via ISCSI over infiniband (dual 40Gb links) ā€¦ itā€™s about 800w when idling ā€¦ but for a corporate homelab it works just fine :wink:

2 Likes
9

Since serving files is an integral part of all servers, I suggest rolling with TrueNAS Scale. Itā€™s been moving along nicely of late and has a stable debian base. I havenā€™t begun on my journey yet, but this is my plan

10

I have run hyper-v, esxi, Ubuntu and unraid as vm hosts. Being a tech forum there are plenty of opinions. It depends a bit on how far down the rabbit hole you want to go.

I donā€™t have a lot of need for work purposes so my use is just as an enthusiast.

I currently run a few of mine on unraid and another on ubuntu. I already have a couple unraid boxes for storage and docker and they are just set and forget (a lot will say the same of their solution so Iā€™m no different). I have a download vm on my ubuntu host NUCā€¦ All of them I have their boot/shut down scripted to my stream deck or phone through home assistant so merely a button press to spin up or down.

I suggest having a play with a couple solutions and see what you like best and find easiest for what your trying to do. Might waste an afternoon but will get an idea of whether you need fine grain control or just a few button clicks to get something basic knocked up

11

Nothing wrong with it, but it seemed to me like if OP wanted to thinker and try them all, he probably would have already, instead of asking us. :slight_smile:

Most topics and answers on the forum I believe to be by far surrounding Proxmox. Iā€™m not saying that XCP-ng, VMWare, TrueNAS Scale or others are bad, Iā€™m saying that for most people who donā€™t like tinkering on their own, Proxmox makes a lot of sense because of the community support on this forum.

There will be people answering questions on XCP-ng and oVirt, but IMO to a lesser extent.

Iā€™ve ran Proxmox, OpenNebula and XenServer in production environments, dealt a little with VMWare vCloud and had to deal with vCenter at some customersā€™ infras and I always found vmware buggy and hard to use (maybe itā€™s just me). Even OpenNebula with its quirks on VM filtering (all the checkboxes remain active even if entries arenā€™t visible anymore after looking for something else, which is pretty insane), was easier to use.

I never used unraid and I can say I definitely gravitate towards open source. I have seen very few people not pleased by Proxmox, among them being me, which is mostly nitpicking, but my displeasure with it still hasnā€™t made me not recommend it to others.

Then that makes it 2 people I know on this forum. 2 for oVirt, 2 for bhyve. Keep in mind that this is just my experience / knowledge, not an actual number. And also from what I am seeing, Proxmox is by far preferred on this forum.

1 Like
12

Nobody mentioned linux and virt-manager/qemu yet

1 Like
13

My first post too :slight_smile: and Iā€™m curious what otherā€™s thoughts are.

I evaluated proxmox, XCP-NG, oVirt, and ESXi for homelab stuff a couple years ago and liked oVirt the most. (For its licensing situation and functioning cloud-init/ansible workflow). I did feel it took a bit more time to figure out where things were than the other options. (I liked proxmox a lot, but cloud-init didnā€™t seem to fully function at the time - some stuff didnā€™t init properly - so I couldnā€™t use it.)

Not wanting to steer you in any particular direction - just want to say they were all worth trying and I feel like they all have their strong points. Doesnā€™t surprise me at all that there is no clear ā€œwinner.ā€ (Although Iā€™m surprised oVirt isnā€™t more popular than it is in the homelab space.)

14

The setup proceudre is clonky, it requires a VM to run the web management console, it uses LVM to do snapshots, the cluster requires a SAN or a three node minimum deployment, thereā€™s no immediate way to load an ISO to deploy a VM ā€¦ a very good product that Redhat first and IBM now kept difficult to use on purpose so they could bill consultancy hours to make it work for you ā€¦ not surprised at all itā€™s not on the homelabberā€™s radar
And the devs on the forums are dickheads for the most part ā€¦ and since the switch to IBM I have seen a lot of valid feature requests being pointed to paid support if someone wanted them to be considered ā€¦

And donā€™t get me started on the ansible ā€˜integrationā€™ that has never worked properly to begin with and is been redesigned since at least two years without a way to really use it for real production deployments ā€¦

15

Indeed, Iā€™d even suggest bhyve in FreeBSD for this task :slight_smile:

16

I went with Proxmox over Free/TrueNAS as those are both a storage first systems with the ability to run virtualization whereas Proxmox VE is a hypervisor first and can do some storage-related tasks.

I run all my storage servers using TrueNAS even on my older QNAP devices and it works amazing and find that Proxmox as a hypervisor does 99% of what I want and the new things that it does not are either handled another way that is supported or on their road map (multiple cluster management for example).

17

Thank you all for your replies. Sorry for a slow reply to this thread. I have tried proxmox and while everyone is correct it will do 100% of what I am trying to achive I just looking for some alternatives that I may have not thought off.

A bit of further background that I should have included in the first post.

I am just looking to build a potentially a small multi foerst DC.
Thank you all for your replies. Sorry for a slow reply to this thread. I have tried proxmox and while everyone is correct it will do 100% of what I am trying to achive I just looking for some alternatives that I may have not thought off.

A bit of further background that I should have included in the first post.

I am just looking to build a potentially a small multi foerst DC.
Thank you all for your replies. Sorry for a slow reply to this thread. I have tried proxmox and while everyone is correct, it will do 100% of what I am trying to achieve I am just looking for some alternatives that I may have not thought of.

A bit of further background that I should have included in the first post.

I am just looking to build a potentially a small multi forest AD lab. So my idea is 2 Domain controllers and 3 differnet Windows clients along side a pfense vm for networking. I think I am going to go with Proxmox after playing with Esxi and free nas over the weekend.

18

Perfect, all i can say is that you need to consider few factors before choosing one OS. Try check on the virtualization method you want to adopt. Regardless of hypervisor type, there are 3 main virtualization methods and choosing the right one is essential. One method is based on binary hypervisor translation which allows multiple operating systems to be run at the same time without any conflicts. On the other hand, using hypervisors with hardware assist means that embedded CPU virtualization support is needed in order to pass control automatically if needed. There are several reasons to choose either of these methods but it is important to note that no matter the decision, supporting guest operating systems wonā€™t be a problem. The third method is called paravirtualization and can return higher performance by calling the hypervisor directly through an application programming interface.

19

Hi there, i don know why you would prefer using a virtualized OS, but ild suggest you try doing the installation of Vmware Fussion or Workstation. The software will help you virtualize any OS that you would like to use alongside the host.