To start I will be new to Proxmox, but you gotta start somewhere! I was a ESXI guy for many years until everywhere I have worked lately have been primarily Hyper-V houses. Since that has been the environment I live in daily, it is what I chose for my home server. My home network is not really critical and more a lab with file share, media center, Unifi controller, things like that. I do want to start playing around with containers, play with ZFS, automation, etc and still keep my ability to have VM’s for my lab. Also, I spin up W10/W11 and some Linux VM’s for troubleshooting some work issues.
Originally, I was looking at TrueNAS and Unraid (especially after watching some of the videos put out by L1) but need to retain that ability for the VM’s. From what I am reading is that Proxmox should cover all of this and more, is that correct? I will be putting this on a few years-old Dell T430 with dual Xeons, currently 64GB RAM, 8 x 3TB drives.
If this does check all the boxes as it seems, any gotcha’s you have run into when first starting with Proxmox? I am new to it but have been in IT in some capacity for about 15 years (mostly in Windows environments) so I will not be a complete newb. Only partial! Any advice is appreciated.
Well, in all honesty I am new to the container / docker world and want to learn them. I have found so many solutions that could meet most, but not all of my requirements. I have heard that Proxmox is becoming more popular out there and figured it’s not a bad skill to learn at the same time as the previously mentioned. So far I have the server backed up to restore to a Hyper-V server if I just say “f-it” and installed Proxmox on a dedicated 1TB SSD. I have a PERC H730 I left out and (as of now) the plan is to pass that through to a TrueNAS VM to use a ZFS pool. Reason I am going to do TrueNAS with it is to have a NAS and seems TrueNAS is the best fit for what I am wanting to do, but currently only have one piece of hardware to work with. I do have some other machines I might put together in the future, but I only have the one server to work with currently. Thank you for answering my question, and your suggestion!
Proxmox, TrueNAS, and Ubuntu Server do ZFS natively.
Wendell did a video recently on using TrueNAS Scale, and running a Debian VM with docker and a bunch of containers in it.
Since you have a bunch of disks you’ll be a) exposing over the network and b) will be carving out for various workloads, it might make sense that the disks go to host and you don’t do VM<->VM storage.
Craft computing showed a truenas scale inside of Proxmox setup which sounds like what you’re after, and Jeff mentions how virtio networking can be incredibly quick, but in my own personal experience the VM<->VM performance can be incredibly fiddly.
Down the line … you can run containers on TrueNAS Scale directly, but it’s using k3s internally (variant of kubernetes), and it’s not exactly configured the same way k3s you get from rancher would be on your typical linux, it’s quirky, on top of more quirky, on top of learning that reddit shows below.
I think I’d go with Wendell’s approach… non virtualized storage, one VM for docker-compose and more VMs for windows stuff or things that really like having their own kernel.
Thank you for all of that, I appreciate it! You’re correct on my original plan. I actually have it at least half built currently. The reason I was diong ProxMox with a TrueNAS vm (passthru the HBA controller with 8x3TB disks. Then use ProxMox for my Windows VM’s and some other containers like my Unifi Controller. Other than the one Windows Server, the rest will likely be containers and Linux vm’s, with the occasional Windows client and server OS just for testing / lab / work / etc.
Currently I have ProxMox installed and was able to get my Windows server built (well finishing, but up and running), my Unifi Controller moved to a container, and installed TrueNAS as a Linux VM. I am having a ton of issues with getting static IP to work on the TrueNAS though. I got it to work once, but then after I rebooted the host it went back to DHCP. If I cannot get it with another install / version I am going to just wait until I get my DHCP server built again and set a reservation for it.
I was going to do TrueNAS on the baremetal, but I really want to use that server (currently my only one, but will get more again soon) for labs and VM’s as well. I didn’t go with TrueNAS since it doesn’t have real VM’s. My plan for the TrueNAS will be to move it to it’s own hardware eventually as I just have to put it together and maybe order one or two parts along with HDD’s. When I move it do you have any resources, or recommendations, for hyper-visor to use for purely VM’s of all sorts? I have been pretty familiar with ESXi and Hyper-V as it’s what all my clients / jobs have been using. I have no experience with any other hyper-visors and not sure which are worth it, used in real world, or add some benefit or skillset.
Again, thanks for your assistance and dealing with my lack of knowledge with the container world. It does interest me and looking forward to learning it.
Are you booting windows server off of pxe + iSCSI … or are you somehow mounting a network filesystem into Proxmox and emulating storage backed by files…or are you connecting zvols as block devices on Proxmox? How did you set up your storage?
Is your truenas root/os … booting off the same hba or are you having Proxmox file backed block device?
I’m contemplating what would be needed to “migrate in place” so to speak?
Quick update, I thought I got the network adapter to stick, but I was wrong. I noticed a notification in the top right and saw that they now want you to have a different account than root for the WebGUI when it used to be the ONLY one with access. I was able to create a new account and add it to the built-in administrator group, then I was able to log in to the web-gui as the new account. Still having issues with the IP sticking I decided to add a network adapter to the machine and set that as the static I wanted (leaving the original as well). I was able to log in from either IP, even after a reboot! I deleted the original one and as soon as you reboot you can no longer access the WebGUI just like before. I think I might need to do the DHCP option.
To answer your question about my Windows server, yes it would be a DC, probably my DHCP and DNS server for now (I’m more a server admin, than a network guy) until I can research what to deploy as it’s replacement. The other main purpose of this server will be for the domain and playing around with M365 services as I am going to be finally upgrading from those 03 / 08 certs. Nothing super critical (other than DNS really) as I do not use my domain for any business at the current time.
The storage, I am still trying to straighten that all out (both in my head and on the server ). As of right now ProxMox is installed on a 1TB SSD with to vol on it, 100 and 850ish respectively. Containers and VM’s are on the 850 vol on the 1TB SSD. I am currently trying to get ProxMox to see the disks so I can pass them through directly to the TrueNAS VM. It’s a Dell Perc H730 that I have already wiped the raids and configuration, then changed to HBA mode. I boot up, the old raid VOL are gone, but no disks to be found anywhere. Once I can pass those through, I am going to be passing all 8 x 3TB drives to the TrueNAS and then using ZFS (completely new to ZFS, but intrigued thanks to @wendell and his multitude of videos on it. Especially the recovery of LInus’ OOPS). I will run most containers, PLEX, etc. from the TrueNAS leaving ProxMox to be soley for my Unifi controller, Windows Server, maybe some Linux VM’s, and testing.
As far as migrating, are you referring to migrating the TrueNAS VM to a new physical appliance when I build it? I have no clue but figured at worse case since I do not have an extreme amount of data, I will back up the data to an old QNAP/Synology or USB temporarily, build the new hardware fresh, then copy the data back. Right now, it’s only maybe a TB or two. That is until I get my media ripped and build the library.
Adguard home in a container – even if you’re not using it for filtering it has a decent DNS over TLS implementation and checks signatures. Alternatively pihole + some other DNS.
). As of right now ProxMox is installed on a 1TB SSD with to vol on it, 100 and 850ish respectively. Containers and VM’s are on the 850 vol on the 1TB SSD. I am currently trying to get ProxMox to see the disks so I can pass them through directly to the TrueNAS VM. It’s a Dell Perc H730 that I have already wiped the raids and configuration, then changed to HBA mode. I boot up, the old raid VOL are gone, but no disks to be found anywhere. Once I can pass those through, I am going to be passing all 8 x 3TB drives to the TrueNAS and then using ZFS (completely new to ZFS, but intrigued thanks to