Hello Level1Forums! This is my first post here and it’s gonna be a lengthy one so better brace yourselves! For the past 2 years I’ve been using a very janky homeserver setup. I use ubuntu 22.04 (not fresh install, continually updated from 20.04, yeah I know, bad practice) and even though I have QEMU KVM set up I didn’t do any virtualization on it. I run all of my services on bare-metal ubuntu because why would I spawn VMs when my server is literally connected to a monitor and all of my services are compatible? In fact I do most of my server stuff directly with a monitor and keyboard even though I have VNC and SSH set up. Weird right? Anyways, the services I use are:
- PiHole + PiVPN (Wireguard)
- qbittorrent + jackett
- plex media server
- a minecraft server
- 2 discord bots
With the current server I am not hesitant to break stuff and spend a full day repairing it. It doesn’t store any critical data anyway. It’s just a useful home appliance that can have downtime. So after breaking my drivers multiple times and fixing them, breaking and reinstalling plex at least 3 times, breaking and somehow reviving qbittorrent when transitioning from snap to apt, SMB nightmares, discovering, reporting and finding a fix for a bug in ubuntu/firefox and much more, it really doesn’t matter in the end if it all goes to hell tomorrow. And honestly as much as it was annoying it was also kinda fun fixing issues and making it all work.
But recently after a bad experience with OneDrive, seeing how my mom asks me on a yearly basis to copy her phone images to her PC and how my dad has his google drive full 24/7 and documents everywhere I decided it’s finally time to fix all the plaguing issues in my family’s storage of data. Actual important data. I’ve been planning to turn my server into a NAS for a while now but this was the push I needed to finally get started. Yeah, get some nextcloud set up as well so the data from all our phones and PCs is nicely and automatically backed up. Beautiful. And unlike with the current setup I want this one to be stable as hell. Because for the first time I’ll be making something that needs to store important data and not risk losing it. Maybe I can get a system specifically for messing around later but for this one I can’t make any unnecessary risks.
So I’ve been doing my homework. Watching youtube, reading up and planning my setup. When TrueNAS Scale dropped it sounded really promising, ideal for my use case: a NAS that does all the virtualization for the services I need, in a single box. Truly an Ultimate Home Server OS right? Or so I thought. Just as I was planning the setup wendell dropped a video about TrueNAS with Nextcloud and Tailscale which really opened my eyes as to how broken TrueNAS Scale actually is and I had no idea about it. If I hadn’t seen this video I would’ve walked into this minefield and stepped into every single trap it has laid out for me.
Initially I was planning on using the apps tab for services like nextcloud, plex and qbittorrent. The idea of all my storage-based services having direct access to storage instead of through an NFS share sounded promising to me. But after seeing this video I came to know about the issues I previously didn’t know existed. Yeah turns out that both apps and virtualization on TrueNAS Scale are kinda broken. Apps make weird uninitiated snapshots of container persistent volumes and makes lots of them. I’ve seen a temporary solution here involving a cron job and a docker-prune command but I am not really comfortable with this solution as I feel like it could break very very badly. Another drawback with containers is that it would probably be a nightmare getting them to talk to each other (qbittorrent+jackett) although I could be wrong. Plus I have 0 actual experience with containers.
Next in line are VMs. Sure I can just as easily set up a VM with said services as I can using containers. The only drawback would be having to use an NFS share to access storage instead of having direct access which won’t really matter for performance because all of these apps are limited by network anyways. But behold! VMs are also broken. From the video I’ve learned about a weird debian default/bug that was fixed more than 10 years ago but somehow still exists in TrueNAS Scale. Yeah talking about the one where the VM can’t ping the host at all. The solution to this is setting up a bridge connection through the CLI (because GUI is broken of course) which is all so unnecessarily difficult for something that should just work out of the box. But I guess this is a one-time thing and then smooth-sailing right?
Well, another thing that really pushed me away from TrueNAS occurred to me when reading the docs and doing further research. Apparently a TrueNAS install takes up the entirety of your boot drive however large it is. You can’t put anything on the drive TrueNAS is installed in. Given that I have a 1TB boot SSD I intended to use to store VMs as well, losing >900GB of storage for basically nothing is a massive waste. TrueNAS doesn’t let you store anything other than the OS on this drive and messing with the linux underbelly to try to remedy that breaks stuff. Partitioning the boot drive is not supported officially but community managed to succeed, although I am not comfortable with using this as it can kill my install AND all my VMs if it breaks. I happen to have a 250GB SSD in an M.2 enclosure because I thought I need it but I honestly don’t. So I can steal that SSD but that’s still a loss of >200GB.
So I’ve reached my last option and the one I intended to use before TrueNAS Scale even existed. Use Proxmox as my hypervisor and TrueNAS inside a VM. I am aware Proxmox supports ZFS and network shares out of the box but TrueNAS makes it infinitely more simple and time-saving. This perfectly remedies all of the above issues: VMs will be handled by Proxmox which has way more virtualization features anyway, TrueNAS will do what TrueNAS does best which is just storage and file sharing, and since it will be installed on a virtual disk, I won’t have to sacrifice any SSD space. The drawback with this setup is that it adds complexity. I would have to learn to use Proxmox and reading through the docs left me with more questions than answers, mostly storage and backup related i.e. idk which filesystem to use for the install. Another thing is that managing snapshots and backups would be more difficult as I would have to deal with proxmox snapshots as well and I would want my VMs and/or certain data from VMs to be replicated to TrueNAS as well.
So help me decide, which setup to go with? Are there any more traps I don’t know about, ready to ruin my day and eat my data?