I recently had a motherboard fail which caused some issues on my Unraid based homelab.
I have 2 systems built for virtualization and backups. I’d be more interested in using one system but I keep getting the sense from reading around the internet and on the forums that I may be asking for too much or risking stability.
Systems -
1: ryzen 2700x with 32 gb of ecc memory on an asus prime B550M-A
2: ryzen 2950x with 128GB ecc on a Gigabyte Aorus Extreme X399
Right now system one is running Unraid and it’s used for file management and plex.
System 2 is being used as a place to get some familiarity with proxmox and truenas and ubuntu.
Goal -
I’d like to use some combination of these 2 systems ( or just system 2) to do the following:
Gaming Vm
router
Plex
davinci resolve/ blender Render server
Cloud Storage
Stable diffusion
System Backups
Video ingestion
editing from network storage
hackintosh VM
So far it looks to me like I should run Proxmox as the Hypervisor on system 2 and pass my sas controller through to a truenas VM to manage my array now that I have ECC I also have an optane 900p.
I know that a lot of these can be done with either system but I’m struggling to get started because there is 36Tb of storage used and I’d like the best shot at getting the right configuration set up the first time.
Can you share best practices and suggestions/experiences with doing this kind of thing?
Best practice would be to run TruNAS on machine one and proxmox on machine two. Run all your VM’s and containers on machine two and just use machine one as a rock solid reliable NAS.
Now, if you want to have a “Forbidden NAS” then I would run Proxmox with shares controlled by either containers (Nextcloud, Jellyfin, etc) or command line utilities (NFS, SMB, etc). Proxmox has native ZFS and is just Debian under the hood. It also won’t wipe all of your custom configs like TruNAS will when you go off the iX systems script.
Once I found out I couldn’t just import my smb.conf script into TrueNAS Scale I was out. I guess if I were starting from scratch it might be fine, but I’ve got VMs/containers, file sharing, and snapshots/send/receive already sorted, so really all Scale was going to get me was a prettier dashboard than Proxmox has. I might use TruNAS if, like @ucav117 says, you’re going to run a pure NAS on one machine and your hypervisor on the other. But for an all in one, Proxmox gets my vote.
I love the idea of running things from a singular system. I know there are always tradeoffs when weighing performance/stability but hopefully I can build it with stability in mind first.
I love the idea of everything being in one system so I’m leaning toward the Forbidden NAS setup as the goal.
I’m coming from Unraid, is the learning curve to get the NAS functionality of that setup running steep enough that it’s better to start with 2 separate systems first?
I think plenty of folks have done OK running TrueNAS core in a VM with a HBA passed through to the VM. This would get you all the good-looking dashboardy stuff that you’re used to from Unraid for the NAS portion of your Forbidden NAS. From there, most of the VM/container things you want to do are exposed via the UI in Proxmox.
On the other hand, if you’re comfortable with the command line and/or want to learn, there’s very little to be afraid of running your ZFS storage pool in Proxmox. Proxmox tends to run a recent version of OpenZFS, and ZFS’s commands are extremely simple. It even has scheduled scrubs built in, which is great! But you have to roll your own for file sharing and backing up your storage pool, which is a non-insignificant portion of what one expects their NAS software to accomplish. I had already set up my own file sharing and backups, so TrueNAS’s color-only-within-the-lines approach was unappealing to me personally.
Lots of ways to skin this cat–you just need to decide which way fits your personality/needs best.
You obviously need to have a enough RAM for both the Proxmox host and the TrueNAS VM, and I’d recommend having a dedicated HBA to pass-through if you can.
Also, TrueNAS CORE uses all available memory for LARC, whereas SCALE only uses half (by design), so I use CORE for this reason in my VM.
And, as you said, you can setup Proxmox fairly easily to be a NFS and/or SMB file server… it’s just not possible via the Web GUI.
(and I personally really dislike that Proxmox mounts it’s ZFS pools at “/” instead of “/mnt/” or somerthing similar… )
I appreciate the detailed response. I guess the all I need to know to solidify my decision is the following. In order to use this solution as a store for critical data, what else would I need to do? Or is this just a thing that I need to accept if I’m being stubborn?
My plan was essentially to follow the IBRACORP Proxmox/TrueNas tutorial later today since my data is finally ready to move and I have the day for building and initial configuration.
Since I’m just a regular person you could argue that none of my data is truly “critical”, and I was being a little hyperbolic, so IMHO virtualizing TrueNAS is fine for us normal people…
For the stuff for me that’s “critical” - where I’d be emotionally upset if I lost it - e.g. personal photos/movies, tax returns, things I’ve created (text, images, software, etc…) - then I have them stored in multiple places (and physical locations)… laptop/workstation, primary NAS, backup NAS, and the cloud…
So, my recommendation for the stuff that’s really critical to you - and it doesn’t / shouldn’t have to be everything you store on your NAS - at a minimum, you need at least one (easily recoverable) backup, whether it’s external hard drives, a backup NAS, the cloud, tape-drive, whatever…
For everything else, just remember that “RAID (i.e. NAS) is not a backup”, so If you were to lose the data, you need to figure out how desperately you’d want it back, and how much time, effort and money you’d be willing to invest to make it happen…
e.g. a music / TV / movie collection : if you keep your physical CDs, blu-rays, etc… you’d have to take the time to re-rip/re-encode them; if they’re gone, or you downloaded your files, you have to figure out how you’d get them back…
and/or or use old HDDs and create a slow backup
and/or use something like Amazon cold storage where storage is cheap, but retrieval is slow/painful
No worries, I’m a normal guy too. I’m also just mentally predisposed to optimizing everything I have and selling the rest. I do video editing for weddings so I try to keep the raw files for about a year so that’s the level of critical data I’d like to store. I didn’t consider this before, dy have an opinion about this system vs Wendell’s genetically altered TruNas system?