I am looking to build my first home server, and I want it to be a bit of jack of all trades. I have no previous experience with ZFS or TrueNAS, just what I got from reading articles so far.
Its main uses will be:
Media storage, transcoding and delivery (plex)
Torrents
NAS (ftp, SMB)
Game servers (mostly modded minecraft)
Other occasional VMs (docker)
My problem is that these use cases have pretty varied storage requirements, and this article: “ZFS / TrueNAS Best Practices?” in this category (sorry can’t link) provided a lot of good insight, however my questions slightly different.
I have a limited budget and my idea is to buy something like 3x 8TB HDDs and one 2TB NVME SSD.
Considering I want:
mostly read only, redundancy not required storage for media that is going to be by far the largest use of space
infrequently accessed NAS storage where redundancy is critical but not a lot of space is required, probably <1TB
higher performance storage for game servers and other potential VMs, where redundancy would be nice and the space requirements are also fairly low.
My preliminary naïve idea is to use TrueNAS Scale and ZFS, I could not find a way to achieve varied levels of redundancy without having completely separate pools, which makes little sense with so few drives, so my idea was to have RAIDZ1 pool for the HDDs and just use and allocate space on the SSD to applications as needed.
I have also read a bit about hybrid pools, but I am not sure if I could configure one to be both fast for applications like VMs and gameservers but also be good at storing large files accessed infrequently.
With that in mind I would love your opinions, suggestions and experiences on what approach I should take.
Questions that I would roughly like answered are:
how would the architecture look like: Multiple freeNAS isntances? Do I even use freeNAS? Do I use anything on top?
Without knowing what you’re familiar with or how much work you want to put in to start, i’d comment on the truenas angle.
Using only what you’ve stated above - it is possible to raidz 3 8TB HDD’s for your bulk storage in one pool, with whatever datasets you want for your mainly read only storage.
The one flash drive for shares of anything latency sensitive, you can also configure to snapshot and replicate the dataset(s) from that, to backup datasets on the bulk storage, every hour if you want. There are tools built into truenas for this. (There are those that would argue that, within the same system it’s more a case of ‘redundancy’, than ‘backup’, you are welcome to decide on your own)
I’d probably not install truenas itself on any of the drives in question (so that your storage drives all serve the network) and put truenas’ system dataset on the redundant pool.
I don’t use truenas to host vm’s (rather, run it in them), I’d imagine however it can get it done.
Spinning rust for seeding is ~fine. For downloading on the other hand, you’ll likely be limited by iops of your three drives on raidz1… (unless you have a really slow internet connection).
Three drives in raidz1 as your normal storage, carve out a big L2ARC partition out of the SSD, and carve out e.g. 500G out of your SSD for your torrent incomplete directory (basically /tmp as a separate single ext4 partition… specifically because ext4 is not cow, cow for torrents makes no sense, at least not in 2023) and setup your client to move torrent data to raidz1 on completion for long term seeding - which will copy what you download as nice sequential files.
That’ll mean you should be able to simultaneously download and seed at e.g. gigabit speeds, and eventually your seeding ability will drop to 300 iops (assuming e g. 100iops per HDD, with e.g. 1MiB record size and some prefetching you’d be fine for iops, … depending on what you torrent you might run out of capacity).
Other stuff, whatever, … you don’t have much choice wrt making durable writes faster, L2ARC will absorb some of the read load for data that’s frequently read, which will free up your spindles / reduce write access latencies somewhat.