I’m building a home server for my whole family, I needed some help with respect to the pools and vdev. I have 8x20tb HDDs and 2x1tb sata ssd and 256gb boot nvme. The NAS will be used for storing mostly media (photos,videos) and some backups of phone, documents, etc.
What would be the optimal layout that I should go for ?
I am thinking using both SSDs for cache and rest of the HDDs in raidz2.
Does this seem sound enough ?
I have also heard about metadata but can it be safely kept on single ssd ? that way I can use one for cache and other for metadata, or do I need to have a parity drive for metadata aswell aka both ssds will be for metadata ?
P.S : this is my first time doing anything related to NAS, hence the topic
There is no optimal layout. There is only compromises
special metadata vdev needs redundancy same as your data vdevs. So a mirror of both SSDs will give you 1TB for metadata and optionally also small files. Or you use both SSDs as L2ARC. But you can’t do both with just two drives. A third NVMe as L2ARC would cover both types.
Media doesn’t use that much metadata because you can operate at high recordsizes, but ~100TB will still have substantial metadata regardless, so you better take care of this with memory and optional L2ARC/special vdev.
For pool configuration…either 8-wide RAIDZ2/3 or 2x 4-wide RAIDZ. The latter trades some redundancy for performance and easier expansion. Resilver time on full 20TB disks may feel uncomfortable, so people like having two or three disks of redundancy.
There is also the option to go for striped mirrors, which have the lowest capacity, shortest resilver time, good performance and easy expansion with 1-4 drives being able to fail.
Take a look here. Wendell takes a deep dive into Metadata disks. I’ve tested and used this and it works great with HDD’s.
As for pool layout, I agree with Exard3k. My personal choice would be Z2 or Z3. I don’t know how fast you’re planning to fill it up but 120TB should be enough for the coming 5 years, maybe more.
I’m far from a ZFS expert, you can see all the threads I’ve made asking for advice, but here is my two cents
I used to have a NAS that I wanted to store everything on, all my important stuff like pictures, documents etc, everything you described PLUS bulk media like movies
I quickly learned that no NAS really fit what I needed, and that’s when I realized I was the problem, not the NAS. So I moved my bulk media onto a separate NAS and built something as reliable and flexible as possible to store the data you are storing.
I went with three pairs of mirrors, which just makes it SO much more flexible. Then I have 2 x SSD’s for metadata which I’ve found helps greatly going through large image library’s, a 1TB NVMe L2ARC, and 64GB of RAM. So far I am very happy, and I feel like I can keep going with this same NAS pretty much forever, because of how flexible mirrors are
I know it’s more expensive than RAIDZ, but I would highly suggest mirrors just because of how easy they are.
For my second NAS that has the less important bulk media, I made a big ol RAIDZ2, no L2ARC, no metadata cache, and much less RAM. Performance is meh, but who cares? PLEX still works
How much bulk unimportant media do you have? I suggest making either 2 NAS’s or at least 2 pools. One for genuinely important data you access, and a second for the bulk junk. ESPECIALLY if your family is relying on this NAS
Thanks for the replies everyone, I have went ahead with 8 HDDS in raidz2 and metadata route.
Now, I wanted to know if I can add the metadata SSDs later on after the pool has been created, because the pci to sata HBA that I ordered was faulty so it will take a week for the replacement to arrive, in the meantime I thought maybe I should start building the pool and start migrating the data.
Does this approach seem fair enough ?
Unless you’re on faster than 1 gig Ethernet to everyone speed is unlikely to be a factor. Cache doesn’t make a lot of difference with small numbers of users as they don’t typically keep pulling the same content.
So your layout will be constrained by resiliency requirement vs. capacity. Unless your family has 10+ users all hitting this box hard at the same time you’re already likely overkill on performance without ssd for cache etc.
Curious, are these SAS drives? Are they in an external enclosure or internal backplane or combination?
If enclosure how many ports and what speed?
What controller are you using and how many ports?
The amount of port or SAS lanes available to use is likely the widest you would want to go.
