Yet another ZFS drive layout thread

Hardware HDD's & SSD's
1

In a question as old as time I’d like to pick the forums collective brain on how best to layout the hardware I have for my use case with ZFS.

I have the following disks on hand:

  • 4x 20tb hdd
  • 3x 18tb hdd
  • 3x 4tb ssd (sata crucial mx500)

My use case is I’m a 1 man band video creator. I am often ingesting Prores footage over a 10gig network link via Samba. A single project can easily get to be 1tb in size over just a few hours worth of footage. I am chasing consistent write speeds getting as close to maxing out that network link as possible. I’ll also be editing from the pool too.

The system is an AMD Epyc 7402 with 256GB RAM.

I have been going back and forth on the right layouts the past couple of days. A 7 wide RaidZ2 with the spinners gave me just about 10gig line speeds on the network with writes, and seemed to max out reads without too much of a problem (this was a mix of the 18s and 20tb drives for testing purposes - yes i know mixing sizes is ill advised).

What I’m seriously considering is the 3x 4tb drives as a separate pool in Raid Z1 (or buying a 4th and doing two sets of mirrored vdevs in a single pool), doing all my ingestion, editing and whatnot there and then moving over to long term storage when I’m done with the 4x 20tb drives in some likely mirrored vdev or z2 configuration as well. That would free up the 3x 18s for their original purpose of going into the Linux ISOs box I have.

Typically, I’ll only have a couple of projects on the go at one time and my “working space” need only be in that 4-8tb range afforded by the Sata SSDs.

It’s actually such a fascinating topic. Async vs sync writes. RAM cacheing affecting benchmarking. Future proof expansion plans. Etc etc etc. It’s such a rabbit hole and I would appreciate just talking to some of my closest internet strangers about it at length :slight_smile:

1 Like
2

I really like the idea of using the 3 (or 4) SATA SSDs as “scratch” space for your ingestion of videos and then video-editing.

You could set up some sort of periodic sync task to push the contents of the SSD pool over to your pool of HDDs. After building some faith in this sync task, I would be tempted to yell “YOLO!” at the top of my lungs, destroy the SSD pool, and replace it with a stripe the SSDs.

My brainstorm is less sane and more risky than yours, but I’m confident in my brainstorm’s viability!

Your HDD pool isn’t ill-advised, either. You’re getting a good amount of redundancy with raidz2. While 7x20TB would be Ivory Tower ideal, your layout is 90% of the usable capacity. Obtaining or exceeding that missing 10% would be expensive in terms of money (buying more storage) and/or accepting more risk (a less redundant layout).

3

You can make a 4x2tb partitions at the end of those 20TB drives, and stick lancache there or other junk you can wipe any time without checking and with 0 regrets.