You hint at a very diverse set of storage requirements that benefit from tuning and proper storage selection.
You will find a lot of passionate zfs fans because zfs allows very detailed tuning to different workloads, often even within a single storage pool.
Let me start to translate your use cases into proper technical requirements for review and discussion. Then I’ll propose solutions again for discussion.
Use case 1: media storage on home server
Characteristics:
- Low access concurrency, if any
- WORM (write once, read many) access pattern
- Mostly, even exclusively large files (GB+), data access pattern mostly sequential (copying, reading)
- Media files are typically not effectively compressible (media can include video, audio, image formats)
- No indication that media files will share identical blocks, so no candidate for deduplication
- Relatively large storage pool (tens of TB)
- Archival in nature - redundancy critical to prevent data loss
Use case 2: torrent storage
Let’s assume this is used for legal and ethical purposes, meaning downloading and seeding of torrents. I assume that this use case involves mostly compressed data files (media or compressed containers).
Characteristics:
- High access concurrency
- WORM (write once, read many) access pattern
- Mostly, even exclusively large files (GB+) , data access pattern highly random both for writing and reading
- Media files are typically not effectively compressible
- No indication that media files will share identical blocks, so no candidate for deduplication
- Storage longevity: relatively temporary. Download, seed for a limited amount of time, then replace with new torrent. Requirements for data redundancy are probably lower than in first use case.
- Storage pool size probably < 1TB, maybe single digit TBs
Implementation proposals
Use case #1
- Use relatively cheap storage medium. Existing hdds are great.
- Performance requirements are relatively low due to lack of expected concurrency; watching, listening to media does not require lots of bandwidth.
- To minimize cost I’d recommend a RAIDZ configuration (as opposed to pool of mirrors). The exact configuration depends on your risk level. I’d make pools of identical drives (18TB into a separate pool from 6TB drives).
- The general recommendation is not to have too many drives per RAIDZ (I think 6-12). With your existing set of drives configuring three RAIDZ of 6 drives makes sense to me.
- Personally, I did not have any issues with RAIDZ1 configurations for media storage (again to minimize cost). There are significantly differing opinions out there, including in this thread. Consider higher RAIDZ levels depending on your risk profile.
- I understand DRAID to be mostly beneficial for pools with significantly larger drive numbers in enterprise situations. I don’t have hands-on experience with DRAID due to that.
- HDDs accel at sequential access patterns. To keep them accessing mostly large continuous sections of data consider adding a “special device” configuration for metadata and the small amount of small block sizes. SSDs, especially Optane SSDs are great for that and currently relatively cheaply available. Search this forum for discussion of zfs special devices.
- Configure storage pool without deduplication and compression
- Follow general best practices when creating pool (4k alignment, noatime, nodiratime, etc.)
Use case 2:
- Relatively high performance requirements with random access patterns of small sector sizes indicate SSDs as perferred storage medium.
- Relatively small amount of required storage makes SSDs affordable.
- Number of SSDs depends on exact performance and storage pool size requirements. 1 SSD may be sufficient.
- I’d consider this pool to be completely temporary and therefore expendable. I’d configure multiple SSDs in a RAID0 configuration to maximize performance. Consider mirrored or RAIDZ configurations based on your risk profile.
- Assuming lots of writes watch for wear on SSDs. Consider enterprise class NAND or Optane drives. Gen 1 Optane drives are on sale at the moment.
- Don’t forget to turn on auto trim when setting up storage pool.