New Server FileSystem & Raid Setup Questions

High-Performance Computing Workstations & Servers
1

Hey All,

I am building a new server used as a family Media Streamer and would like some advice on drive setup and file system.

This is for Media (Plex), Game Servers, Ect…

The server is a powerful SuperMicro with a AMD Genova Chip.

Drives
2 x nVME (OS & Transcode Drive for Plex)
12 x 7200 Spindle HD’s for Media

OS Will be Ubuntu 22.04 LTS.

My questions are:

  1. What is the best file system for my use case?
  2. I am thinking Raid 5 with 2 used for Parity. What do you think?

Any feedback would be great.

Thankyou

–
D

1 Like
2

Hi & welcome!

  1. I’m using JFS. But IIRC Ubuntu uses ext4 as default, and it’s fine.
  2. RAID5 with dual parity is called RAID6. Your Genua chip is powerful enough not to suffer a lot from the write penalty that comes with RAID6 parity calculations. Having an NVMe-based L2ARC will reduce this even further.

HTH!

1 Like
3

Hi!

Welcome and have fun!

I have 6 7200rpm disks at present in an ext4 FS in RAID6, one cold standby ready if a disk dies. My goal was write-rarely and read-random, for not knowing which files I wanted to watch/listen/stream, with high sequential read access. I had to tune the LVM2 volume group (VG) and the logical volume (LV) to use 1024KB blocks so that the 100-ish operations-per-second of the disks could be close to the 220-ish maximum MB/s throughput of these HGST DC-350’s. My combination of Linux/LVM2/ext4 couldn’t get hundreds of MB/s of sequential reads (for media streaming) without doing this tuning.

The L2ARC mentioned above is specific to ZFS, you can add faster devices to LVM2 volume groups for caching. I don’t do this because my array is faster than the gigabit network connection it’s on and has that write-rarely and read-random pattern of use, making it not worth my effort to set up and manage a write cache nor worth the effort of finding ‘hot’ data that would benefit from an SSD or NVMe read cache.

If I were building a new array now, I’d look at XFS (built at Silicon Graphics for large high-performance disk arrays) and ZFS-on-Linux.

K3n.

1 Like
4

Thank-you for the feedback.

-=Updated=-

I have made many changes so here are my new specs and file system information:

|MB|SuperMicro H13SAE-MF|
|CPU|AMD EPYC 4584PX|
|CPU Cooling|Dynatron A49|
|RAM|128GB DDR5 - 4 x 32GB 4800Mhz (Crucial CT32G48C40U5.M16A1)|
|GPU|PNY NVIDIA RTX A4000|
|nVME-1|Kioxia KXG80ZN84T09 4TB PCIe Gen4 NVMe (OS)|
|nVME-2|Kioxia KXG80ZN84T09 4TB PCIe Gen4 NVMe (Transcode, Docker)|
|nVME-3 (HBA) |Kioxia CD6-V 2.5" 1.6TB KCD61VUL1T60 SSD (L2ARC/Cache)|
|HBA Card|Broadcom 9500-16i 12Gb/s HBA TriMode|
|HBA Cables|Set F - (2x) 2x SFF 8643 → SFF 8654 [2x Mini-SAS-HD (Backplane) to Slim-SAS]|
|Networking | 10G Mellanox ConnectX-3 CX311A SPF+ |
|PSU|Corsair AX1000 80PLUS Titanium|
|Case|45Drives - HL15|
|HD Array|15 x 22TB IronWolf Pro 7200RPM|

The Array will be ZFS Raidz2. I will have two parity drives out of the total of 15 drive array.

In addition to this I will have a Kioxia CD6 L2ARC/Cache Drive for the VFS array.

This meets my requirements and I was vary careful to make sure everything is compatible.

My question now is can I use one port on my HBA to split off and cover the back-planes on the HL15 case (15 Drives) and the second port for an nVME cache drive?

–
D