Arch Linux ZFS Custom config Question

I am getting ready to get my system built and configured. I do However have a question. I am pretty sure this is possible, However when I search for ZFS, the instructions are for ZFS to take over the entire system, ZROOT etc. My Filesystem config I would like to see is this

/boot CPU NVME
/swap CPU NVME
/root (ext4) CPU NVME
/home (ext4) CHIPSET NVME

/dev/sda1 sda2 sda3 sda4 (ZFS Filesystem) or something like that

I want to buy 4 hard drives for my onboard Sata that is on a Gigabyte Aorus Master X570S, and have ZFS on only those volumes. Is this Possible? On the motherboard I have 6 SATA (SATA0 - SATA5)

Thank you in advance.

PS. I know after the install I need to reconfigure the kernel and add the ZFS-utils. But do I need the entire system ZFS? or can I Just have the SATA Drives ZFS?

1 Like

Yes, ZFS will coexist with any other filesystems you have already, it does not need to be the root filesystem.

It is a little strange in the way you don’t use /etc/fstab to configure mounts, but the configuration of zfs mountpoints using the zfs utility works alongside the normal /etc/fstab way. If you prefer doing everything from /etc/fstab then you can zfs set mountpoint=legacy pool/dataset and then ZFS won’t manage mounting, then add ZFS datasets to /etc/fstab yourself.

In the Arch Linux wiki ZFS page, you should skip all the instructions referring to setup of the initramfs with ZFS hooks (don’t need to modify /etc/mkinitcpio.conf and re-generate the initramfs). Otherwise you’ll probably end up with a boot delay or even failure while it is looking for a ZFS dataset to use as the root filesystem. If you accidently end up in that position, use your bootloader menu / edit the bootloader config to add disablehooks=zfs to the kernel parameters.

I’ve had most success with an LTS kernel and the ZFS DKMS package on Arch (pacman -S linux-lts zfs-dkms zfs-utils) - no issues with kernel updates and ZFS updates. Use of the bleeding-edge kernel and binary ZFS package can have issues because the ZFS support for kernels sometimes lags behind a version.

If you’re not on an LTS kernel and don’t want to change kernel version, I’d start with pacman -S zfs-dkms zfs-utils, create a partition on each disk for ZFS, then zpool create with the partition names, passing -m <mountpoint> for the root dataset mount-point, where-ever you wish to mount it. Datasets within the pool are default mounted under that root, but can be mounted anywhere else using zfs set mountpoint=<mountpoint> pool/dataset (or the special legacy keyword to manage with /etc/fstab).

zpool create should create partitions for you if you pass it an entire disk as a parameter, but I like creating partitions myself to be sure :slight_smile: The partition code is officially bf/bf00 (Solaris root), but using the Linux data code 83/8300 I think is recommended for Linux installs.

You could add a small (<1GB) UEFI ESP partition to each disk and copy your UEFI loader / kernel / initramfs to each one, in case you need to boot from the SATA disks for some reason (NVMe SSD goes whoopsie one day).

So if you have SATA disks /dev/sda…sdd , it should just be a case of:

striped pool - fastest - No redundancy

zpool create -m /mnt/pool mypool /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

raidz1 pool - slower - but any single disk can be lost without data loss

zpool create -m /mnt/pool mypool raidz1 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

striped mirrors - fast and safe - any single disk from each mirror can be lost (2 disks worth of reundancy)

zpool create -m /mnt/pool mypool mirror /dev/sda1 /dev/sdb1 mirror /dev/sdc1 /dev/sdd1
1 Like

Like many on the forum here I am a big fan of ZFS. However, I don’t use ZFS for key OS partitions.
It’s not that this is not technically possible - there is documentation on how to setup root on ZFS.
The adversarial relationship of OpenZFS and the Linux kernel (due to a difference in licensing) IMHO does not make ZFS the best choice for the root file system.
I run my systems relatively close to the cutting edge (Fedora) and experience once in a while that the newest kernel version changed something that prevents the ZFS modules to work requiring me to wait for a ZFS update to overcome this situation.
Things have gotten better on this front and I am a hopeful optimist that the bigger picture will change in the future. But until then ZFS will be used as storage for specific use cases/applications in my home lab.

Looking at your specific configuration, even a single SATA SSD as boot/root drive will completely outperform a ZFS setup consisting on 4 SATA HDDs.
On modern system like yours (X570S based mobo) I would try to run as much data from NVMe with HDDs relegated to cold-tier high-capacity storage role. ZFS enables such a setup quite nicely.
E.g. 1x NVMe for boot/root, 4x HDD for ZFS supported by 1+NVMe for special device, ZIL, L2ARC depending on your use cases and budget.

1 Like

@xzpfzxds

You are awesome, thank you for the information.

This ZFS is a “Playground Experiment” I found Kingston A400 SSD at 240GB for 26.00 CDN each, I will buy 4 of them and I want to see and play around with a few configs (Or maybe buy and extra two for a total of 6)

I want to try playing with raidz2 for ZFS and see how it goes. I will be looking closer at your response, and giving it a good read.

Thank you so much for the help

You are awesome.

1 Like