Installing Manjaro on encrypted ZFS root pool!

Wikis & How-to Guides
, ,
1

THIS PAGE IS OUTDATED, PLEASE VISIT LINK FOR MORE UP TO DATA INSTRUCTIONS

The following instructions should guide you through setting up Manjaro on an encrypted ZFS pool.

Boot an up-to-date Manjaro ISO on the system you want to install on and open a terminal. This guide will only work on a computer booting in UEFI mode.

create a short handle to reference the ssd by id e.g.:

DISK=/dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_123ABCDEF12345G

Now we format the disk into a 1GB EFI partition, a ZFS root pool filling everything remaining except the last 16GB, and the last 16GB on disk we will use for an encrypted swap partition. If you prefer not to use an encrypted swap partition you are free to modify the following commands here, and later in the guide when we set it up. If you need guidance on how large your swap partition should be have a look at: Swap space on Linux systems: A primer | Opensource.com

sudo sgdisk --zap-all $DISK
sudo sgdisk -n1:0:+1G -t1:EF00 $DISK 
sudo sgdisk -n2:0:-16G $DISK     
sudo sgdisk -n3:0:0 $DISK

Warning

If you prefer using a swap file, I want to point out that it is not recommended to put a swap file on the root pool or any ZFS pool for that matter. The practise of having a swap file in a zfs pool has to be shown to cause problems and crashes.

Create a random UUID to append to the zfs pool names so that importing the pools on other computers e.g. for rescue or backing them up won’t result into name conflicts!

poolUUID=$(dd if=/dev/urandom of=/dev/stdout bs=1 count=100 2>/dev/null |tr -dc 'a-z0-9' | cut -c-6)

Create the root pool with the setting recommended in the Arch wiki:
https://wiki.archlinux.org/index.php/Install_Arch_Linux_on_ZFS#Setup_the_ZFS_filesystem
I want to point out that if you want to setup a more complex disk layout with ZFS your chance to do so is here in modifying to following command e.g. to setup a mirrored layout!

sudo zpool create \
	-f -o ashift=12 -o autotrim=on \
	-O acltype=posixacl       \
    -O relatime=on            \
    -O xattr=sa               \
    -O dnodesize=legacy       \
    -O normalization=formD    \
    -O mountpoint=none        \
    -O canmount=off           \
    -O devices=off            \
    -R /mnt                   \
    -O compression=lz4        \
    -O encryption=aes-256-gcm \
    -O keyformat=passphrase   \
    -O keylocation=prompt     \
    rpool_$poolUUID ${DISK}-part2

Now we are going to create the datasets. Nothing is stored directly under rpool, hence: canmount=off! canmount is set to noauto for root as advised here: Install Arch Linux on ZFS - ArchWiki

sudo zfs create -o canmount=off -o mountpoint=none rpool_$poolUUID/HOME
sudo zfs create -o canmount=off -o mountpoint=none rpool_$poolUUID/ROOT
sudo zfs create -o mountpoint=/ -o canmount=noauto rpool_$poolUUID/ROOT/manjaro
sudo zfs mount rpool_$poolUUID/ROOT/manjaro
sudo zfs create -o mountpoint=/home rpool_$poolUUID/HOME/manjaro
sudo zfs create -o mountpoint=/root rpool_$poolUUID/HOME/manjaro/root
sudo zfs mount -a

Formatting and mounting the EFI partition.

sudo mkfs.vfat -n EFI $DISK-part1
sudo mkdir /mnt/boot
sudo mount $DISK-part1 /mnt/boot

Configuring the root filesystem as advised here:
https://wiki.archlinux.org/index.php/Install_Arch_Linux_on_ZFS#Configure_the_root_filesystem

sudo zpool set bootfs=rpool_$poolUUID/ROOT/manjaro rpool_$poolUUID
sudo zpool set cachefile=/etc/zfs/zpool.cache rpool_$poolUUID

Now the installation procedure will start, run the “setup” command in the terminal to start the interactive installer. In the following section I will give you hints which menu points you are required to use. At the time of writing this guide to most up-to-date linux kernel is Kernel 5.10 (package linux510), if you follow this tutorial at a later point you will need to adjust some parts of the following instructions accordingly:

Prepare Installation -> Set Virtual Console -> change -> <your selection>
Install Desktop System -> Install Manjaro Desktop -> yay + base-devel; linux510 -> <your selection> -> Yes -> cryptsetup; linux510-zfs -> <your selection> -> <your selection>
Install Desktop System -> Configure Base -> Generate FSTAB -> fstabgen -U -p
Install Desktop System -> Configure Base -> Set Hostname
Install Desktop System -> Configure Base -> Set System locale -> <your selection> -> <your selection>
Install Desktop System -> Configure Base -> Set Desktop Keybaord Layout -> <your selection>
Install Desktop System -> Configure Base -> Set Timezone and Clock -> <your selection> -> <your selection> -> Yes -> utc
Install Desktop System -> Configure Base -> Set Root Password 
Install Desktop System -> Configure Base -> Add New User(s) -> <your selection> -> <your selection>
Done -> Yes -> No

Leaving the setup unmounts the zfs datasets and the efi partition so we need to remount them.

sudo zfs mount rpool_$poolUUID/ROOT/manjaro   
sudo zfs mount -a
sudo mount $DISK-part1 /mnt/boot

Copy over the zpool.cache file to the new system

sudo cp /etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache

Now we are going to create the encrypted swap partition. If you are sure you do not want to use a swap partition don’t execute the following commands.

echo " " | sudo tee --append /mnt/etc/crypttab
echo "# encrypted swap partition" | sudo tee --append /mnt/etc/crypttab
echo "swap $DISK-part3 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256" | sudo tee --append /mnt/etc/crypttab
echo " " | sudo tee --append /mnt/etc/fstab
echo "# mounting the encrypted swap partition" | sudo tee --append /mnt/etc/fstab
echo "/dev/mapper/swap none swap defaults 0 0" | sudo tee --append /mnt/etc/fstab

Activating the zfs services like advised in:
https://wiki.archlinux.org/index.php/Install_Arch_Linux_on_ZFS#Configure_systemd_ZFS_mounts

sudo systemctl enable zfs.target --root=/mnt
sudo systemctl enable zfs-import-cache --root=/mnt
sudo systemctl enable zfs-mount --root=/mnt
sudo systemctl enable zfs-import.target --root=/mnt

When running ZFS on root, the machine’s hostid will not be available at the time of mounting the root filesystem. There are two solutions to this. You can either place your spl hostid in the kernel parameters in your boot loader. For example, adding spl.spl_hostid=0x12345678, to get your number use the hostid command.
The other, and suggested, solution is to make sure that there is a hostid in /etc/hostid, and then regenerate the initramfs image which will copy the hostid into the initramfs image. To write the hostid file safely you need to use the zgenhostid command.
The hostid string has to be 8 alphanumeric characters.

manjaro-chroot /mnt zgenhostid
manjaro-chroot /mnt mkinitcpio -P

Now we install the bootmanager!

manjaro-chroot /mnt bootctl install
manjaro-chroot /mnt sudo pacman -S systemd-boot-manager
manjaro-chroot /mnt sdboot-manage gen

Finally we unmount all the disks and then restart into our newly installed system!

sudo umount /mnt/boot
sudo zfs umount -a
sudo zfs umount rpool_$poolUUID/ROOT/manjaro
sudo zpool export rpool_$poolUUID

If you have any questions or need assistance following this procedure please post in this thread.

3 Likes
2

I made some edits to try to clear near the partition setup where it sounded a bit repetitive, and to put emphasis on the ZFS+swap warning being about swap files specifically; on first read through I kept thinking this was a warning about swap of any kind sharing a drive with a ZFS partition.

1 Like
3

Hello [Sapiens] and thanks a lot for your explanation. I want to install Arch-KDE on zroot and start it with rEFInd.

The reason of my decision is (as I read now more than one time) ZFS have some small little problems with GRUB and SWAP-file. If I remember well zroot should be positioned just after EFI-partition hence swap-partition will come at end of disk, right? I know also Ubuntu on ZFS use GRUB and Swap-file but for running this they use a special configuration.

Please don’t take me wrong I just “discover” the Arch world and here my questions:

  1. Command for recognize disk by-id is: ls -la /dev/disk/by-id/ right?

  2. Why not using DISK=/dev/disk/by-id/wwn-0x[+16 digits] or DISK=/dev/disk/by-id/nvme-eui.[+16 digits]? WWN = world wide number, this never change.

  3. For what is the third, 483G 8300 partition for? You use and encrypt only ${DISK}-part2.

  4. Do I need create a random poolUUID if I use wwn or nvme-eui? If yes, can I use something like this?:

poolUUID=$(dd if=$DISK of=/dev/stdout bs=1 count=100 \
2>/dev/null |tr -dc 'a-z0-9' | cut -c-6)
  1. When the system will ask for password? Do this method let ask twice? one by opening of zroot and one by display manager?

Sorry for asking you so many questions but I don’t understand well the explanation in the wiki.

Thanks in advance and greetings.

1 Like
4

Exactly, you can cross reference with a disk or partitioning program like gparted to make sure you have the correct disk.

The reason to use by-id is that handles like /dev/sda are assigned dynamically and can change with every boot. When you use disks for simple storage ZFS will handle their mounting also dynamically and figure out which disks belong together by the metadata on the drives not by their device name. This however does not work for the boot drive as far as I know, so you need to take an identifier that never changes.
WWN notation will work as well, but it has a disadvantage. You will get output like the following from ZFS which will make it harder for you to tell which disk has failed because you would have to translate the WWN to a actual device e.g.

  pool: example-pool
 state: DEGRADED
config:

	NAME                                      STATE     READ WRITE CKSUM
	example-pool                                ONLINE       0     0     0
	  mirror-0                                ONLINE       0     0     0
	    wwn-0x1234567890abcdef				  FAULTED      0     0     0
	    wwn-0x1234567890abcdeg                ONLINE       0     0     0

Whereas with regular IDs you get the model and serial number as output e.g.

  pool: example-pool
 state: DEGRADED
config:

	NAME                                      STATE     READ WRITE CKSUM
	example-pool                                ONLINE       0     0     0
	  mirror-0                                ONLINE       0     0     0
	    ata-TOSHIBA_ABCDEFGHIJ_123456789  	  FAULTED      0     0     0
	    ata-TOSHIBA_KLMNOPQRST_987654321	  ONLINE       0     0     0

$DISK-part3 is the encrypted SWAP partition I setup further down in the instructions. It is the section where i echo some lines into the /etc/fstab. In the example given it has a size of 16GB.

Actually you don’t need the poolUUID at all. The reason why I put it there is simply to avoid name conflicts. Let’s say you name your root pools rpool on all of your machines. Now one machine has troubles and you try to rescue one machines rpool on another machine. You now won’t be able to import the rpoolsince a pool with that name already exists on the machine and pool names have to be unique. You can`t have two pools with the same name on the same computer. To avoid problems like these, especially for the root pool it makes sense to append a random string to the name so naming conflicts become very unlikely. This is also everything the poolUUID command does. You can execute the following command in your terminal to see what it does. It does nothing more than to create a random 6 symbol long string.

dd if=/dev/urandom of=/dev/stdout bs=1 count=100 2>/dev/null |tr -dc 'a-z0-9' | cut -c-6

Exactly. It will first ask during the boot process to decrypt your pool and then the login manager will ask again for your Linux users password. I would advise to use a longer and secure password for the encryption.

5

Hi Sapiens, Thanks for the guide it came at the perfect time to coincide with me setting up a manjaro system with encrypted zfs on root!

FYI I ran into a slight issue when running the mkinitcpio -P while chrooted, the command produced an error saying something to the effect of/proc was not available. I found this similar sounding issue:
https://bugs.archlinux.org/task/26344
and ran the following commands then mkinitcpio -P worked for me

mount -t proc proc /proc
mount -t sysfs sys /sys
mount -t devtmpfs udev /dev
6

I don’t really understand how this would throw an error. For what reason have you been chrooted while running this command? Have you followed a deviating set of instructions?