LUKS on boot/root partition with software RAID1?

[lapsio]

So finally (after around 6 months since I built it on hardware level) I found some time to finish setting up my workstation on OS level. For many years I’ve been using full disk encryption with LUKS. I have some notes that I scribbled years ago that more or less describe process of creating such setup on Arch Linux and it goes as follows:

cryptsetup -v --type luks1 -s 512 -h sha512 -i 5000 --use-random -y luksFormat  /dev/sdXY
cryptsetup luksOpen /dev/sdXY DISKNAME

pvcreate /dev/mapper/DISKNAME
pvscan ; pvdisplay
vgcreate MACHINE /dev/mapper/MACHINE
vgdisplay
lvcreate -L SIZE -n NAME MACHINE #size 10M or +100%FREE #repeat for root and swap
lvdisplay

mkfs.btrfs -L MACHINE -m dup /dev/mapper/MACHINE-NAME
mkswap -L swap /dev/mapper/MACHINE-swap
swapon /dev/mapper/MACHINE-swap
mount -t btrfs -o noatime,autodefrag,compress=lzo,space_cache,ssd /dev/mapper/MACHINE-NAME /mnt

pacstrap /mnt base linux-hardened grub nano fish openssh sudo btrfs-progs htop screen
genfstab -U /mnt >> /mnt/etc/fstab

arch-chroot /mnt

nano /etc/mkinitcpio.conf #add lvm2 encrypt hooks
nano /etc/default/grub #add GRUB_ENABLE_CRYPTODISK=y and GRUB_CMDLINE_LINUX="cryptdevice=/dev/sdXY:lvm"

To people who did set up something like this on Arch I believe it should look quite familiar, despite not-so-elaborate comments The crucial part are two last lines which implement GRUB stuff (things above are rather to give context about my partitioning scheme)

And well yeah it somewhat works, GRUB stage 1 asks me for password to decrypt root partition (to access /boot which is on the same partition) then vmlinuz asks for password again to decrypt root partition and continue booting. With EFI iirc it’s a little bit more convoluted since iirc /boot/efi simply cannot be encrypted on software level, period. I know there are hacks to make it ask only once for password but I never felt urge to bother.

However I’m not really sure how to make it work when my /boot partition (and whole root partition) will be dual-device LUKS. I’m always using btrfs software RAIDs in various configs so I’d also like to go this route this time and I have two NVME SSDs. Typically for non-boot RAID arrays I’m simply encrypting multiple devices on block level (/dev/sdX) with the same LUKS header. Which of course asks me for passwords as many times as I RAID members but it’s, ok I don’t mind. Then btrfs simply does magic and finds all decrypted drives and mounts RAID.

Is it possible to tell GRUB and vmlinuz to decrypt multiple devices in order to access /boot and root partition? I’m also not entirely sure what to do with SWAP and efi partitions… I believe there’s no much sense in making SWAP RAID1 and i believe it’s not possible to set up efi partition in software RAID at all?.. And I can’t mirror only part of drive in hardware right?

[xradeon]

It’s not quite the same thing, but maybe this will give you some ideas:

Would it be possible to setup arch with luks over lvm across 2 disk and a encrypted boot and detached luks header?

I do something similar with Pop_OS (just with RAID1 instead). I followed these guides: Generally I followed the first guide, but since I was installing Pop_OS 22.04, there are times where you need to jump the second guide to get the right settings (like the fstab mount options). You can do RAID0 with this by just converting BTRFS to RAID0 (kee…

[oO.o]

@lapsio

I have some notes that I scribbled years ago that more or less describe process of creating such setup on Arch Linux

I did this but in a much more involved way (albeit no EFI or btrfs). Parts of it may help you though:

@lapsio

Is it possible to tell GRUB and vmlinuz to decrypt multiple devices in order to access /boot and root partition?

Yes, you need to configure mkinitcpio for this, and if there are additional encrypted drives, you use crypttab.

@lapsio

efi simply cannot be encrypted on software level, period

Correct, it’s always a no frills FAT partition.

@lapsio

it’s not possible to set up efi partition in software RAID

There are hacks, but generally correct. Put the EFI on an arbitrary mirror member and make sure the system is set to boot from that disk. This is not the case for traditional bios systems and is a clear disadvantage for EFI.

If you really want to, make EFI partitions on both disks and back one up to the other after updates (that’s the hack).

@lapsio

there’s no much sense in making SWAP RAID1

See the swap config in my github link, it’s a novel way to do it with encryption.

[lapsio]

Unfortunately I didn’t have much time to test that so far but today I finally managed to set this up - both on OpenSUSE and Arch.

I partially followed those POP_OS! tutorials mentioned in first answer, that said they unfortunately didn’t cover initrd generation part.

So I’ll start from OpenSUSE since it’s gonna be short - if anyone didn’t know OpenSUSE graphical installer is absolutely f*cking mental and creating such setup is non-issue for it. I just created two separate LUKS encrypted partitions on two drives with LVM on them and then set up btrfs raid1 with all the bells and whistles using installer.

But that was easy part and easy stuff is always boring so then I tried to replicate this setup under arch and it was quite challenging

1. Standard Arch linux encryption tutorials use encrypt mkinitcpio hook which doesn’t apparently support multiple LUKS partitions - you have to use systemd and sd-encrypt hooks for initramfs in order to even begin
2. Most articles that I found on Arch wiki were quite rough and mentioned putting drives in /etc/crypttab but omitted information that it actually has to be /etc/crypttab.initramfs in order to get properly executed on initramfs level.
3. With multi-LUKS setup it’s recommended to entirely omit kernel parameters and GRUB options in /etc/default/grub (apart from GRUB_ENABLE_CRYPTODISK that is) so there’s no issue with cryptdevice params since they shouldn’t be used at all. With sd-encrypt hook you completely rely on crypttab (or rather crypttab.initramfs)

So basically in order to make it work you only have to enable
GRUB_ENABLE_CRYPTODISK=y
in /etc/default/grub and nothing else on GRUB side, while on initramfs side just follow those articles for sd-encrypt method:
https://wiki.archlinux.org/title/Dm-crypt/System_configuration#Using_systemd-cryptsetup-generator

I had unreasonably high number of issues with enabling systemd in initrd due to some missing fonts and other bs, that’s mostly why it took me almost half a day but overall assuming you won’t run into some dumb problems like me, it’s not all that complicated.