Building Custom Kernel with ZFS Built In

timtierney · July 14, 2017, 1:16pm

Pre-NOTE: I’m a total kernel building n00b. Please, please, please comment if I made any errors or could improve the folowing procedure. I just wanted to contribute and help anyone looking to ride the bleeding-edge a little.

My daily driver laptop has Ubuntu 17.04 installed with the OS running on a ZFS root pool on my m.2 SSD (120GB). I also have a zpool on a second HDD (500GB). With the release of kernel 4.12, I was interested in trying it out. Basically as a learning experience building the kernel. I know if I didn’t build it with ZFS enabled, the OS wouldn’t even be able to get to GRUB during boot. Leaving the machine in an unbootable state.

So I spun up a VM (in Virtualbox) with the same file system layout to test it. NOTE: I’m following the procedure below as I write this so I’m 100% sure it works. I’ll be using Ubuntu 17.10 Artful Daily ('cause why not. ).

TL;DR: Here’s the procedure.

Preface (OS Setup)

Start a VM installing Ubuntu onto a ZFS root pool. Following this procedure. The system has two pools.

ssdpool: 120GB SSD (Ubuntu OS)
hddpool: 500GB HDD (Empty Zpool)

Install Kernel Build Software

Install required dependencies for building the kernel. `sudo apt install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache`

Install dependencies for building SPL and ZFS
sudo apt install build-essential autoconf libtool gawk alien fakeroot linux-headers-$(uname -r) sudo apt install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libdevmapper-dev sudo apt install parted lsscsi ksh

Download Kernel, SPL, and ZFS Source

`mkdir ~/src && cd ~/src` `git clone -b linux-4.12.y git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git git clone https://github.com/zfsonlinux/spl git clone https://github.com/zfsonlinux/zfs`

Copy Working Kernel Config and Prepare

`cd ./linux-stable cp /boot/config-"$(uname -r)" .config yes '' | make oldconfig make prepare scripts`

Configure and Build SPL

`cd ~/src/spl git checkout master sh autogen.sh ./configure --prefix=/ --libdir=/lib --includedir=/usr/include --datarootdir=/usr/share --enable-linux-builtin=yes --with-linux=/home//src/linux-stable --with-linux-obj=/home//src/linux-stable ./copy-builtin /home//src/linux-stable make sudo make install`

Configure and Build ZFS

`cd ~/src/zfs git checkout master sh autogen.sh ./configure --prefix=/ --libdir=/lib --includedir=/usr/include --datarootdir=/usr/share --enable-linux-builtin=yes --with-linux=/home//src/linux-stable --with-linux-obj=/home//src/linux-stable --with-spl=/home//src/spl --with-spl-obj=/home//src/spl ./copy-builtin /home//src/linux-stable make sudo make install`

Enable SPL/ZFS & Build The Kernel

`cd ~/src/linux-stable make menuconfig`

Enable the following (pressing ‘Y’ to include)

Then navigate into “File systems” and activate ZFS.

Press tab to navigate to “Exit” and press enter. Then press tab to navigate to “Save” and press enter overriding “.config”. Then press tab to navigate to “Exit” and press enter to exit.

make clean make -j <number of CPU cores> deb-pkg LOCALVERSION=-custom

Install Kernel DEB Packages

NOTE: Prior to installing the kernel, do a ZFS snapshot of your operating system. If the system becomes unbootable, boot up a live CD/USB and do a rollback.

Inside the ‘src’ directory should be several installable DEB packages. Install them one-by-one with the following commands:
cd .. sudo dpkg -i linux-firmware-image-4.12.1-custom_4.12.1-custom-1_amd64.deb sudo dpkg -i linux-libc-dev_4.12.1-custom-1_amd64.deb sudo dpkg -i linux-headers-4.12.1-custom_4.12.1-custom-1_amd64.deb sudo dpkg -i linux-image-4.12.1-custom-dbg_4.12.1-custom-1_amd64.deb sudo dpkg -i linux-image-4.12.1-custom_4.12.1-custom-1_amd64.deb

Reboot and you should be on the latest LInux kernel with ZFS support.

Resources

https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.10-Root-on-ZFS https://www.maketecheasier.com/build-custom-kernel-ubuntu/ https://github.com/zfsonlinux/zfs/wiki/Building-ZFS http://www.linuxquestions.org/questions/linux-from-scratch-13/%5Bhow-to%5D-add-zfs-to-the-linux-kernel-4175514510/

Kat · July 14, 2017, 2:14pm

The only that wasn’t mentioned here that should be done as a precautionary measure is, making your initramfs and updating your grub-config file.

reavessm · July 14, 2017, 2:24pm

I assume this is a typo, as I could not find a package named install. Just lettin ya know

anon75264233 · July 14, 2017, 2:41pm

then you are using an older version of apt.
You will need to append all his apt commands with apt-get.

anon75264233 · July 14, 2017, 2:43pm

also important thing to note that I don’t think the OP did, is that in the kernel config all things marked with an M next to them, are compiled as module’s for the kernel, but certain things need to be compiled directly into it, these are marked as a *.

Kat · July 14, 2017, 2:52pm

The TL;DR of compiling a Kernel.

Download a kernel source. Whether it be available from the official repos.
CD /use/SRC/linux-4.x.x (whatever kernel you are compiling. If you don’t know type in ls /use/src)
sudo make clean && sudo make mrproper
sudo make menuconfig
add in feature you want into your kernel. Important driver based features, modulize them instead of building them in for convenience)
sudo make
sudo make modules_install
sudo make install

Now generate your initramfs

mkinitpcio -p linux-4..x.x
Then update grub-config
sudo grub-mkconfig -o /boot/grub/grub.cfg
reboot

anon75264233 · July 14, 2017, 2:59pm

accept for things like filesystems, and a few others I can’t remember. These need to be in there directly or the system won’t boot*.

unless you use a ramfs.

timtierney · July 14, 2017, 3:28pm

Thanks for the tip. I did notice as it installed, it did that stuff automatically. Guess it couldn’t hurt to do it manually after everything is done though.

timtierney · July 14, 2017, 3:29pm

Yep. Fail! It’s fixed.

Kat · July 14, 2017, 3:30pm

Some distros won’t update grub automatically that’s why it’s good to do as a precautionary measure.

timtierney · July 14, 2017, 3:46pm

The other thing I was worried about. What if Ubuntu pushes a zfs update (like a bug fix)? Would it affect this custom kernel if it was modular? If Ubuntu’s update essentially installed an older version of ZFS and you tried to reboot, could it become unbootable?

I might be wrong here. Not sure how modules work.

SgtAwesomesauce · July 14, 2017, 7:25pm

A little tip, instead of using the <username> placeholder, you can use $(whoami) and no user intervention in the command is needed!

I’m pretty sure dpkg sorts that out when it installs the kernel packages, but that’s definitely something that should at least be touched on.

anon75264233 · July 15, 2017, 3:31am

or $USER

its shorter and its what I remember.

anon42298116 · August 23, 2017, 10:17am

Thank you this was very helpful

hondaman · January 18, 2018, 8:42pm

Just found this and want you to know it was exactly what I was looking for. Thank you.

MLWALK3R · February 7, 2019, 10:31pm

Could this get updated for the upcoming 8.0 release since SPL has been merged into the ZFS repo would be much appreciated

2bitmarksman · February 7, 2019, 11:16pm

This looks really interesting. Bookmarking this as a project for later

Suleiman · February 8, 2019, 12:36am

Kernel config looks a bit old. This is what you will need for kernel up to 4.20 …

$ zcat /proc/config.gz | grep -i zlib
CONFIG_SQUASHFS_ZLIB=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y

When in menuconfig you can use / to search for the three configs and dependency, and don’t install ZLIBs as module.

Just my two cents.

timtierney · April 23, 2019, 8:39pm

I’d be happy to do that. I’ve been doing this with kernel 5.0.* and ZFS 0.8.* so it’s pretty fresh.

timtierney · April 24, 2019, 5:33pm

Just for you my friend…