Building Custom Kernel with ZFS Built In

Operating Systems & Open Source Linux
1

Pre-NOTE: I'm a total kernel building n00b. Please, please, please comment if I made any errors or could improve the folowing procedure. I just wanted to contribute and help anyone looking to ride the bleeding-edge a little. :smile:

My daily driver laptop has Ubuntu 17.04 installed with the OS running on a ZFS root pool on my m.2 SSD (120GB). I also have a zpool on a second HDD (500GB). With the release of kernel 4.12, I was interested in trying it out. Basically as a learning experience building the kernel. I know if I didn't build it with ZFS enabled, the OS wouldn't even be able to get to GRUB during boot. Leaving the machine in an unbootable state.

So I spun up a VM (in Virtualbox) with the same file system layout to test it. NOTE: I'm following the procedure below as I write this so I'm 100% sure it works. I'll be using Ubuntu 17.10 Artful Daily ('cause why not. :stuck_out_tongue:).

TL;DR: Here's the procedure.

Preface (OS Setup)

Start a VM installing Ubuntu onto a ZFS root pool. Following this procedure. The system has two pools.

  • ssdpool: 120GB SSD (Ubuntu OS)
  • hddpool: 500GB HDD (Empty Zpool)

Install Kernel Build Software

Install required dependencies for building the kernel.
sudo apt install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache

Install dependencies for building SPL and ZFS
sudo apt install build-essential autoconf libtool gawk alien fakeroot linux-headers-$(uname -r)
sudo apt install zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libdevmapper-dev
sudo apt install parted lsscsi ksh

Download Kernel, SPL, and ZFS Source

mkdir ~/src && cd ~/src
git clone -b linux-4.12.y git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
git clone https://github.com/zfsonlinux/spl
git clone https://github.com/zfsonlinux/zfs

Copy Working Kernel Config and Prepare

cd ./linux-stable
cp /boot/config-"$(uname -r)" .config
yes '' | make oldconfig
make prepare scripts

Configure and Build SPL

cd ~/src/spl
git checkout master
sh autogen.sh
./configure --prefix=/ --libdir=/lib --includedir=/usr/include --datarootdir=/usr/share --enable-linux-builtin=yes --with-linux=/home/<username>/src/linux-stable --with-linux-obj=/home/<username>/src/linux-stable
./copy-builtin /home/<username>/src/linux-stable
make
sudo make install

Configure and Build ZFS

cd ~/src/zfs
git checkout master
sh autogen.sh
./configure --prefix=/ --libdir=/lib --includedir=/usr/include --datarootdir=/usr/share --enable-linux-builtin=yes --with-linux=/home/<username>/src/linux-stable --with-linux-obj=/home/<username>/src/linux-stable --with-spl=/home/<username>/src/spl --with-spl-obj=/home/<username>/src/spl
./copy-builtin /home/<username>/src/linux-stable
make
sudo make install

Enable SPL/ZFS & Build The Kernel

cd ~/src/linux-stable
make menuconfig

Enable the following (pressing 'Y' to include)

Then navigate into "File systems" and activate ZFS.

Selection_005.png580x592 48.1 KB

Press tab to navigate to "Exit" and press enter. Then press tab to navigate to "Save" and press enter overriding ".config". Then press tab to navigate to "Exit" and press enter to exit.

make clean
make -j <number of CPU cores> deb-pkg LOCALVERSION=-custom

Install Kernel DEB Packages

NOTE: Prior to installing the kernel, do a ZFS snapshot of your operating system. If the system becomes unbootable, boot up a live CD/USB and do a rollback.

Inside the 'src' directory should be several installable DEB packages. Install them one-by-one with the following commands:
cd ..
sudo dpkg -i linux-firmware-image-4.12.1-custom_4.12.1-custom-1_amd64.deb
sudo dpkg -i linux-libc-dev_4.12.1-custom-1_amd64.deb
sudo dpkg -i linux-headers-4.12.1-custom_4.12.1-custom-1_amd64.deb
sudo dpkg -i linux-image-4.12.1-custom-dbg_4.12.1-custom-1_amd64.deb
sudo dpkg -i linux-image-4.12.1-custom_4.12.1-custom-1_amd64.deb

Reboot and you should be on the latest LInux kernel with ZFS support.

Resources

https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.10-Root-on-ZFS
https://www.maketecheasier.com/build-custom-kernel-ubuntu/
https://github.com/zfsonlinux/zfs/wiki/Building-ZFS
http://www.linuxquestions.org/questions/linux-from-scratch-13/%5Bhow-to%5D-add-zfs-to-the-linux-kernel-4175514510/

7 Likes
2

The only that wasn't mentioned here that should be done as a precautionary measure is, making your initramfs and updating your grub-config file.

1 Like
3

I assume this is a typo, as I could not find a package named install. Just lettin ya know

4

then you are using an older version of apt.
You will need to append all his apt commands with apt-get.

5

also important thing to note that I don't think the OP did, is that in the kernel config all things marked with an M next to them, are compiled as module's for the kernel, but certain things need to be compiled directly into it, these are marked as a *.

6

The TL;DR of compiling a Kernel.

  • Download a kernel source. Whether it be available from the official repos.
  • CD /use/SRC/linux-4.x.x (whatever kernel you are compiling. If you don't know type in ls /use/src)
  • sudo make clean && sudo make mrproper
  • sudo make menuconfig
  • add in feature you want into your kernel. Important driver based features, modulize them instead of building them in for convenience)
  • sudo make
  • sudo make modules_install
  • sudo make install

Now generate your initramfs

  • mkinitpcio -p linux-4..x.x
    Then update grub-config
  • sudo grub-mkconfig -o /boot/grub/grub.cfg
  • reboot
2 Likes
7

accept for things like filesystems, and a few others I can't remember. These need to be in there directly or the system won't boot*.

  • unless you use a ramfs.
8

Thanks for the tip. I did notice as it installed, it did that stuff automatically. Guess it couldn't hurt to do it manually after everything is done though.

9

Yep. Fail! It's fixed. :slight_smile:

10

Some distros won't update grub automatically that's why it's good to do as a precautionary measure.

1 Like
11

The other thing I was worried about. What if Ubuntu pushes a zfs update (like a bug fix)? Would it affect this custom kernel if it was modular? If Ubuntu's update essentially installed an older version of ZFS and you tried to reboot, could it become unbootable?

I might be wrong here. Not sure how modules work.

12

A little tip, instead of using the <username> placeholder, you can use $(whoami) and no user intervention in the command is needed!

I'm pretty sure dpkg sorts that out when it installs the kernel packages, but that's definitely something that should at least be touched on.

13

or $USER

its shorter and its what I remember.

1 Like
14

Thank you this was very helpful

1 Like
15

Just found this and want you to know it was exactly what I was looking for. Thank you.

1 Like
16

Could this get updated for the upcoming 8.0 release since SPL has been merged into the ZFS repo would be much appreciated :grinning:

1 Like
17

This looks really interesting. Bookmarking this as a project for later :slight_smile:

18

Kernel config looks a bit old. This is what you will need for kernel up to 4.20 …

$ zcat /proc/config.gz | grep -i zlib
CONFIG_SQUASHFS_ZLIB=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y

When in menuconfig you can use / to search for the three configs and dependency, and don’t install ZLIBs as module.

Just my two cents.

19

I’d be happy to do that. I’ve been doing this with kernel 5.0.* and ZFS 0.8.* so it’s pretty fresh.

1 Like
20

Just for you my friend… :stuck_out_tongue: