Manually Compile Linux Kernel with ACS Override Patch Applied
This is an extension to the section on “Enable IOMMU in GRUB”
If your GPUs are in the same IOMMU group after you have completed the section of this guide covering how to “Enable IOMMU in GRUB”, then, like me you have an older CPU (in my case an intel i7-4790K) where IOMMU support is hit and miss. Even though your CPU and motherboard support vt-d, the older CPUs can be hit and miss with how assign IOMMU groups are assigned. This situation may have improved from the Haswell refresh or 5th Gen-intel CPUs when DDR4 RAM became hardware requirements. I am not sure if AMD APU/CPUs have this issue, but this solution should work for AMD too.
So, you find that after enabling IOMMU your GPUs are in the same IOMMU group. Not to worry, you can still use GPU passthrough, you will just need to manually compile your kernel from source and apply the ACS patch. This is actually easier than it sounds at first.
To do compile your Linux kernel from source, you will first need to install a number of build dependencies
sudo apt-get install linux-source libqt4-dev build-essential libssl-dev flex bison
Next, download the ACS override patch that best matches the Linux kernel you want to install from:
ACS override patches
Then download the source files for the latest stable Linux kernels and latest release candidate from
Older release candidates can be downloaded from
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
Once downloaded extract the Linux kernel. I have put the commandline commands here but it is much easier to do these using “Files” also included.
For example
tar -xvzf linux-4.20-rc7.tar.gz
Or
Right click on the tarball in your download folder and select extract here
Then go into the extracted folder
cd /linux-4.20-rc7
Or
Right click on the extract folder and select “open in terminal”
Next apply the acs override patch you downloaded earlier by typing in the terminal BUT DON’T hit ENTER /return yet
patch -p1 <
Next drag and drop the acs override patch from the download folder into the terminal. Your final command should look something like this
patch -p1 < ‘/home/USERNAME/Downloads/acso.patch’
Hit ENTER to apply the patch.
Now run
make xconfig
Ctrl+F and search for KVM.
Make sure it is checked.
For some fun search for logo and enable the boot logo uncheck the last two logos (16bit and black and white).
Now search for version and double click to change the version string to -acs-patch. Hit ENTER/return and click save in the top and close the window.
Now to compile your patched kernel: Still in the terminal where you applied the patch, type
make -j4 deb-pkg
If you have a multi-thread CPU with more than 4-cores you can change the -j4 to -j6 if you have 6-cores. The compile can take anything from 25mins to a couple of hours depending on your CPU’s speed.
If your build stops with an error about SSL keys, edit the config file manually with
nano .config
Ctrl+W to search for CONFIG_SYSTEM_TRUSTED_KEYS and comment out the line, save and try to recompile again.
Once the acs patched kernel has been compiled you will find a number of deb packages in you Downloads folder or the folder one level up from the kernel source folder. You will only need the linux-image and linux-headers.
In the terminal type the following BUT DON’T hit ENTER yet
sudo dpkg -i
Now drag and drop the linux-image and linux-headers deb packages into the terminal.
Your terminal command will look something like this
sudo dpkg -i ‘/home/USERNAME/Downloads/linux-4.20-rc7-acs-patch-linux-image.deb’ ‘/home/USERNAME/Downloads/ linux-4.20-rc7-acs-patch-linux-headers.deb’
Hit ENTER to install the packages.
Once the kernel is installed, you will now need to edit the grup to use the acs override. In the terminal type the following
sudo nano /etc/default/grub
and add following to the GRUB_CMDLINE_LINUX_DEFAULT line:
pcie_acs_override=downstream,multifunction
save file (ctrl + x, y, ENTER) and run:
sudo update-grub
Reboot and confirm you have your devices are in separate IOMMU groups as described in the section of this guide titled “Check if IOMMU is Working”
Just as a heads up:
The downstream call enables the asc override, and multifunction call forces all components or functions of a PCI device to be assigned to separate IOMMU. The multifunction call is needed if you are passing an Nvida GPU through to your VM. AMD GPUs tend to separate the VGA, Audio and other components of the GPU into separate IOMMU groups by default but with Nvidia they tend to be bunched together. For Pascal (i.e. GTX 10##) and older Nvidia cards this is not an issue for the VM, however with the Turing (i.e. RTX) GPUs the VGA, Audio, USB-C, and USB controller will all be under the same group if the “multifunction” call is not included in your grub default. As a precaution to prevent issues in or booting the VM, I would assign all 4 functions/device IDs of the RTX cards to the VFIO-PCI driver. The USB controller may still say it is using the xhci_hcd driver but it should pass straight though to the VM as a host PCI device.
