GPU Passthrough on Pop_OS 20.04 with systemd-boot!

Software & Operating Systems VFIO
1

Hi everyone,

because of the super fancy UI I am currently giving Pop_OS a try.
I now need to setup GPU passthrough but was not able to find specific instructions what to do in regards to the systemd-boot setup. Wendels 2019 Guide is not very specific what to do in a simple setup with two different GPUs, also I am not understanding parts of it.

What I have done so far is I added amd_iommu=on to the ‘options’-line in /boot/efi/loader/entries/Pop_OS-current.conf. Also I added ‘vfio-pci’ to the end of /etc/initramfs-tools/modules.

Next I figure I somehow need to regenerate the image that is loaded during the boot process that includes the new options line, how do I do that in systemd-boot?

Also I think I need to specify somewhere that vfio-pci is actually loaded as well I need to bind my PCI device to vfio-pci!

Hope some of you PopOS guys can point me in the right direction!
Thanks in advance!

2

I have found this site: https://github.com/rmayobre/PopOs-VFIO-Tutorial/wiki

But the instructions do not work.
This is the GPU I want to pass through:

lspci -vnn | grep Navi
08:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c1) (prog-if 00 [Normal decode])
09:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479] (prog-if 00 [Normal decode])
0a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev c1) (prog-if 00 [VGA controller])
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:0b36]
0a:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]

And this is my Pop_OS-current.conf:

title Pop!_OS
linux /EFI/Pop_OS-f55f8b4c-9fa0-cd4c-9f1d-ced165d4b1a1/vmlinuz.efi
initrd /EFI/Pop_OS-f55f8b4c-9fa0-cd4c-9f1d-ced165d4b1a1/initrd.img
options root=UUID=f55f8b4c-9fa0-cd4c-9f1d-ced165d4b1a1 ro  loglevel=0 splash systemd.show_status=false quiet vfio-pci.ids=\"1002:731f, 1002:ab38\" iommu=pt amd_iommu=on
3

Disclamer: I have not successfully gotten GPU passthrough to work yet so this may not be correct, but I think I’m doing this part right. I have succeeded with a couple guest OSs so I know this part is correct.

I never had to edit the conf file, with systemd you can add kernel parameters with kernelstub.

for info:

kernelstub --help

Show Kernel Parameters:

sudo kernelstub -p

Add kernel parameter:

sudo kernelstub -a "amd_iommu=on vfio.pci-ids=1002:731f,1002:ab38"

To remove kernel parameter (if you need to do this later to change a parameter):

sudo kernelstub -d "vfio.pci-ids=1002:731f,1002:ab38"

Reboot and then check that it loaded right:

lspci -nnk -d 1002:731f

and

lspci -nnk -d 1002:ab38

Look for “Kernel driver in use: vfio-pci”

Example (my system):

lspci -nnk -d 10de:128b
4c:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208B [GeForce GT 710] [10de:128b] (rev a1)
	Subsystem: ZOTAC International (MCO) Ltd. GK208B [GeForce GT 710] [19da:5360]
	Kernel driver in use: vfio-pci
	Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
1 Like
4

Thanks for your reply. While I understand that using the kernelstub command is the preferred way here, it should not differ from editing the file directly.

The weird thing is while vfio-pci was bound to my graphic cards audio device successfully it was unable to bind the card itself. The card always was in use by amdgpu while the sound device was vfio-pci. Since I was unable to resolve this, found no resources mentioning this problem online and am out of time to tinker I am back to Manjaro for now.

P.S. I also switched the two cards around in the different PCIe slots.

P.P.S. I might give this a try again if I have a few days with nothing to do.