Reverting vfio changes

Software & Operating Systems VFIO
1

I recently tried to get PCI passthrough working on my system for something like the 6-7th time. Still not able to get it. I want to reverse the changes, and get my GPU back, but I am having troubles. See here to see the original setup.

As it stands, When I run lspci -k, the GPU doesent have any driver asigned to it.

/etc/initramfs-tools/modules

amdgpu
vfio
vfio-iommu-type1
vfio-virqfd
vfio-pci

/etc/modules

vfio 
vfio-iommu-type1 
vfio-pci 
vfio-virqfd

im assuming I can clear both of these (minus the amdgpu?) and then update-initramfs?

I have also cleared my /etc/modprobe.d/nvidia.conf of

softdep nouveau pre: vfio-pci 
softdep nvidia pre: vfio-pci 
softdep nvidia* pre: vfio-pci

which is most likely an artifact of one of my guides.

2

You have an amd and an nvidia and have vfio and amdgpu drivers installed and no Nvidia drivers.

What are your GPUs - which drivers should you be using?

3

Primary (currently working) GPU is a 1070TI. Secondary is a RX550, which is not getting a kernel driver. Im honestly not sure what drivers I should be using in this case.

4

Hello again! I’m sorry to hear your adventures in VFIO has not paid dividends :frowning_face:

You are on the right path - Clear everything “vfio”-related from

/etc/initramfs-tools/modules
/etc/modules

and update-initramfs.

Also remember from the last post - You are using systemd boot, check out your

esp/loader/ or esp/loader/entries

if you have added boot flags (kernel parameters) to your boot manager. You need to clear vfio-related kernel parameters to unbind the vfio.

What does “apt search amdgpu” return?

also check out this tutorial: amd-radeon-ubuntu-20-04-driver-installation

EDIT
Did you get vfio to bind to your amdgpu? Looking at your modules files, you are not specifying the IOMMU ID for your gpu that you want to be bound.
From archwiki

Vfio-pci normally targets PCI devices by ID, meaning you only need to specify the IDs of the devices you intend to passthrough.

So your modules file (or kernel parameters) should specify the IOMMU ID for your gpu like this:

vfio-pci.ids=10de:13c2,10de:0fbb (ids=your gpu, your gpu_audio)

I suggest using the kernel parameters as they are easier (IMO)

5

I was able to get the gpu to get the right driver, but unable to create the VM for some reason. It just Hung on creation, and I have broken the time box that I put on this project.
I have never really had luck with this MOBO/CPU. 1950X TR, (one of the first off the line), and a Zenith Extreme that I think has had a corrupted BIOS since the day I got it (even after a BIOS update)

Unfortunatly, I am still not getting a driver on the GPU or it’s audio after removing all the VFIO stuff.
“apt search amdgpu” output.

Sorting... Done
Full Text Search... Done
libdrm-amdgpu1/focal,now 2.4.101-2 amd64 [installed,automatic]
  Userspace interface to amdgpu-specific kernel DRM services -- runtime

ricks-amdgpu-utils/focal,focal 2.6.0-1 all
  AMD GPU performance adjustment and monitoring

xserver-xorg-video-amdgpu/focal,now 19.1.0-1 amd64 [installed,automatic]
  X.Org X server -- AMDGPU display driver

xserver-xorg-video-amdgpu-hwe-18.04/focal 3:14.5 amd64
  Transitional package for xserver-xorg-video-amdgpu-hwe-18.04

I feel like I should note that I still have not been able to find the ids to actually remove, from my grub (which I installed) or systemd-boot(which I have now re-installed) kernel params. they were just the default params.

edit: cat /proc/cmdline I got Using initrd=\EFI\Pop_OS-005808cb-b3f3-40a4-9b61-045121c3a7bf\initrd.img root=UUID=005808cb-b3f3-40a4-9b61-045121c3a7bf ro quiet loglevel=0 systemd.show_status=false splash so I assume that I have removed the params.

edit 2:
“sudo lshw -c video” shows *-display UNCLAIMED if that tells you anything

6

Try making sure that you are not blacklisting AMDGPU anywhere.

If you are not sure of all of the config files you changed, try running this to search.
sudo grep -ir "blacklist amdgpu" /etc/

Expected locations would be in /etc/modules or in /etc/initramfs-tools/

7

Unfortunately, the grep doesn’t list any blacklisted.

Followed Method 2 from the amd driver instilation link didnt result in anything.

$ lsmod | grep amd
edac_mce_amd           32768  0
kvm_amd                98304  0
kvm                   663552  1 kvm_amd
ccp                    86016  13 kvm_amd
amdgpu               4575232  0
amd_iommu_v2           20480  1 amdgpu
gpu_sched              32768  1 amdgpu
ttm                   106496  1 amdgpu
drm_kms_helper        184320  2 amdgpu,nvidia_drm
drm                   491520  12 gpu_sched,drm_kms_helper,amdgpu,nvidia_drm,ttm
i2c_algo_bit           16384  2 igb,amdgpu
gpio_amdpt             20480  0
gpio_generic           20480  1 gpio_amdpt

based on the above, Im guessing that the driver is loading, but just not claiming the card?

8

GOT IT!!
I forgot to remove the bind_vfio.sh script from /etc/initramfs-tools/scripts/init-top

1 Like