How-to Guide - Single GPU Passthrough using Fedora 34

Some time ago I opened another thread asking for help. Unfortunately not a lot of help was provided. Now that I got this working on my setup I decided to write a guide and share it with you. The same information was also shared on reddit, FYI.


- Long post ahead -

After weeks (on and off) trying to pass my only GPU to a Windows 10/11 VM for gaming, I managed to get it working. I felt I could write a little guide to help out those wanting to do the same in Fedora (34), since most guides are written for either Ubuntu or Arch and the ones for Fedora require 2 GPUs (or are incomplete). I gathered all this information in different places, Github, Youtube, blogs, etc.

Benefits over dual boot?

  • Total OS isolation.
  • Preventing Windows Update from taking over your Linux partition and potentially messing things up.
  • Being able to play -a few extra- online games that have an anti-cheat. We know the struggle, Proton is amazing but some games just won’t work. You’ll now be able to play Rainbow Six: Siege, Hunt Showdown, to name a few.
  • VM Snapshots
  • Actually quicker than having to reboot and back
  • Probably great for people who record instruments over an audio interface (still haven’t tested) and miss using apps such as Ableton.

Cons (for some)

  • When you shut down your VM, you’ll go back to your Linux host in a new session. It doesn’t bother me but it might bother some.
  • In the long “trial and error” process to get this working, I ended up with a borked system several times. My Win10 VM would end up with a black screen which forced me to do a hard-reboot. My guess? my FS wasn’t really happy about this happening many times over short periods of time. I wasn’t able to fix a broken btrfs FS (probably due to lack of knowledge from my side) so I’d end up reinstalling everything out of laziness. From my experience XFS FSs seem to play better. Before getting it to work, I ended up with a borked XFS FS (once) which I easily fixed with xfs_repair. If this happens, don’t panic, it’s totally fixable.

My specs

  • AMD Ryzen 7 3700X 8-Core Processor (16 threads)
  • AMD PowerColor Red Dragon 5700XT
  • Fedora 34 (GNOME)

Good to Know

  • You shouldn’t pass everything to your VM. Leave some resources for your Linux host. e.g : pass 14 CPU threads out of 16 and pass 24GB RAM out of 32GB available.
  • AFAIK x64 systems need a minimum of 4GB of ram (you can correct me in the comments) so if you have less than 8GB of RAM, you’re out of luck.

Pre-requisites

  • Have virtualization enabled in your BIOS
  • Some previous knowledge of VFIO concepts. This is NOT meant to be a full guide explaining every single detail. The guide is already longer than I expected
  • Up to date system ’sudo yum update -y’
  • Patience and willingness to troubleshoot. Consider this a fun mini-project

Let’s start by installing the virtualization package group.

sudo yum -y install @virtualization
or
sudo dnf -y group install Virtualization

This should install all your goodies for virtualization on KVM, including virt-manager, libvirt and qemu. Enable and start the libvirtd service.

Now let’s enable IOMMU in your system. To do this you can pass some arguments to your grub.

sudo vim /etc/default/grub

Find the line with 'GRUB_CMDLINE_LINUX=’ and add these 2:

iommu=1 amd_iommu=on

Save and exit. Now it’s time to re-build your grub by running (as sudo) 'grub2-mkconfig -o /boot/grub2/grub.cfg’ (if using BIOS) OR 'grub2-mkconfig -o /boot/efi/EFI/fedora/grub2.cfg (if using UEFI) . After our grub is recreated feel free to reboot your PC. Once you’re logged into a new session, open up a terminal and run 'cat /proc/cmdline’. If everything worked as expected, the grub parameters you passed in the previous step should be visible. In my case:

cat /proc/cmdline

BOOT_IMAGE=(hd1,gpt3)/vmlinuz-5.12.11-300.fc34.x86_64 root=/dev/mapper/fedora_localhost--live-root ro rd.lvm.lv=fedora_localhost-live/root rhgb quiet iommu=1 amd_iommu=on

VM Creation

It makes sense to create your VM now. Many guides make you configure your hooks BEFORE creating your VM, which in my opinion is a big mistake. You want to be able to create your Windows 10 VM and make sure it’s working fine before doing your magic.

Open up virt-manager. If you see an error message, it’s probable because your user has not been added to the libvirt group. To fix that:

sudo useradd -aG libvirt $USER

I won’t go into details on how to create the VM but I’ll clarify: you do NOT need to pass anything at this point. You just want to create a Windows 10 virtual machine, configure the boot options and order correctly (first the SATA device, second the Win10 ISO), select the right BIOS (Q35/UEFI OVMF_CODE.fd) and that’s pretty much it. Don’t change anything under the CPU or Memory tabs yet. You will do this after configuring the hooks.

Assuming you succeeded at creating your VM, shut it down, close virt-manager and let’s configure the libvirt hooks. In short, the libvirt hooks are scripts that will be executed at the moment you START your VM and the moment you STOP it (hence the chosen naming convention)

My directory tree for the hooks looks as follows…

[[email protected]]/etc/libvirt# pwd
/etc/libvirt

[[email protected]]/etc/libvirt# tree hooks 
hooks
├── kvm.conf
├── qemu
└── qemu.d
    └── win10
        ├── prepare
        │   └── begin
        │       ├── cpu_mode_performance.sh
        │       ├── cpu_pinning.sh
        │       └── start.sh
        └── release
            └── end
                ├── cpu_mode_ondemand.sh
                ├── cpu_unpinning.sh
                └── stop.sh

6 directories, 8 files

… and here’s the content of each file:

Needless to say, the IOMMU ids will be different from yours. Again, check this yourself with 'lspci -nnv | less’ and fix accordingly. Same goes for the CPU pinning/unpinning and start.sh/stop.sh scripts.

Having the hooks in place we can move forward to dealing with the vendor-reset bug (this is important). You can follow THIS mini guide to see how it’s done.

I followed that guide and went a bit further to make sure the kernel module was loaded at boot time. To do that, just create a file in the modules-load.d directory and echo the name of the module. In my case:

❯ pwd
/etc/modules-load.d

❯ cat vendor-reset.conf 
vendor-reset

❯ dracut -fv

Reboot your PC then check if the module is loaded at boot time.

❯ lsmod | grep vendor
vendor_reset          106496  0

Now, for the last part. Open up virt-manager, click “Open” on your VM and re-configure a few things.

On the CPU tab, make sure to select host-passthrough , also make sure to configure the CPU pinning matching the number of cores/threads you defined in your cpu_pinning.sh hook. Remove everything related to Spice, tablet and tty’s. Last but definitely not least, go to Add Hardware > PCI and pass your GPU. Additionally pass any other USB device you need, in my case I also passed a bluetooth dongle (works great) , Razer Viper Ultimate (works great too) and my wireless HyperX headset. For reference I feel no delay whatsoever.

Additionally

If you have an audio interface you can also pass it as an USB device. If the audio breaks in your VM make make sure the interface is using the right Hz. Right click the Sound icon in the Windows taskbar > Sounds > Playback > Scroll down to your interface and right click > Properties > Advanced > “24 bit , 48000 Hz”. That’s how I got my Scarlett 2i2 3rd Gen interface working.

We can now press start and if we are lucky we should have a perfectly functional Windows 10 VM.

Feel free to update the OS (step 1) and also install the latest WHQL drivers (step 2) from AMD’s website. No need to use DDU, just install the WHQL on top of whatever drivers Windows Update installs.

Conclusion

I’ve been testing for 2 days and so far the performance is matching a native W10 installation. The main reason for me to do this was to be able to play Hunt: Showdown on my Linux partition, since the game uses EAC and that’s a no-go with Proton. I’m getting around 75 FPS in 3440x1440 + “recommended settings”, same as when I was playing on native Windows.

I’ll try to fix any typos I might have made, or add any information I might have missed. Hopefully this will help those running Fedora and full AMD setups!

Cheers.

7 Likes

Just wanted to mention, if you have fedora installed with UEFI enabled your grub config file is located at: “/boot/efi/EFI/fedora/grub.cfg”. You’ll get all kinds of conflicts on your next grub update if you keep it at /boot/grub2/grub.cfg

https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/kernel-module-driver-configuration/Working_with_the_GRUB_2_Boot_Loader/

2 Likes

Good one, I totally missed that. I edited the guide pointing this out :slight_smile:

1 Like

Awesome guide! I know a bit about VFIO since I’ve been researching on and off for the past 3 years, but since this is for a single gpu I am stoked on certain parts.

  1. Since I am using a 6800 XT, does the vendor_reset still apply?
  2. What video options should I use? Does it passthrough my monitor or is it an application window?
  3. Does having the default usb options work with usb keyboard and mouse? Or do I have to do something different via passing them through?
  4. Have you tried looking-glass vs native and was there any differences?

Cheers.

no need for vendor reset for 6800xt.

1 Like

Also a side note, when I run it, all my monitors go black, wait 3-4 seconds and it returns me to the plasma login screen.

I’ll ssh in and see what’s happening, but not until tomorrow.

I think I found the issue…or…issues:

/var/log/libvirt/qemu/win10.log

2021-12-21 01:42:01.925+0000: starting up libvirt version: 7.6.0, package: 3.fc35 (Fedora Project, 2021-09-08-12:20:40, ), qemu version: 6.1.0qemu-6.1.0-10>
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
HOME=/var/lib/libvirt/qemu/domain-1-win10 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-win10/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-win10/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-win10/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=win10,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-win10/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/edk2/ovmf/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/win10_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"u>
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-q35-6.1,accel=kvm,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram \
-cpu host,migratable=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff \
-m 16384 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":17179869184}' \
-overcommit mem-lock=off \
-smp 12,sockets=1,dies=1,cores=6,threads=2 \
-uuid 9267a076-ba9e-4953-b33d-f81088327ca0 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=27,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot menu=on,strict=on \
-device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
-device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
-device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
-device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
-device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
-device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
-device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \
-blockdev '{"driver":"host_device","filename":"/dev/sda1","aio":"native","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-rea>
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-hd,bus=ide.0,drive=libvirt-1-format,id=sata0-0-0,bootindex=1,write-cache=on \
-netdev tap,fd=29,id=hostnet0 \
-device e1000e,netdev=hostnet0,id=net0,mac=52:54:00:23:51:6a,bus=pci.1,addr=0x0 \
-audiodev id=audio1,driver=spice \
-device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0,audiodev=audio1 \
-device vfio-pci,host=0000:0b:00.0,id=hostdev0,bus=pci.5,addr=0x0,rombar=0 \
-device vfio-pci,host=0000:0b:00.1,id=hostdev1,bus=pci.6,addr=0x0,rombar=0 \
-device usb-host,hostdevice=/dev/bus/usb/007/004,id=hostdev2,bus=usb.0,port=1 \
-device usb-host,hostdevice=/dev/bus/usb/007/003,id=hostdev3,bus=usb.0,port=2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
audio: Could not init `spice' audio driver
audio: warning: Using timer based audio emulation
2021-12-21T01:42:02.021651Z qemu-system-x86_64: warning: This family of AMD CPU doesn't support hyperthreading(2)
Please configure -smp options properly or try enabling topoext feature.
2021-12-21T01:42:03.675355Z qemu-system-x86_64: vfio: Cannot reset device 0000:0b:00.1, depends on group 27 which is not owned.

The audio device that’s from the gpu isn’t resetting, the display parameter is empty (not sure if that’s right) and it seems that it’s telling me my 3900X is not hyperthreaded? I allocated the first 6 cores and all it’s assigned threads which I thought would make sense, since you don’t want to mix nodes. The spice audio driver shouldn’t exist, I made sure to remove all spice assets. I triple checked my config and I can’t fathom why this isn’t working.

Here’s the win10 vm xml:

<domain type='kvm'>
  <name>win10</name>
  <uuid>9267a076-ba9e-4953-b33d-f81088327ca0</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <vcpu placement='static'>12</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-6.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <boot dev='hd'/>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='partial' migratable='on'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/sda1'/>
      <target dev='sda' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:23:51:6a'/>
      <source network='default'/>
      <model type='e1000e'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <audio id='1' type='spice'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
      </source>
      <rom bar='off'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/>
      </source>
      <rom bar='off'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc539'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x0c45'/>
        <product id='0x8611'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Looking at the IOMMU groupings using the script it shows:

IOMMU Group 25 0b:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] [1002:73bf] (rev c1)
IOMMU Group 26 0b:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT] [1002:ab28]
IOMMU Group 27 0b:00.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:73a6]
IOMMU Group 28 0b:00.3 Serial bus controller [0c80]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 USB [1002:73a4]

So it says the (group 27 is linked to the audio gpu device which is group 26) 00:2 is not owned…which means I have to add it in as a host device? I’m not too sure.

So I added this to my kvm.conf

VIRSH_GPU_VIDEO=pci_0000_0b_00_0
VIRSH_GPU_AUDIO=pci_0000_0b_00_1
VIRSH_GPU_USB=pci_0000_0b_00_2
VIRSH_GPU_BUS=pci_0000_0b_00_3

and in both my start.sh and stop.sh

# Unbind the GPU from display driver
virsh nodedev-detach $VIRSH_GPU_VIDEO
virsh nodedev-detach $VIRSH_GPU_AUDIO
virsh nodedev-detach $VIRSH_GPU_USB
virsh nodedev-detach $VIRSH_GPU_BUS
# Reattach the gpu
virsh nodedev-reattach $VIRSH_GPU_VIDEO
virsh nodedev-reattach $VIRSH_GPU_AUDIO
virsh nodedev-reattach $VIRSH_GPU_USB
virsh nodedev-reattach $VIRSH_GPU_BUS

Thought that would work to fix the “device is not owned” error. But it still comes up with being in a different group (27), which begs the question, all of them are in different IOMMU groups, why this particular one that get’s errored out?

Upon more research it seems my audio device from the gpu does not have the ability to reset unless powercycled, I’ve used this command:

for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done

And my results are:

IOMMU group 17
[RESET] 03:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
IOMMU group 7
        00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 25
[RESET] 0b:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] [1002:73bf] (rev c1)
IOMMU group 15
[RESET] 02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57ad]
IOMMU group 33
        0d:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
IOMMU group 5
[RESET] 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU group 23
[RESET] 09:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c1)
IOMMU group 13
        00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 0 [1022:1440]
        00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 1 [1022:1441]
        00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 2 [1022:1442]
        00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 3 [1022:1443]
        00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 4 [1022:1444]
        00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 5 [1022:1445]
        00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 6 [1022:1446]
        00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 7 [1022:1447]
IOMMU group 31
[RESET] 0d:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
IOMMU group 3
        00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 21
[RESET] 04:00.0 Non-Volatile memory controller [0108]: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive [1cc1:8201] (rev 03)
IOMMU group 11
[RESET] 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU group 1
[RESET] 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU group 28
        0b:00.3 Serial bus controller [0c80]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 USB [1002:73a4]
IOMMU group 18
[RESET] 03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
[RESET] 06:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        06:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
[RESET] 06:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU group 8
        00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 26
        0b:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT] [1002:ab28]
IOMMU group 16
[RESET] 03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
IOMMU group 6
        00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 24
[RESET] 0a:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]
IOMMU group 14
[RESET] 01:00.0 Non-Volatile memory controller [0108]: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive [1cc1:8201] (rev 03)
IOMMU group 32
[RESET] 0d:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU group 4
        00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 22
[RESET] 05:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU group 12
        00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 30
[RESET] 0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
IOMMU group 2
[RESET] 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU group 20
[RESET] 03:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
[RESET] 08:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU group 10
        00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 29
[RESET] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
IOMMU group 0
        00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU group 19
[RESET] 03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
[RESET] 07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU group 9
[RESET] 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU group 27
        0b:00.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:73a6]

The ones that don’t have [RESET] next to their IOMMU ID does not have the ability to reset. Am I screwed?