Cannot get gpu output, would really appreciate some help, Ryzen 9, Pop os

Software & Operating Systems VFIO
, ,
1

System Specs

AMD Ryzen 9 3900x
AMD Vega 56 (host)
AMD Rx580 (guest)
MSI Mag B550 Tomahawk
Pop OS 20.04 with 5.8.1 Kernel and the ACSO patch for better IOMMU groupings

IOMMU is enabled in the bios, and I have enabled IOMMU in the systemd boot
Here are my current boot parameters
systemd.show_status=false splash amd_iommu=on iommu=pt pcie_acs_override=downstream,multifunction vfio-pci.ids=1002:67df,1002:aaf0
Where 1002:67df and 1002:aaf0 are my guest gpu IDs

A snippet of my IOMMU groups
IOMMU Group 24 20:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 25 21:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7)
IOMMU Group 26 21:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
IOMMU Group 27 23:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX200 [8086:2723] (rev 1a)`

I have confirmed that the vfio driver is being loaded onto the guest GPU with the lspci -vnn command
21:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) (prog-if 00 [VGA controller]) Subsystem: XFX Pine Group Inc. Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1682:c580] Flags: fast devsel, IRQ 34 Memory at d0000000 (64-bit, prefetchable) [size=256M] Memory at e0000000 (64-bit, prefetchable) [size=2M] I/O ports at e000 [size=256] Memory at fc800000 (32-bit, non-prefetchable) [size=256K] Expansion ROM at fc840000 [disabled] [size=128K] Capabilities: <access denied> Kernel driver in use: vfio-pci Kernel modules: amdgpu

21:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0] Subsystem: XFX Pine Group Inc. Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1682:aaf0] Flags: fast devsel, IRQ 42 Memory at fc860000 (64-bit, non-prefetchable) [size=16K] Capabilities: <access denied> Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel

From here I have been making temporary windows 10 VMs to test the GPU output but I have not been successful. I can get the Windows VM installed and working fine with the emulated video card but as soon as I try to pass through the 580 all I get is a black screen.

I started with the Q35 chipset and UEFI OCMF_CODE.fd bios but have also tried the i440FX chipset and legacy bios.

When I attempt to pass through my GPU I’d also remove the Display Spice, Sound ich9, Serial 1, Channel spice, and Video QXL if that makes a difference.

I am just out of ideas and am not making any progress so I was hoping someone here would be able to help me.

2

Here is my VM XML if that helps

VM gui
VM gui830×713 65.7 KB

3

what are you using to see the gpu output?

4

I just have the guest gpu hooked up to an extra monitor over HDMI.

5

you could try attaching a spice display and qxl card to it, that way you can see if windows is reporting an error initializing the card

6

The furthest I was able to get was Windows 10 about to log in but the the guest and host froze and both became unresponsive.

Nothing was outputted from the guest GPU onto the monitor during this, I was just going off of the QXL display. No error as far as I could tell.

7

Try to uncomment #relaxed_acs_check = 1
in /etc/libvirt/qemu.conf
and remove “splash” in bootloader string, to watch loading process.

8

After making those changes in qemu.conf and the bootloader string I was able to get a GPU output on the guest GPU. Yay, finally!

I have a weird issue where all of the usb devices power off for the host and guest. I’ll be making more virtual machines to see if it occurs again.

In the mean time should I be using the AMD AGESA kernel patch? I know you need it for 0.0.7.2 and newer but since the number reset with Zen 2 it got confusing quick and now I am not sure. I am currently on the latest BIOS for my board, which I think is 1.0.0.2.

Thanks.