VFIO Passthrough on Ryzen 2: VM running, Card seemingly isolated properly, yet no output.

Hi, I would be really thankful if someone could help me out a bit, as I’m not able to find what’s causing the problem with my passthrough. The VM is running properly using the host display in virt-manager and seems to run, when using the passed PCI Devices, yet I don’t get any output on the card.

I followed these guides:

Hardware

  • Ryzen 2600
  • Gigabyte AB-B350 Gaming
  • Guest GPU: AMD Radeon RX580
  • Host GPU: Nvidia GT710
  • 16GB RAM
  • Arch Linux

Output for "dmesg |grep -e DMAR -e IOMMU"

[    0.604144] AMD-Vi: IOMMU performance counters supported
[    0.605912] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[    0.607310] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

Output for lspci -nnv

 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/580] 
 [1002:67df] (rev e7) (prog-if 00 [VGA controller])
    Subsystem: Micro-Star International Co., Ltd. [MSI] Ellesmere [Radeon RX 470/480/570/580] [1462:3418]
    Flags: bus master, fast devsel, latency 0, IRQ 76
    Memory at d0000000 (64-bit, prefetchable) [size=256M]
    Memory at e0000000 (64-bit, prefetchable) [size=2M]
    I/O ports at f000 [size=256]
    Memory at f7900000 (32-bit, non-prefetchable) [size=256K]
    Expansion ROM at f7940000 [disabled] [size=128K]
    Capabilities: <access denied>
   **Kernel driver in use: vfio-pci**
    Kernel modules: amdgpu

06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
    Subsystem: Micro-Star International Co., Ltd. [MSI] Ellesmere [Radeon RX 580] [1462:aaf0]
    Flags: bus master, fast devsel, latency 0, IRQ 77
    Memory at f7960000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: <access denied>
    **Kernel driver in use: vfio-pci**
    Kernel modules: snd_hda_intel

IOMMU Groups

These are my IOMMU-Groups. As the Card I’m trying to pass through seems to be isolated in Group 13, I didn’t apply the ACS patch.

VM Configuration

  • Emulator: qemu-system-x86_64
  • Firmware: UEFI
  • Chipset: i440FX
  • CPU: Copy host CPU configuration
  • Every Default Display Module deleted
  • Added the 2 580-related Devices

I have trouble deducing what the problem could be, as the card seems to be isolated properly and the VM seems to be set up correctly. If someone could help me troubleshooting I would be very thankful.

Can you add part of libvirt logfile of VM?

I my case (HostOS: Xubuntu 17.10x64) has for sample (Ubuntu18.04 … VM without passthrough!) logfile /var/log/libvirt/qemu/Ubuntu18.04.log.
Particulary part of logfile shows qemu OPTS.

Something like this

2018-05-09 14:28:20.606+0000: starting up libvirt version: 3.6.0, package: 1ubuntu6.6 (Christian Ehrhardt [email protected] Wed, 25 Apr 2018 09:24:08 +0200), qemu version: 2.10.1(Debian 1:2.10+dfsg-0ubuntu3.5), hostname: xubu1710
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=spice /usr/bin/kvm-spice -name guest=Ubuntu18.04,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Ubuntu18.04/master-key.aes -machine pc-i440fx-artful,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu Opteron_G3 -m 4096 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid 8de9bffc-d26b-41ab-b8d5-9d8b14c3c405 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-2-Ubuntu18.04/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/var/lib/libvirt/images/Ubuntu18.04.qcow2,format=qcow2,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-0-1,readonly=on -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=26,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:32:f7:4c,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=1 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on

Just resolved this issue. Weirdly enough, a fresh install on a Q35 Chipset did the trick

Congrats! It is supposed to be my second advice. I’m running GFX passthrough on Q35 chipset (OVMF). :wink: