Hi.
Been trying for the last few weekends to get GPU passthrough to work but no success.
My setup:
CPU: Ryzen 1400
MB: Asrock fatal1ty AB350 gaming g4
GPU1: Saphire Nitro 580 4g
GPU2: Asus turbo 1070.
OS: Manjaro
Mainly following an online tutorial, but looks like i can’t post the link. GPU is not using the normal driver, it it using vfio.
This MB has 2 PCI-e slots:
upper one is x16, currently contains the AMD card.
lower one is x4, currently contains the nvidia card.
Trying to pass the upper one to the vm (from what i understand, that should be the corect slot, different IOMMU group, even though linux seems to say both GPUs are in separate iommu grous with no other devices).
The MB is using the latest bios for this cpu (5.80). Weird thing i noticed after updating to the latest bios, is the with 2 GPUs installed , the 2nd one (x4 slot) displays the BIOS information, not the x16 as i would have expected (and how it was in the previous 5.40 BIOS).
when staring the machine up, dmesg says this:
[ 1303.424279] audit: type=1100 audit(1586008955.236:127): pid=4003 uid=1000 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:authentication grantors=pam_unix acct=“bugsbunny” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success’
[ 1303.424508] audit: type=1101 audit(1586008955.236:128): pid=4003 uid=1000 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct=“bugsbunny” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success’
[ 1303.424859] audit: type=1110 audit(1586008955.236:129): pid=4003 uid=0 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:setcred grantors=pam_unix acct=“root” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success’
[ 1303.430713] audit: type=1105 audit(1586008955.243:130): pid=4003 uid=0 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct=“root” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success’
[ 1304.070778] pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:00.0
[ 1304.070785] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID)
[ 1304.070788] pcieport 0000:00:03.1: AER: device [1022:1453] error status/mask=00200000/04400000
[ 1304.070792] pcieport 0000:00:03.1: AER: [21] ACSViol (First)
[ 1304.070870] pcieport 0000:00:03.1: AER: Device recovery successful
[ 1304.193408] AMD-Vi: Completion-Wait loop timed out
[ 1304.322797] AMD-Vi: Completion-Wait loop timed out
[ 1304.451120] AMD-Vi: Completion-Wait loop timed out
[ 1304.578720] AMD-Vi: Completion-Wait loop timed out
[ 1304.706641] AMD-Vi: Completion-Wait loop timed out
[ 1304.834088] AMD-Vi: Completion-Wait loop timed out
[ 1304.963111] AMD-Vi: Completion-Wait loop timed out
[ 1305.072410] audit: type=1101 audit(1586008956.883:131): pid=4033 uid=1000 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct=“bugsbunny” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/1 res=success’
[ 1305.072543] audit: type=1110 audit(1586008956.883:132): pid=4033 uid=0 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:setcred grantors=pam_env,pam_fprintd acct=“root” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/1 res=success’
[ 1305.073796] pcieport 0000:00:03.1: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:00:00.0
[ 1305.073805] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID)
[ 1305.073808] pcieport 0000:00:03.1: AER: device [1022:1453] error status/mask=00200000/04400000
[ 1305.073811] pcieport 0000:00:03.1: AER: [21] ACSViol (First)
[ 1305.073900] pcieport 0000:00:03.1: AER: Device recovery successful
[ 1305.076379] audit: type=1105 audit(1586008956.890:133): pid=4033 uid=0 auid=1000 ses=2 subj==unconfined msg=‘op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct=“root” exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/1 res=success’
[ 1305.194551] AMD-Vi: Completion-Wait loop timed out
[ 1305.323174] AMD-Vi: Completion-Wait loop timed out
[ 1305.451263] AMD-Vi: Completion-Wait loop timed out
[ 1305.580915] AMD-Vi: Completion-Wait loop timed out
[ 1305.581451] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a21a0]
[ 1305.708645] AMD-Vi: Completion-Wait loop timed out
[ 1305.841362] AMD-Vi: Completion-Wait loop timed out
[ 1305.971683] AMD-Vi: Completion-Wait loop timed out
[ 1306.068199] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a21e0]
[ 1306.068203] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a2200]
[ 1306.076700] pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:00.0
[ 1306.076710] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID)
[ 1306.076714] pcieport 0000:00:03.1: AER: device [1022:1453] error status/mask=00200000/04400000
[ 1306.076718] pcieport 0000:00:03.1: AER: [21] ACSViol (First)
[ 1306.076810] pcieport 0000:00:03.1: AER: Device recovery successful
[ 1306.195940] AMD-Vi: Completion-Wait loop timed out
[ 1306.323691] AMD-Vi: Completion-Wait loop timed out
[ 1306.416321] vfio-pci 0000:0a:00.0: Refused to change power state, currently in D3
[ 1306.433110] vfio-pci 0000:0a:00.0: Refused to change power state, currently in D3
[ 1306.449649] vfio-pci 0000:0a:00.0: Refused to change power state, currently in D3
[ 1306.451510] AMD-Vi: Completion-Wait loop timed out
[ 1306.579069] AMD-Vi: Completion-Wait loop timed out
[ 1306.706676] AMD-Vi: Completion-Wait loop timed out
[ 1306.834355] AMD-Vi: Completion-Wait loop timed out
[ 1306.962034] AMD-Vi: Completion-Wait loop timed out
[ 1307.070062] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a22b0]
[ 1307.079473] pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:00.0
[ 1307.079481] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID)
[ 1307.079484] pcieport 0000:00:03.1: AER: device [1022:1453] error status/mask=00200000/04400000
[ 1307.079487] pcieport 0000:00:03.1: AER: [21] ACSViol (First)
[ 1307.192990] vfio-pci 0000:0a:00.0: timed out waiting for pending transaction; performing function level reset anyway
[ 1307.197646] AMD-Vi: Completion-Wait loop timed out
[ 1307.325138] AMD-Vi: Completion-Wait loop timed out
[ 1307.452855] AMD-Vi: Completion-Wait loop timed out
[ 1307.580452] AMD-Vi: Completion-Wait loop timed out
[ 1307.708163] AMD-Vi: Completion-Wait loop timed out
[ 1307.835943] AMD-Vi: Completion-Wait loop timed out
[ 1307.963560] AMD-Vi: Completion-Wait loop timed out
[ 1308.071939] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a22e0]
[ 1308.199634] AMD-Vi: Completion-Wait loop timed out
[ 1308.327578] AMD-Vi: Completion-Wait loop timed out
[ 1308.419649] vfio-pci 0000:0a:00.0: not ready 1023ms after FLR; waiting
[ 1308.455595] AMD-Vi: Completion-Wait loop timed out
[ 1308.583425] AMD-Vi: Completion-Wait loop timed out
[ 1308.711005] AMD-Vi: Completion-Wait loop timed out
[ 1308.838619] AMD-Vi: Completion-Wait loop timed out
[ 1308.966189] AMD-Vi: Completion-Wait loop timed out
[ 1309.073819] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a23c0]
[ 1309.073821] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0a:00.0 address=0x6235a23e0]
when running the vm, i get this:
QEMU 4.2.0 monitor - type ‘help’ for more information
(qemu) pulseaudio: Wrong context state
pulseaudio: Reason: Access denied
pulseaudio: Failed to initialize PA contextaudio: warning: Using timer based audio emulation
qemu-system-x86_64: vfio_err_notifier_handler(0000:0a:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest
qemu-system-x86_64: vfio_err_notifier_handler(0000:0a:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest
Also tried it with LinuxMint 19.2 (since that is what is used in the tutorial) and it behaved pretty much the same, only thing is that after 10 seconds it would freeze the entire PC :P.
Tried switching GPUs (so passing the nvidia one, on the x16 slot) but that resulted in it being stuck in D3. Tried disabling D3 state with a boot flag but it would still give me that error…
Using this script to start it:
#!/bin/bash
vmname=“windows10vm”
if ps -ef | grep qemu-system-x86_64 | grep -q multifunction=on; then
echo “A passthrough VM is already running.” &
exit 1else
use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_PA_SAMPLES=8192
export QEMU_AUDIO_TIMER_PERIOD=99
export QEMU_PA_SERVER=/run/user/1000/pulse/nativecp /usr/share/ovmf/x64/OVMF_VARS.fd /tmp/my_vars.fd
qemu-system-x86_64
-name $vmname,process=$vmname
-machine type=q35,accel=kvm
-cpu host,kvm=off
-smp 1,sockets=1,cores=1,threads=1
-m 4G
-rtc clock=host,base=localtime
-vga none
-nographic
-serial none
-parallel none
-soundhw hda
-usb
-device usb-host,vendorid=0x24ae,productid=0x1001
-device vfio-pci,host=0a:00.0,multifunction=on
-device vfio-pci,host=0a:00.1
-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd
-drive if=pflash,format=raw,file=/tmp/my_vars.fd
-boot order=dc
-drive id=disk0,if=virtio,cache=none,format=raw,file=/home/bugsbunny/windows_vm/win.img
-drive file=/home/bugsbunny/Downloads/Win10_1909_English_x64.iso,index=1,media=cdrom
-drive file=/home/bugsbunny/Downloads/virtio-win-0.1.173.iso,index=2,media=cdrom
-netdev type=tap,id=net0,ifname=vmtap0,vhost=on
-device virtio-net-pci,netdev=net0,mac=02:68:b3:29:da:98exit 0
fi
Does anyone have any ideas on what i could try? Or is my hardware not compatible with it? (heard rx580 is not the best card, but 1070 should have worked… maybe newer BIOS is to blame?)
Thanks!