Nvidia GPU passthrough with Virtual Machine Manager – How to fix error 43?

Software & Operating Systems VFIO
, ,
1

Hello humanz!

I always wanted to get red of Windows on my PCs. A few years back I just did it especially since I had all but stopped gaming and in the last year or so with Lutris, Wine and DXVK gaming has become a lot easier on Linux. But for some things it’s still not quite there.

So finally I have 2 older mid-level GPUs in my desktop tower (which barely works. btw. one of them is blocking 3 PCIe Slots and they are directly next to each other since that is the best of the configs supported by my motherboard).

Now I have a problem with a PCIe passthrough of a NVIDIA GeForce GTX 980 graphics card to a Windows 10 VM set up on QEMU/KVM via Virtual Machine Manager. I think I managed to pass the GPU through to the Win10 VM and I get the error 43 on the guest system.

I tried to add to the XML via the XML tab in Virtual Machine Manager.

<domain type="kvm">
<name>win10</name>
  <uuid>0f3909f2-6481-4545-be43-f57d103d4acf</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">16777216</memory>
  <currentMemory unit="KiB">16777216</currentMemory>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-4.2">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/OVMF/OVMF_CODE.ms.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <bootmenu enable="yes"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vendor_id state="on" value="1234567890ab"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>
  <cpu mode="host-model" check="partial"/>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/win10-amdgpu.qcow2"/>
      <target dev="sda" bus="sata"/>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/media/goldsteal/00A69A7C5693935E/var/lib/libvirt/configuration_media/virtio-win-0.1.187.iso"/>
      <target dev="sdb" bus="sata"/>
      <readonly/>
      <boot order="2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-to-pci-bridge">
      <model name="pcie-pci-bridge"/>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="scsi" index="0" model="lsilogic">
      <address type="pci" domain="0x0000" bus="0x07" slot="0x01" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:32:d3:5a"/>
      <source network="default"/>
      <model type="e1000e"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="3"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <video>
      <model type="cirrus" vram="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </hostdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
</domain>

Sadly I still get the error in the device manager. All tutorials I can find are for Arch Linux or older versions of Linux Mint and I am sure I am missing something that changed. Does anybody know what I am missing?

My host system
OS/Kernel: Linux Mint 20 with 5.9.0-050900rc7-generic
CPU: Intel Core i7-5820K CPU @ 3.30GHz × 6
Main GPU/driver: Radeon R9 380x with amdgpu driver

P.S.: If you are a moderator feel free to move this to a more appropiate section.

2

NVidia does not want you to use their GTX cards with VM’s. You have too spoof it so that the driver does not detect the VM.

Replace the top line in the XML with:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

Then at the bottom, before the </domain> tag add:
<qemu:commandline>
<qemu:arg value='-cpu'/>
<qemu:arg value='host,hv_time,kvm=off,hv_vendor_id=null'/>
</qemu:commandline>

This should resolve the problem.

4 Likes
3

Awesome! Thanks for the fast answer.
I was aware that I needed to somehow pretend to the card that it isn’t on a VM but I wasn’t sure if I did all that is required for it already.
I knew I was missing something and the arguments you wrote seem very familiar to me from a tutorial by Mathias Hüber on his blog. When I followed that though I just had no idea how to add them to the virt-manager XML.

Now I get an error when I just copy paste the xml arguments. Maybe there is a space missing in

and it should be

<qemu:arg value=‘host,hv_time,kvm=off,hv_vendor_id=null’/>

instead?

Update: Anyways after a reboot the card now seems to be running as intended.
I will update you should I run into other problems.

Thank you so much again for the swift answer.
It would have taken me forever to find out how to correctly amend all the tutorials for my use case – most tutorials obviously don’t use the newest kernel you can throw at linux (save for one you compile yourself I suppose) and Linux Mint 20 AND Virt-Manager.

Now on to figuring out Looking Glass. So excited. Thanks dude!

4

I went ahead and moved this to the vfio section.

These are the instructions you would want to use.
https://looking-glass.hostfission.com/wiki/Installation
If you have issues, there is a specific topic on this forum for looking glass support.

And the host application also has a chocolatey package now:
https://chocolatey.org/packages/looking-glass-host

1 Like
5

Yes, there was a space missing. I fixed it in the solution.

1 Like