[help] vfio GPU Passthrough = Microsoft Basic Display Adapter

Hello, i am trying to setup a Windows 10 VM and Passthrough my 2x GTX 1070, but in the Windows 10 VM he shows 2x Microsoft Basic Display Adapter instead of the 2x GTX1070. I have tried to Passthrough only one GTX 1070 in case it don‘t work with two [they are in a SLI connection] but it won’t work. I installed for a test in the VM Fedora 32 and there the GPU‘s shows right without any problem.

When i install the NVIDIA driver, windows crashes and after the reboot it shows the repair screen. I am wondering why Linux shows the GPU’s correctly in the VM but not Windows, i mean its windows but this …

I’m sorry if I made a stupid mistake but I’m doing this for the first time now, I thank you in advance for any help :slight_smile:

Info:

	OS:
		Host: Debian Buster (10)
		VM: Windows 10 Pro (19041.264)
			virtio-win-0.1.185 & virtio-win-0.1.171 Driver tested
	CPU: i7-6900k
	GPU:  
		Host: GT 710
		VM: 2x GTX 1070 [SLI connection]

lspci -nn [Host]output:

...
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
02:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
03:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 710B] [10de:128b] (rev a1)
04:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)
…

find /sys/kernel/iommu_groups/ -type l [Host]optput (IOMMU Groups):

...
/sys/kernel/iommu_groups/28/devices/0000:03:00.0
/sys/kernel/iommu_groups/28/devices/0000:03:00.1
...
/sys/kernel/iommu_groups/29/devices/0000:04:00.1
/sys/kernel/iommu_groups/29/devices/0000:04:00.0
...
/sys/kernel/iommu_groups/27/devices/0000:02:00.0
/sys/kernel/iommu_groups/27/devices/0000:02:00.1
…

XML Configuration Settings:

<domain type='kvm'>
  <name>win10</name>
  <uuid>61ccfdad-343a-4a5d-9697-72bc5ba7808c</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>20971520</memory>
  <currentMemory unit='KiB'>20971520</currentMemory>
  <vcpu placement='static'>14</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='off'/>
      <vapic state='off'/>
      <spinlocks state='off'/>
      <vendor_id state='on' value='whatever'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/win10.qcow2'/>
      <target dev='sda' bus='sata'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/********/Downloads/Win10_2004_English_x64.iso'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:6a:d4:48'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x24ae'/>
        <product id='0x1006'/>
      </source>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x0930'/>
        <product id='0x6545'/>
      </source>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='1'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
</domain>

[First of all, sorry for my bad English] What else can I do ? I have now gone through the Trobbelshooting section of the ArchWiki and other sides (everything that fits my problem) adapted the XML, but the error still remains. I mean Linux (fedora 32) shows the graphic cards correctly if it is installed in the same VM but not windows.

Is it normal that I despair? x-D

Hello and welcome to lvl1techs!

How are the cards situated in your motherboard? I’d arrange them like so:
pci-e port 1: GT 710
-"^"- 2: GTX 1070
-"^"- 3: GTX 1070

I assume lspci -k shows both of your 1070’s (+audio) bound to vfio-pci like this:

Kernel driver in use: vfio-pci

What does the properties of “Microsoft Basic Display Adapter” show? hopefully not code 43

What happens when you do the repair after the reboot? (after installing the nvidia driver and crashing) Have you tried to do the repair?

You mentioned a sli connection. Have you tried disconnecting the sli connector from the cards and trying the passthrough without the sli connector?

Looking at your xml - it’s quite similar to mine except:

<hyperv>
  <relaxed state='on'/>
  <vapic state='on'/>
  <spinlocks state='on' retries='8191'/>
  <vendor_id state='on' value='b875g4696782'/>
</hyperv>

Good luck with your travels!

1 Like

First of all many thanks for your answer ! !! :slight_smile:

Yes I used the GPU’s as you described, and at “Kernel driver in use:” it says “vfio-pci” [I also added the output here].

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd GP104 [GeForce GTX 1070] [1458:3703]
	Physical Slot: 6
	Flags: fast devsel, IRQ 16, NUMA node 0
	Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
	Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Memory at e0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at fb000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [250] Latency Tolerance Reporting
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [420] Advanced Error Reporting
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900] #19
	Kernel driver in use: vfio-pci
	Kernel modules: nvidia

02:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
	Subsystem: Gigabyte Technology Co., Ltd GP104 High Definition Audio Controller [1458:3703]
	Physical Slot: 6
	Flags: fast devsel, IRQ 17, NUMA node 0
	Memory at fb080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd GP104 [GeForce GTX 1070] [1458:3703]
	Physical Slot: 4
	Flags: fast devsel, IRQ 16, NUMA node 0
	Memory at f8000000 (32-bit, non-prefetchable) [size=16M]
	Memory at b0000000 (64-bit, prefetchable) [size=256M]
	Memory at c0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at d000 [size=128]
	Expansion ROM at f9000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [250] Latency Tolerance Reporting
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [420] Advanced Error Reporting
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900] #19
	Kernel driver in use: vfio-pci
	Kernel modules: nvidia

03:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
	Subsystem: Gigabyte Technology Co., Ltd GP104 High Definition Audio Controller [1458:3703]
	Physical Slot: 4
	Flags: fast devsel, IRQ 17, NUMA node 0
	Memory at f9080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 710B] [10de:128b] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Micro-Star International Co., Ltd. [MSI] GK208B [GeForce GT 710] [1462:8c93]
	Physical Slot: 4-1
	Flags: bus master, fast devsel, latency 0, IRQ 35, NUMA node 0
	Memory at f6000000 (32-bit, non-prefetchable) [size=16M]
	Memory at e8000000 (64-bit, prefetchable) [size=128M]
	Memory at f0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at c000 [size=128]
	[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: nvidia
	Kernel modules: nvidia

04:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)
	Subsystem: Micro-Star International Co., Ltd. [MSI] GK208 HDMI/DP Audio Controller [1462:8c93]
	Physical Slot: 4-1
	Flags: bus master, fast devsel, latency 0, IRQ 17, NUMA node 0
	Memory at f7080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

I also tried to repair windows, but after a while it only says that you have to reinstall it and then the driver installation starts all over again.
i removed the SLI adapter and tried again but with the same result. what really confuses me most is that fedora 32 recognizes both graphic cards and with the nvidia drivers can run without problems in SLI in the VM but not in windows.

even though I’m afraid to write it but:
one “Microsoft Basic Display Adapter” [I call it MBDA now] indicates that everything is fine but the other MBDA says code 43
i already read about code 43 in the archwiki and actually used what was there as a solution. how bad is it now ?

According to my understanding, code 43 must not be present when installing nvidia drivers. (iirc the driver just wont install or it scuffs windows…) You have made the correct addition to prevent code 43 in your .xml file with:
<vendor_id state='on' value='b875g4696782'/>
<kvm> <hidden state='on'/> </kvm>

You could try to pass only one GTX 1070 and hope that the code 43 won’t show up. And try to install the drivers afterwards.

If that won’t work I would try to reinstall windows again, following the archwiki guide to the T. Attaching the gpu('s) after you have succesfully installed windows.

After a quick googling - It seems to get sli working inside a vm you would need a patched nvidia drivers + some kind of Q35 bios sli hack? Is there any other reason besides sli that you want to passthrough both of your 1070’s? My advice is to try to get the vm working with one gpu and start there. SLI inside a vm seems like a BIG PITA.

Sorry for my late answer, the last days were quite stressful.

I have completely omitted the SLI adapter for the tests now to exclude it as error source.

I reinstalled the windows system every time I made major changes to the XML and since I only forward one GPU the error code 43 is no longer displayed [which I didn’t notice when I first tried to forward only one GPU] but it is still so that windows gets completely crushed when I try to install the drivers.

I also followed the archwiki guide and under “Attaching the PCI devices” it says “If everything went well, the screen plugged into your GPU should show the OVMF splash screen” but at the start up i only see “TianoCore” “Open Platform Firmware Development” is this the “OVMF splash screen” ?.

and sorry for starting too high to get the SLI through, you’re right that you should get a GPU working first.

Additionally info: I have installed the non-free nvidia drivers on the host system. Even if the vfio-pci drivers are displayed under “lspci -k”, can this cause problems ? because I have seen in all tutorials that the “Nouveau” drivers are used.

windows gets completely crushed when I try to install the drivers.

:frowning_face:
Do you use geforce experience to install the drivers or do you download the drivers straight from nvidia website? I personally used the geforce experience to install the drivers and was able to do it that way.

should show the OVMF splash screen

yes what you described is the splash screen :slight_smile:

Also it does not matter what drivers you use for your other card (host gpu). I’m using the nvidia-blob on my host gpu and vfio-pci driver on guest gpu. (The vfio-pci driver is loaded before the nvidia/nouveau is on the boot process)

I’ve got no idea why windows would just scuff when installing the drivers - You’ve got all the steps right… Hopefully you get it working