Hello everyone,
I am currently trying to get a GPU passthrough with QEMU/KVM/Libvirt/Virt-Manager to work on my Thinkpad X1 Extreme with also bumblebee installed on the host (Archlinux with linux-hardened kernel).
The guest installation (Windows 10) is the 2nd NVME SSD in my laptop which I used as dual booting so far. I passthrough the whole NVME SSD in libvirt.
So far I can:
- Boot Arch with a normal bumblebee setup
- Isolate the nvidia gpu on the fly without rebooting
- Pass it through to the VM
- Boot VM and use VM
- Shutdown the VM
- Remove the isolation / unbind the nvidia gpu on the fly without rebooting
- Use bumblebee again on my host
… I also tried it by hard-binding isolating the nvidia GPU via kernel parameters.
How ever, inside my Windows 10 VM I get a “Microsoft Basic Display Adapter” which shows “Code 31”.
The programm “Speccy” and also GPU-Z on Windows 10 lists my NVIDIA adapter as graphic adapter named as “Microsoft Basic Display Adapter”.
Meaning my NVIDIA card isn’t detected as NVIDIA by Windows 10.
Here are all the information I could think off that would help. Let me know if I should provide more:
Host: Thinkpad X1 Extreme, Kernel: x86_64 Linux 5.5.11.a-1-hardened
Guest: Windows 10 Home (2nd SSD, used for dual booting as well)
Host CPU: i7-8850H
Host GPU: iGPU + NVIDIA GTX1050Ti Max-Q
Host RAM: 32GB
Host SSD: 1TB Samsung 970 Pro
Guest SSD: Intel NVME SSD (passed through)
I did this already: https://old.reddit.com/r/VFIO/comments/ebo2uk/nvidia_geforce_rtx_2060_mobile_success_qemu_ovmf/
Basically I converted this text into a file and added it to my .xml:
U1NEVKEAAAAB9EJPQ0hTAEJYUENTU0RUAQAAAElOVEwYEBkgoA8AFVwuX1NCX1BDSTAGABBMBi5f
U0JfUENJMFuCTwVCQVQwCF9ISUQMQdAMCghfVUlEABQJX1NUQQCkCh8UK19CSUYApBIjDQELcBcL
cBcBC9A5C1gCCywBCjwKPA0ADQANTElPTgANABQSX0JTVACkEgoEAAALcBcL0Dk=
sudo cp /home/myusername/Downloads/SSDT1.dat /var/lib/libvirt/qemu/acpi/SSDT1.dat
<acpi>
<table type="slic">/var/lib/libvirt/qemu/acpi/SSDT1.dat</table>
</acpi>
I extracted my VBIOS/ROM of the nvidia GPU like described here: https://forums.laptopvideo2go.com/topic/32103-how-to-grab-a-notebooks-vbios-that-is-not-supported-by-nvflash/
Output of rom-parser from the extracted and modified BIOS:
./rom-parser nvidia.rom
Valid ROM signature found @0h, PCIR offset 170h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1c8c, class: 030000
PCIR: revision 3, vendor revision: 1
Last image
sudo cp /home/myusername/Downloads/nvidia.rom /var/lib/libvirt/qemu/nvram/nvidia.rom
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</source>
<rom file="/var/lib/libvirt/qemu/nvram/nvidia.rom"/>
<address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
</hostdev>
I also found a friendly guy who helped me with running this patcher: https://github.com/bitthief/ovmf-with-vbios-patch
Patched files can be downloaded here:
https://files.bestmail.ws/PCI/ovmf-vbios-patched.tgz
My vbios.rom:
https://files.bestmail.ws/PCI/nvidia.rom
The .reg file from which I extracted the .rom file:
https://files.bestmail.ws/PCI/vbios.reg
My current libvirt .xml file:
https://pastebin.com/zt00XJEN
VFIO related kernel parameters:
intel_iommu=on,igfx_off iommu=pt kvm.ignore_msrs=1
IOMMU groups:
IOMMU Group 0:
00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec4] (rev 07)
IOMMU Group 1:
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107M [GeForce GTX 1050 Ti Mobile] [10de:1c8c] (rev ff)
01:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev ff)
IOMMU Group 10:
00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0)
IOMMU Group 11:
00:1e.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH Serial IO UART Host Controller [8086:a328] (rev 10)
IOMMU Group 12:
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a30e] (rev 10)
00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 10)
IOMMU Group 13:
02:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD Pro 7600p/760p/E 6100p Series [8086:f1a6] (rev 03)
IOMMU Group 14:
04:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU Group 15:
05:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU Group 16:
05:01.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU Group 17:
05:02.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU Group 18:
05:04.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU Group 19:
06:00.0 System peripheral [0880]: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] [8086:15eb] (rev 06)
IOMMU Group 2:
00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 07)
IOMMU Group 20:
3a:00.0 USB controller [0c03]: Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018] [8086:15ec] (rev 06)
IOMMU Group 21:
71:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
IOMMU Group 3:
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
IOMMU Group 4:
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
IOMMU Group 5:
00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9560 [Jefferson Peak] [8086:a370] (rev 10)
IOMMU Group 6:
00:15.0 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #0 [8086:a368] (rev 10)
IOMMU Group 7:
00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 8:
00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 [8086:a340] (rev f0)
IOMMU Group 9:
00:1b.4 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 [8086:a32c] (rev f0)
/etc/mkinitcpio.conf:
MODULES=(ext4 i915 atkbd vfio vfio_iommu_type1 vfio_pci vfio_virqfd vhost-net)
HOOKS=(base udev autodetect keyboard keymap modconf block encrypt lvm2 filesystems fsck)
Procedure to bind/isolate GPU on my host:
sudo bash -c "echo '10de 1c8c' > '/sys/bus/pci/drivers/vfio-pci/new_id'"
Verification:
lspci -nnk -d 10de:1c8c
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107M [GeForce GTX 1050 Ti Mobile] [10de:1c8c] (rev a1)
Subsystem: Lenovo GP107M [GeForce GTX 1050 Ti Mobile] [17aa:2267]
Kernel driver in use: vfio-pci
Kernel modules: nouveau, nvidia_drm, nvidia
Unbinding on the host again:
sudo bash -c "echo '0000:01:00.0' > '/sys/bus/pci/drivers/vfio-pci/0000:01:00.0/driver/unbind'"
sudo bash -c "echo 'OFF' >> /proc/acpi/bbswitch"
In Windows 10 device manager I have two problems:
- Display Adapters - Microsoft Basic Display Adapter -> Code 31
- Other devices -> Unknown device -> Code 28 (no idea what that is).
I have been trying all kind of things during the last few days. Any help is very appreciated!
Thanks in advance!