Wow. That worked flawless. Thanks a lot @tkoham 
I’m not sure why everyone has problems downgrading it. I just re-flashed the older BIOS version without any problems. Are there some manufacturers who are restricting that?
from what I can tell, AMD required it to be non-reversible because it has something to do with support for the 3000 series cpus on the older chipsets.
Some manufacturers routinely ignore AMD recommendations (or have dual bios) though.
Well, I’m glad Gigabyte did
But now I’ve got another Problem: It seems that I’m unable to pass any usb controller through to the virtual machine, because non of them gets the vfio driver loaded, regardless if I try to force it or not. Any ideas?
Thanks a lot in advance.
This might help:
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Passing_though_other_devices
Thanks again, worked.
And now I’ve got another problem (guess this never ends lol):
I’m unable to install the Nvidia drivers in Windows. It shows one PCIe device which isn’t recognized and it shows the RTX 2080, so I guess it isn’t able to fully get it working…?
Code 43? In device manager? You will need to adjust your VM xml to hide the fact it is a vm
Oh my …
I feel so stupid right now, but I must ask:
How?
(I really sorry for my missing experience.)
$ virsh edit [vmname]
...
<features>
<hyperv>
...
<vendor_id state='on' value='levelone'/>
...
</hyperv>
...
<kvm>
<hidden state='on'/>
</kvm>
</features>
Also, do the hugepages thing, it’ll help performance a lot.
It worked, thanks a lot.
When using 2 NVIDIA GPU (one for the host, one for the VM), how to set the vfio driver on the card being passthroughed… and maintain the NVIDIA driver for the host?
That’s what I did in the op. If you do the initram fs script the way I did, only the guest gpu is affected. The vfio driver grabs it before the Nvidia driver has a chance because of where the script runs in the boot process.
Just a question As the process isolate the pcie slot and not the vendor id of the card if I were say to swap my rx570 for a 590 would it still work or should I go through the process again?
This is a great post and I don’t want to be nit-picky, I really am thankful for the detailed write-up. Since it’s a draft, I wanted to help by doing some proof reading and here are some things I stumbled upon:
Here is a slight problem with the automatic markup.
how about 20 * 1024 * 1024 / 2048 =10240
Just a tiny spelling mistake “to the”
Just one space missing
And it might only be me, but I spend 2 days before I figured out that I had to set:
…
<features>
<ioapic driver='kvm'/>
</features>
…
in order to get Windows 10 to boot past the UEFI and not loop to recovery mode
1 Like
I have a little problem getting m.2 sata to passthrough.
My IOMMU groups
The motherboard is an ASRock Phantom Gaming 6
IOMMU Group 0:
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 1:
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 10:
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59)
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 11:
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1466]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU Group 12:
00:19.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
00:19.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
00:19.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
00:19.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
00:19.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
00:19.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
00:19.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1466]
00:19.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU Group 13:
01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller [1022:43ba] (rev 02)
01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller [1022:43b6] (rev 02)
01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge [1022:43b1] (rev 02)
02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
02:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
04:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
IOMMU Group 14:
08:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSDPEKNW020T8 [660p, 2TB] [8086:f1a8] (rev 03)
IOMMU Group 15:
09:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function [1022:145a]
IOMMU Group 16:
09:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU Group 17:
09:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Zeppelin USB 3.0 Host controller [1022:145f]
IOMMU Group 18:
0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function [1022:1455]
IOMMU Group 19:
0a:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 2:
00:01.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 20:
0a:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller [1022:1457]
IOMMU Group 21:
40:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 22:
40:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 23:
40:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 24:
40:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 25:
40:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 26:
40:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 27:
40:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 28:
40:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 29:
40:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 3:
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 30:
40:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 31:
41:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1080] [10de:1b80] (rev a1)
41:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
IOMMU Group 32:
42:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1470] (rev c1)
IOMMU Group 33:
43:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1471]
IOMMU Group 34:
44:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] [1002:687f] (rev c1)
IOMMU Group 35:
44:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
IOMMU Group 36:
45:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function [1022:145a]
IOMMU Group 37:
45:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU Group 38:
45:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Zeppelin USB 3.0 Host controller [1022:145f]
IOMMU Group 39:
46:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function [1022:1455]
IOMMU Group 4:
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 40:
46:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 5:
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 6:
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 7:
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 8:
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 9:
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
The sata SSD I used is in group 19 with the ID [1022:7901]. I could pass it through fine and after I booted from the Win10 installer ISO it was detected by the installer and the initial installation went smoothly. But after the first reboot request the qemu uefi could not find and disk to boot from.
I am using the latest qemu build from the git master tree.
So my assumptions: the Q35 chipset does not know how to initialize the sata controller and therefore does not see the attached drive to boot from.
If I do not capture the controller via vfio and use the /dev/sd* path it works, but not spendidly fast.
Any idea what I am missing in my configuration?
Another quirk I am having is that the qemu-system-x86 process which I am not using freezes on every boot and shows as [defunct] in the process list. My system waits an awful long time to kill it on shutdown or reboots. I have not found a fix, yet.
What is the best resource division between host and guest?
Let’s use an 8 core ryzen.
6/12 threads 8 GB of RAM for win 10 guest be good enough for 1440p 144hz gaming. GTX 1070 would be the host.
Is 2 cores/4 threads enough for Linux host if nothing is really happening on that side?
Appreciate any feedback. Deciding on new build specs.
4/8 is prolly fine for almost all things tbh . 2/4 is fine if the host isn’t doing much
Dat 12 core tho
1 Like
I tried getting vga passthru to work many years ago before VFIO was even a thing. As one might expect I got the card to show up in the windows device manager but no dice on getting it to actually function. At any rate I gave up after going stubbornly at it for a few weeks and I’ve not tried since. Mostly because I didn’t want to go through the disappointment again and/or the expense if I didn’t have just the right hardware for the job.
I’m seriously reconsidering now. I destroyed my Win10 install a couple months back and started gaming on Linux w/Wine/Lutris/etc… But with CP2077 coming out next year… I’m tempted. Barring DX12 being a requirement CP2077 might run fine under Wine, but there’s the performance angle too. CP2077 might just get me to install Win10 again. So long as it’s in a VM.
Anyways, thanks for the guide @wendell. I’m considering the 12 core myself. Hopefully there aren’t any problems with the X470 board I got back in Dec. I really don’t want to get a X570 board if I don’t have to.
@wendell How’s dealing with Systemd boot for the initial RAM disk? Still troublesome? Pop!_OS might just become even more popular because they said they will continue supporting 32bit after 19.10 releases.
Neat approach @wendell. I’ve read a lot about xen’s ability to utilize cards by their PCIe address instead of vendor IDs (which doesn’t require VFIO), but I like this approach as well, and of course it allows us to use KVM.
Personally I’ve shifted more to utilizing Linux natively. I ran PCIe passthrough for a while but I’m focusing more on Proton (DXVK) and just playing native games. It really feels like a waste to run Windows on top of Linux, just to pass your GPU over.
Yep, I said the same thing in the video. Valve and a lot of Indy developers are doing great things in this regard. As soon as mainstream gpus can do things like sr-iov or gvt-b I wouldn’t be surprised if Linux takes over entirely. Even those users that want a “just works” system will be satisfied.