Figured I would document my recent success in getting a Threadripper system to pass through both GPUs to VMs without requiring a third video card, or any kernel patches. This will be just a quick rundown for those that are looking to do the same, I will not be providing help or support on this, but it should get you started should you be looking to do the same.
Requirements:
- The primary video card MUST be something that can be re-posted, such as Intel or NVidia
- If you are using a VEGA, it must be the secondary card and must NOT post at boot as at this time FLR on this card is broken.
- If your primary is a NVidia card and it won’t post first, enable CSM, this was the only way I could achive this on my Gigabyte AORUS x399 Gaming 7.
Follow the usual method of setting up your host for VFIO, be sure you have a method of remote access (SSH) as once it’s booted headless you will not be able to use the machine.
As many have found, blacklisting the primary video driver is not enough to allow the video device to be passed through to a VM as the efifb kernel driver will mmap the BAR of the video card. To prevent this you can add the following to your kernel boot arguments:
video=efifb:off
Once this is done, after the bios boots the kernel, you should see no further screen updates, the system will appear frozen, but it isn’t. The card is now free to be passed through into a VM.
If you are using a NVidia card as your primary device, after posting the VGA BIOS will not be accessible, to work around this dump the BIOS first and provide it to the vfio device in qemu as romfile=/path/to/the/vgabios.rom. You can use the nouveau driver to dump the rom.
Using the above information I have two VM’s operating, one running Windows 10 with a GeForce 1080Ti, and the other running Linux on an AMD Vega 56. The host is headless as it now has no VGA adapter at all.