Graphics Card Passthrough for Gaming and Windows Apps on Linux Mint 18.1

obi_wan_jakobee · April 5, 2017, 7:40pm

DISCLAIMER

This guide assumes you've already confirmed a) your hardware supports IOMMU and b) you don't have any IOMMU grouping issues (specifically, your VGA device group). This post is only going to contain setup information for Intel CPUs. If you're running AMD there's still lots of info out there to help you along.

Before I get started, I'd like to thank all of the Linux geniuses out there that took the time to document their processes and procedures. I borrowed ideas, concepts, commands, and advice from many different articles and forums on the Internet (Gray Wolf being one of them... if you see this, thank you dude!). Without those kind people freely publishing their expertise, this would not be possible for me.

Let's establish one thing about myself first. I know how to get around a Linux OS but I am absolutely not an expert. Although I will gladly try, I probably cannot help you troubleshoot your PCI passthrough setup. I can tell you what worked and what didn't with my specific hardware but the main reason I'm posting this is simply to put the information out there as a reference for those wanting to accomplish the same thing.

Speaking of accomplishing goals, what and why exactly did I want to do this? The long and short of it, I've been more than irritated with Microsoft's sheisty business practices since Windows 10 arrived on the scene. I've fiddled with permanently switching to Linux for a quite a while and Windows 10 was the proverbial "straw that broke the camel's back". Just as I was putting together a brand new Kaby Lake build I heard the news about MS blocking updates for the platform. That was it. I'd had enough. The problem? I support around 100 clients in a strictly Windows environment. Many of the tools I leverage cannot be used on Linux without making a remote connection to a Windows machine and that wasn't something I really wanted. The other big reason, of course, is gaming. While gaming on Linux has come a long way in recent years with both WINE emulation and Steam, the lower-than-native-Windows performance was something to which I couldn't come to terms. In stepped the magical world of KVM PCI passthrough! I knew it was possible but I hadn't been successful in my previous attempts. Fast forward a year and boy, has it come a long, LONG, way! So, not only can I get rid of Windows as my daily driver but I also still have access to the required Windows tools for my work as well as gaming with near native Windows performance. Admittedly, I'm a performance snob when it comes to games. If the performance wasn't there, I wouldn't be doing/using this.

Now that all of that is out of the way, let's begin with my hardware. If you're attempting this yourself you should already be aware that IOMMU technologies (AMD or Intel) are a requirement. Most modern CPU's and chipsets support these technologies but you'll want to double check your own hardware to be sure.

My Hardware Specs:

i7 7700k (running at stock clock as of this writing)
Z270 Asus Maximux IX Code mobo
32 GB Corsair LPX DDR4
EVGA GTX 1070 SC (I'm using the IGP on the 7700k for Mint)
1 500GB Samsung 850 Evo 2.5" SATA SSD (where my backup Windows 10 installation resides)
1 240GB Sandisk 2.5" SATA SSD (where all the testing and magic happens until I'm comfortable enough to permanently move it to the larger SSD)
1 27" Asus PB278 1440p IPS Display
1 27" Dell S2716DGR 1440p Gsync TN Display (yes, Gsync works through passthrough for me)

Preparing the Machine and OS for IOMMU:

Make sure to update to the latest BIOS for your motherboard (I didn't to start and it wouldn't work until I did)
Enable VT-x and VT-d (or their equivalent) in BIOS
Install Linux Mint 18.1 from live USB (I wiped and installed Mint Cinnamon on the 240GB SSD and chose to download/install 3rd party software during install)
Boot into Linux Mint
Run the command sudo xed /etc/default/grub
Change GRUB_CMDLINE_LINUX_DEFAULT="quiet nomodeset" to GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
Save and exit
Run the command sudo update-grub and reboot after it's finished
Run the command dmesg | grep "Virtualization Technology for Directed I/O"
The output should be similar to [ 0.902214] DMAR: Intel(R) Virtualization Technology for Directed I/O
If your output doesn't look similar you have IOMMU issues to deal with before moving on

Installing QEMU/KVM and Other Quality of Life Tools:

Run the command sudo apt-get update
Run the command sudo apt-get install qemu-kvm qemu-utils virt-manager libvirt-bin ovmf
Once the applications are installed, log off and back on to Linux Mint

Create Your VM:

Download the guest OS you wish to install if you don't already have it (ISO image is what I used)
Download Virtio Windows drivers from fedoraproject.org (I used the "latest" version)
Run the command virt-manager
Click the "Create a new virtual machine button" within virt-manager

You can choose to create the VM however you wish and with whatever OS you desire. The remaining steps in this section are what I chose and what worked for me.

Choose the ISO install (was default option for me)
Select ISO install media from download location
Uncheck OS automatic detection option and fill it in manually
Add RAM and CPU(s)
Create your VM disk image
Check the box to customize the configuration before install
Overview>set firmware option to UEFI and chipset to Q35
CPUs>Allocate desired number of logical CPUs (I have mine set to 6)
Check box to copy host CPU configuration
Check box to manually set CPU topology
For hyperthreaded Intel quad core CPUs, Sockets=1, Cores=4, Threads=2
Allocate desired guest RAM (I have 32GB so I've given my VM guest 16GB)
For the virtual disk, select VirtIO from the disk bus type within advanced options and set cache mode to writeback within performance options
Change the existing CDROM device to SATA
Add another CDROM using SATA as the disk bus and mount the VirtIO driver ISO downloaded earlier
NIC should be set to NAT mode and the device model set to VirtIO
Add your PCI devices (graphics card and/or other PCI devices that are grouped compatibly)
Add your desired passthrough USB devices (I have passed my gaming mouse, gaming keyboard, and USB microphone)
Set USB controller to 2.0 (I had microphone issues in the Windows guest while set to USB 3.0)
Remove all devices that aren't required (e.g. graphics, video, smartcard, etc)
Connect the graphics card being passed through to a different monitor than that being used for the Mint setup (or connect it to another input of the same monitor but will require constantly going between inputs for Mint and Windows)
Boot up the VM
Run Windows setup
To load your virtual disk into Windows setup for installation, click "load driver", navigate to your VirtIO mounted ISO drive (will appear as a second CDROM in Windows), navigate to the viostor directory, navigate to your Windows guest version directory, and select the correct architecture
The VirtIO disk driver should install and you can then run Windows setup as you normally would
Once Windows is installed, login and pull up device manager
For any missing drivers, run the driver update wizard and point it to the VirtIO CDROM drive and make sure the "include subfolders" check box is marked
This should install any missing drivers for your VM (most notably, the Ethernet driver)

Prepare for Nvidia Driver Install

Fully shut down your VM
In Mint, run the command sudo virsh edit yourvmname where yourvmname is the actual name of your newly created VM (in my case, the command would look like sudo virsh edit win8.1)
I choose to edit with nano... choose whichever you prefer or you can run the command with your chosen editor in the command itself (e.g. sudo nano virsh edit win8.1)
Change <domain type=’kvm’> to <domain type=’kvm’ xmlns:qemu=’http://libvirt.org/schemas/domain/qemu/1.0’>
Arrow down to the bottom
Between </devices> and </domain> add the following lines
<qemu:commandline> <qemu:arg value=’-cpu’/> <qemu:arg value=’host,hv_time,kvm=off,hv_vendor_id=null’/> </qemu:commandline>
Hit control+x to save the file, Y to overwrite, and enter to save your changes
Boot up your VM and install NVIDIA drivers

Viola!

With any luck, you'll have booted into your Windows VM with your video card and other devices passed straight through to the Windows guest! Game away!!

Worth Noting

I use the TR Edition Astro Mixamp for high definition audio and it kept giving me problems when trying to pass through via USB. To work around this, I used the line-out from my Gsync monitor to feed the auxiliary input of the Mixamp. Audio isn't quite as high quality as it is when I'm using the Mixamp inside Mint however, this workaround allows me to hear audio from both Mint and Windows at the same time... very useful for work.

I also use Synergy for mouse/keyboard sharing so I don't have to use two sets. The way I have it setup; I have my Logitech G403 and G810 in USB passthrough mode to the Windows guest. I have the Synergy server installed on the guest VM and the Synergy client installed in Mint. This allows me to control Mint and the Windows guest at the same time with one m&k combo and I must say, it's quite impressive. Be aware however, there are some games (e.g. Player Unknown's Battlegrounds) where the mouse, even when in first person view, will still overflow to the Mint display when looking around in-game. In those scenarios, I temporarily stop the Synergy server on the guest and start it back up when I'm done gaming.

On the gaming side of things, I tend to play fast paced FPS games that require precision. The games I've tested most thoroughly so far are Overwatch, Battlefield 1 (by far the most), and Battlegrounds. ALL 3 of these games run at 95% native Windows performance for me. There's very little difference and is why I'm so gobsmacked that not only does this work but it wasn't very hard to setup either. My research and testing time was all in one Saturday. I was playing my favorite AAA games on the VM that night.

I hope this guide offers assistance to those looking! Good luck with your setup!