Linux Gaming System using PCI passthrough with KVM

Hardware Hub Build a PC
1

Hi everyone,

I am thinking about building a Linux gaming system using PCI passthrough with KVM. I haven’t built a system for 15 years. I’ve been dual booting with Ubuntu / Windows 7 for over 10 years on a laptop. I got tired of Window’s problems and only use it for games. Now I am tired of duel booting. I recently became aware that VMs and PCI passthrough has reached a point where I can avoid the need to dual boot. I am only in the planning stage.

Okay, I made this build, am I on the right track?

I have a limited budget and would need to buy my components over several months. I don’t need to be on the cutting edge and would prefer just under because of cost. Anyway I am posting for build ideas. My source for the setup is, Play games in Windows on Linux! PCI passthrough quick guide Here are the requirements for the system I would like to build.

Before we start, bonus if I can upgrade the system to cutting edge when the prices come down. And I don’t want poorly made components.

The CPU must support VT-D virtualization technology.

What is the AMD equivalent called? Which processor would be the best for this purpose? I ask because I believe an AMD processors with more cores would be less expensive. I think more cores are better in this case? Or is core performance more important? Or is it just easier to make this work with Intel?

The motherboard must support IOMMU mapping as well as the same Intel VT-d or AMD equivalent.
Motherboard suggestions? I would prefer a motherboard that just works. And it must be compatible with Linux.

16 GB ram is the minimum and I would plan for 32 GB if it is in the budget.

I need two GPUs. The gaming GPU must support UEFI. I plan on nothing higher than HD. The 2nd GPU (main?) is just for desktop software, videos, with HDMI output. An Intel or AMD integrated graphic should work for the desktop GPU.

I’ll also need two monitors, HD again. What are good performing budget monitors?

I’ll also have two Samsung SSD drives in my laptop so I shouldn’t need to buy a drive. One could be used for building the system and is a newer drive that supports current SATA. And then the 2nd could be used for the gaming drive after I have the system working. No gaming until the build is complete. :frowning: The laptop will be used for NAS after I make the switch. So I’ll buy an additional drive for storage, suggestions? I will add the laptop screen has lost several pixels so it is either a new laptop and stay with duel booting or a whole new system. I no longer have a need for a travel laptop since I have a cheap netbook.

I’ll need a case, my space is limited so smaller is better. Suggestions?

I’ll need a power supply. Nothing poorly made, suggestions?

I’ll need two keyboards and prefer back lite keys, nothing fancy. Suggestions? Also, maybe a single keyboard with a switch to move between Linux and Windows? This should also support the mouse. Did I mention I have limited space for this hardware? :slight_smile:

I would like to keep the cost around $1,500 - $2,000, $1,500 is better for me. Is this realistic?

2

AMD-v = VT-x = normal virtualization extensions

AMD-vi = VT-d = IOMMU support = support for passthrough

NPT (AMD) = EPT (Intel) = SLAT (second level address translation) = nested memory page tables.

For which CPU, any CPU using the Zen architecture should have VT-d. So Ryzen, Threadripper, and Epyc. Intel, you could go as far back as Haswell without issues.

For both AMD and Intel, the iGPUs have caveats.
For Intel, you may have to compile your own kernel with the i915 arbiter patch if you are using Intel HD iGPU on the host and have a passthrough VM using Seabios. This is a non-issue if you are using OVMF uefi for the guest or have a second dGPU to use for the host.
For AMD, their CPU’s that have an iGPU(AMD calls this an APU), are only available with upto four cores, which is not ideal for passthrough. Also, the graphics drivers for the current Raven ridge APUs are still buggy, even with a current kernel. It probably would not even boot with Debian stable.

You could buy a cheap dGPU for the host, really anything that can output your desired resolution and is PCIe would work.

For choosing a motherboard, just search the model + linux, see what pops up. Since you are also doing passthrough, search model + IOMMU (or passthrough), mainly to see if anyone has posted their IOMMU groups.

As for the build, it does not look too bad. You may want to get faster ram, although other people would know better the price/performance ratio as compared to the rest of the build. The Meshify C is a great choice.

Since you are gaming at 1080p, do you need 1070 levels of performance? Because Nvidia drivers are specifically made to not work in a VM. Search code 43 passthrough on the web. I would buy a used RX 580 for $150 USD, then just have to upgrade a bit sooner but not have to deal with the hassle of Nvidia and Linux.

3

Its good that you mention the code 43, but you didn’t mention the workarounds that has been documented throughout the years. You can run Nvidia in the VM, but it requires you to add at least 6 lines in libvirt(Or do natively in QEMU / KVM) to circumvent the drivers checking if its in a VM.
So it requires some what work(arguably) to get Nvidia based graphics card to work in a VM then an AMD card.

4

With up to date libvirt+qemu, it is down to adding two lines in many cases. Enable kvm hidden state and change the vendor id.

It requires more lines with outdated versions.

However, sometimes that does not work, mainly with gtx 10*0 cards. You then sometimes have to get a full vBIOS dump and pass it to qemu.

It is more about voting with your wallet and not buying from a company that is anti open source and actively gimps their products for market segmentation then Nvidia cards being that hard to use for passthrough.

5

Thank you for the replies!

Okay, I think I understand how all of the pieces work. One last question, is the passthrough now working as well on AMD as Intel or is AMD support still lagging behind?

6

Setup:

  • Ryzen 1600
  • GA-AX370-Gaming-K7
  • 16 GB 3200 MHZ
  • 1070 TI
  • Some cheap host card

As for my own experiences past 8 months or so(Longer on older hardware, like FX 8320 and I7 2600), the recent X370 and possibly X470 platforms as time of writing still doesn’t allow pass through from the chipset. Which is bit tedious if you want to pass through usb hubs, network adapters and so on.
I think you can pass through from the chipset on intel much easier then on AMD. At least it was the case for Sandybridge and FX990 platform from AMD.
Though you don’t need to pass through USB hubs to get things working, nor network adapters. You can assign USB devices beforehand to pass through.
But it does make it tedious to attach USB drives to your VM.

So… Technically speaking AMD is behind if you want to get pass through stuff from the chipset, but with more cores on AMD at cheaper price I find this a worthwhile trade off.
Though if you really want to pass through usb hubs, intel is probably better. Al be it at a higher price depending where you live.

As for GPU pass through, both are the same. Anything that comes from the CPU aka PCI-E will show up as separate IOMMU group. Unless it is pass through the chipset. This also means that on AMD you can also pass through one NVME drive, because it is connected to the CPU.

There is one excitement I want to share around proper PCI-E detection (And handling I think) on Windows allowing the GPU running at near bare metal, but the patches need to come and the time frame is unknown. Until then this forum is good place to keep eye on: https://forum.level1techs.com/t/increasing-vfio-vga-performance/

Side note:
I cannot validate the GTX 10*0 cards sometimes needs full vBIOS dump as I’ve been running my GPU pass through system just fine for months now.
Apparently there are cases around, so I take his word for it.

Edit note:
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
Link that I use for setup pass through.

1 Like
7

with looking glass… you shouldnt have to pass usb through since you are running the guest through the host, right?

8

Not sure, last time I used looking glass it requires extra setup to make it work without passing usb hubs.
Though due to other reasons, I decided to wait for full release.
I suggest to search for it around here or at looking glass website.

9

Another question came up in my mind while I was visualizing my desktop layout / space. This will be for gaming. My current setup is a laptop with a 1080p 15.5" screen. I believe I should go with a larger monitor, 21" or about. I don’t have room for two large monitors, two keyboards, and two mouses. Can I use a KVM USB / VGA switcher to switch between Linux and Windows? Something like this. Or is this a problem when switching between the OSes?

10

You could or you could use synergy.
I don’t have KVM switch, I think you need pass through the switcher it self?
Maybe someone else know.

11

USB Audio stuff like ASIO likes USB 3.0 controllers with Message Signaled Interrupts. pci-stub with a onboard chipset 2.0 controller doesn’t allow Message Signaled Interrupts, so you want a external USB 3.0 controller with a Renesas uPD720202.

The main reason I’m not pulling the trigger on Ryzen yet as a passthrough rig is that whole Chipset IOMMU situation. X79, X99 and X299 has really good Hardware ACS through the chipset.

12

Does synergy allow me to switch the monitor between GPUs?

13

I also found this HDMI switch.

Since the KVM switch passes the HDMI audio does the audio from the OS get passed into the HDMI output? I know it works in Windows and Linux when I play a video to an HDMI TV. I see no reason why it would not work the same with a HDMI monitor with speaker support.

14

I think I can avoid the USB audio by passing the sound through HDMI cables.

15

Well no unfortunately, it only support mouse / keyboard.

16

Unless I can fit two monitors on this little desk I think it will not be usable for me.

Maybe I would be better served with two smaller monitors. I am currently on a 15.5" laptop display. Although I want to convert the laptop into a NAS. The display has several dead pixies in rows, good news is not all of the colors are dead. The HDMI still functions so I would add that to the KVM HDMI switcher.

17

Same deal. You need to ensure the GPU and it’s HD Audio controller is using Message Signaled Interrupts with a registry modification.

18

I may need help with that, will find out when I get there.

Everyone, thank you for the replies!

19

I’m back and I’ve refined my build centered around AMD.

Reminder, no drives are needed since I have access to two Samsung 512 GB SSD drives.

AMD Ryzen 7 2700X, 8x 3.70GHz, boxed (YD270XBGAFBOX)
stock box cooler, not planning on OC, I read it is a good cooler

ASRock X470 Taichi (90-MXB7J0-A0UAYZ)

G.Skill Flare X Black DIMM Kit 16GB, DDR4-2933, CL14-14-14-34 (F4-2933C14D-16GFX)
I don’t plan to OC anything, it doesn’t appear the gains are worth the cost.

Sapphire Pulse Radeon RX Vega 56, 8GB HBM2, HDMI, 3x DP, full retail (11276-02-40G)
for guest gaming with PCI-passthrough with Win 10 and Linux guests, bonus I can use FreeSync with the monitor

Fractal Design - Meshify C Dark TG ATX Mid Tower Case

EVGA GQ Series 750 GQ 750W ATX 2.3 (210-GQ-0750-V2)

HDMI KVM Switch 4 Port with Cable Kit HDCP Support 1080p

Acer KG241Q - LED Monitor - 59.9 cm (23.6 ") - 1920 x 1080 Full HD
best budget gaming monitor I could find

logitech G710 + keyboard

Reminder, I already have a nice gaming mouse.

MSI GeForce GT 710 1GD3H LP, 1GB DDR3, VGA, DVI, HDMI (V809-1899R)
Host graphic card, this was the cheapest I could find, I only needed it for the host and Netflix

Build cost for everything is around 1,700 Euros. I may need a Win 10 license costing 161 Euro if I can’t move the Win 7 license off the laptop. It is a student Win 7 Pro license.

Any suggested changes? Any holes in this build to poke a stick into?

Will I have the AMD reset problem with the Vega 56 card? I think I read somewhere that this is solved?

Later on I can upgrade as needed or makes sense, this build is to move me off the laptop and provide a nice boost in performance.

20

In place of the

Sapphire Pulse Radeon RX Vega 56

would I be better off with a

Sapphire Nitro + Radeon RX Vega 56

card?

I just spent about 8 hours reading all I could find. The Nitro+ is of course more expensive. However, the Pulse is not a good choice if it has a reset bug or has the reset bug been fixed?