Linux VM with GPU passthrough/Looking Glass list for Noobs

[ubuntux]

Once Issac Asimov’s Foundation series and prequel books were completed, he conveniently strung together an ordered list of approximately 15 books for us to read. I’ll return to this point in a minute…

I’m not new to Linux/Unix or building hardware but I still consider myself a noob when it comes to vfio, iommu, gpu-p, looking glass, and all related acronyms.

At the end of the day I just want to get a hardware accelerated vm running on my setup.

Whats preventing my success is the sheer glut of information not just in the level1techs forum but over the web in general. It’s overwhelming and hard to determine where I should start which is where an “Asimov’s list” for noobs comes in.

I’m sure there are people on this forum who have installed a second gpu, gone into the bios, jiggered their iommu group, installed kvm/qemu (with virt manager?) and magically got looking glass to work.

If this is you (any reader), can you provide your own “Asimov’s list” pointing to the various online tutorials and/or forum threads that helped you get from start to finish?

The best tutorial I’ve found (which is noob legible) is this. But there is zero mention of looking-glass in that post — do I even need looking glass? I’m just trying to get GPU acceleration for Photoshop and a few other apps in my VM to work. My spare GPU is an older GTX1050ti, my main GPU is an RTX2080.

[redocbew]

For a VFIO setup, no you don’t need Looking Glass. If you don’t use LG, then you’ll need to dedicate a separate monitor, or an input on a shared monitor to that VM. With Looking Glass, you can capture the video output of the VM and display it on the host like any other application.

If you are going to use Looking Glass it’d be the last step of the process after the VFIO setup is already working on its own.

[FooLKiller]

You can follow the guide you found. It’s the same one I referenced when getting this setup as the ones I saw were even more lacking in content.

I used this afterwards for Looking Glass: Luxuride / VFIO-Windows-install-walkthrough · GitLab

This is also handy if you want to do single card passthrough: Explaining CSM, efifb=off, and Setting the Boot GPU Manually - The Passthrough POST

Some caveats:

Modern AMD 6000 series XT series cards require Above 4G Decoding off or the passthrough won’t work (error 43 on AMD drivers.) This is related to Smart Access Memory. This may be unique to AMD processors, but I haven’t tested that to verify.

Nvidia cards tend to passthrough just fine now.

One thing I never see mentioned is that you should edit the XML and combine the Audio and Video devices you pass-through. This applies to both AMD and NVIDIA.

You do this by setting both devices to the same PCI bus ID, but different functions:

VGA Portion:
address type=“pci” domain=“0x0000” bus=“0x09” slot=“0x00” function=“0x0” multifunction=“on”/

Audio Portion:
address type=“pci” domain=“0x0000” bus=“0x09” slot=“0x00” function=“0x1”/

Simply matching the main bus, and adjusting the function on the audio portion as shown and Libvirt will add multifunction=“on” for you in the main device.

By default, Libvirt assigns each as an independent device and that caused me some headaches getting things functioning until I noticed the default behavior.

I also recommend Scream: GitHub - duncanthrax/scream: Virtual network sound card for Microsoft Windows

Scream allows for Audio passthrough to the host. Runs over the network interface, supports multi-channel audio and the delay is not noticable.

Basically, follow the first guide and get everything working using a second display connection. Once everything works, then add Looking Glass.

If you try to do everything at once, you won’t know where something messed up.

[ubuntux]

Thank you, this is very insightful/helpful.

[bigsqueaks]

Hello,

I’ve been taking notes as I’ve been on a journey similar to that which you sound to be taking. There are several sources for most information that I needed. It sounds like you’ve already got a handle of some of the things you will be working through to get VM’s working nicely.

You were rather sparse on details of your setup. I’m on AMD R9 3900x, Asus x570 tuf gaming (wi-fi), nvidia gtx 1070 (guest), radeon r9 390 (host), 32 GB ram. I’m on Manjaro XFCE.

There are a ton of options to solve various problems. The above poster mentioned something about scream. I would advise AGAINST using scream. For one, I never used it and my sound is fine. Second, I’m pretty sure scream uses ivshmem which looking-glass also uses. I think the dev of looking-glass also speaks against scream, I think it interferes with looking-glass. You may end up finding some information about tuning a buffer length related to pulseaudio (I think), which at some point I did do to fix my sound crackling. pulse audio fixes (obsolete?).

The first resource I used, maybe, was

heiko-sieger.info

When creating my VM setup, I used a spare keyboard and mouse. When I was ready, I followed

Passthroughpo.st evdev guide .

Also, I probably referenced a google search for ‘Arch Wiki evdev’. There are other options, but I love evdev. I’m a huge arch fan simply because the Arch Wiki normally has answers for every linux related task I’m facing, including many vfio topics.

Another comprehensive source I came across was,

Bryan Steiner’s Github

To save myself punching buttons on my second monitor, I use ddcutil, and a libvirt hooks script I got from passthroughpo.st. Ddcutil is not necessary if you run looking-glass; I run different VM’s different ways.

I use Devilspie2 to move my looking-glass window to my second monitor, maximize the window, then remove the window decorations, and remove the xfce4-panel; and upon closing the VM, to do the reverse.

Arch wiki passthrough

Arch wiki kernel parameters

Single GPU passthrough

Yuri / VFIO

GitHub - joeknock90/Single-GPU-Passthrough - Assumptions

Libvirt.org hooks libvirt: Hooks for specific system management

Things that I remember solving coming off the top of my head. If you calculate the size of your looking-glass ivshmem for 1080p, then later change to 1440p or higher, you’ll need to change the size of that /dev/shm file. I forget exactly how it gets created, but modification to the virt-manager xml file for the VM will not re-size it, but just delete it and it will be re-created at the proper size.

I took a time machine to the future, and here I am. When I left, the virtualization technology available to consumers was so incredibly limited and primitive. It’s really exciting because I’m able to completely remove myself from microsoft products from boot. And, basically run my windows needs in a borderless window, as like a live desktop feature.

I think that reminds me I need to remove or check that I’ve removed the microsoft secure boot keys from my BIOS. Eh, I guess I have a BIOS password so whether or not a microsoft bootloader passes signing still requires my input. I’m still waiting for my boot to break by something manjaro does though.

Good luck

[ubuntux]

Thank you! My system is a 3950X, Aorus Master x570 and 64G ram. The main GPU is a 2080 Super and the spare to be passed through is a 1050TI. I game with Steam Linux so I really only need 2D acceleration in the VM with minimal 3D acceleration.