Unable to setup IOMMU/VT-d and VM on compatible hardware - Ubuntu 19.04

Use PPA, or compile from source, or just use a different distro.

Yesterday installed Manjaro (so arch). Following the tutorial of the arch wiki works perfectly until now (with a small ecxception that nowhere was noted that you should update/generate a new grub.cfg file)
If I run the DMESG commands I correctly see that IOMMU is enabled.
So lets hope the rest goes well and I have a fully functional gaming VM!

You’ll want to use an 1440fx chipset instead of Q35. The new QEMU audio driver fix has some issues with Q35 btw

Thanks for pointing that out!
Any idea if Manjaro has vfio-pci as module loaded by default in the kernel? Or should I follow the steps to add it manually in the /etc/mkinitcpio.conf and /etc/modprobe.d/vfio.conf?

You have to rebuild the initramFS after adding the pciIDs to modprobe so that the GPU that you’re forwarding properly gets grabbed by VFIO before your video driver. Literally just add the pciID to modprobe. Add VFIO to mkinitcpio.conf. Finally sudo mkinitcpio -P and reboot.

After that dmesg | grep -i vfio and you should see your GPU grabbed by VFIO and you’re ready to add it to your VM inside libvirt.

Thanks that did the trick. Firstly i tried to ad the configure the module manually, but dmesg didnt gave the correct result. When doing lspci -nnk -d also showed me it was still using the nouveau driver.
Then i tried to add it in the grub conf. Stil same result.
Why is it that in almost every tutorial some steps like sudo mkinitcpio -P are missing? Same goes for sudo grub-mkconfig -o /boot/grub/grub.cfg after changing the grub conf. If i didn’t execute it it will still show me iommu was disabled. Hope this will be the last thing that annoys me. I’m currently writing myself a “tutorial”/backlog to be able to reproduce this in the future though…

1 Like

Because you’re using Ubuntu tutorials and Ubuntu has stupid scripts to rebuild initramFS when you change grub. Just only use Arch Wiki from now on and delete everything Ubuntu from your brain. They do everything ass backwards.

Is it needed to use the custom virtio-win iso from redhat for the drivers while installing the machine? The wiki does not list anything about this…

EDIT: It is necessary to install viostor AKA virtio windows SCSI driver before installing windows for maximum virtual HDD throughput. You mount the virtio-win iso disk BEFORE windows is installed and search for the viostor driver. Once that’s installed your drives using virtio storage should pop up after its installed.

Once inside the window VM, if it has internet connectivity, either google and download windows guest spice drivers

or use this link, https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-latest.exe

Do you recommend to pass the GPU through directly before installation or use QXL to do the installation process and add it later?

QXL before. Once the computer is fully setup with all of the programs, LookingGlass, the reverse SSH to detach MKB etc… Then you’re free to delete the QXL driver inside windows, but keep it in libvirt. Do not delete QXL until after the video drivers for your passed through GPU are installed. So right before you delete QXL you’ll want to passthrough GPU. Install drivers. Reboot. Delete QXL. Reboot again and you’re golden.

Thanks!
I saw you mentioned the ‘ssh way’ earlier to bind mkb to the vm. But in the arch wiki they talk about setting it up via Evdev.
Is there a preferred way?

I added a virtio-scsi device (now listed as “Controller Virtio SCSI 0”). But I cant seem to find how to change the sata disk to it. Or is it as easy to select the SCSI option in the advanced options: ‘Disk bus’ ?

EVDEV is for this,

The reason we’re using reverse SSH instead of EVDEV is because you’re not using a second set of mouse and keyboard. Because once you hand your MKB over to guest, how do you hand them back to host? With reverse SSH. Or if you have two sets of MKB, then you use a KVM switch or evdev.

This is how your HDD should look,

The storage format defaults to qcow_2. ‘raw’ is the preferred in this case i think?

The wiki mentions the following: “… swapping control of your mouse and keyboard between the host and guest by pressing both the left and right control keys at the same time.”
And nothing about reverse SSH to host.
It seems SSH is a bit overkill?

I’m telling you the correct way to do it. You’re conflating spice control vs handing physical control of the device to the VM. Spice control is laggy AF. Handing the devices over to the VM = full speed and 0 latency. But do it your way buddy.

qcow2 is fine, but all the benefits of using it (such as live snapshots) are negated because we’re passing through the GPU. Also raw is much more performant, especially if you intend to play video games. If you aren’t passing through the GPU then yeah use qcow.

You seem to have forgotten that I linked you to this earlier,

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Swap_peripherals_to_and_from_the_Host

This process, although not explicitly stated as “reverse SSH” is by definition reverse SSH since we’re using it to connect BACK to us rather than an actual remote device you know?

Sorry for asking seemingly “stupid” questions.
Yes I could’ve read the whole article again.
Well actually i did, but not in too much exact detail. But i did read about the part linked by you. Thanks for pointing that out again.
The first time creating a setup like this always gives some hiccups, or in this case a lot…

I ran into the cpu problem again but managed to fix it. But while doing so i noticed that on the Windows task manager the cache is shown as Not Applicable. Should this concern me?
This was also the case on my last ubuntu config, but i din’t notice it earlier.

I’m now trying to passthrough the gpu. Do the same options in the XML file apply as shown in the ubuntu tutorials linked earlier? Or is less/more needed on Arch?

1 Like

You’re fine bro. I didn’t mean to get snarky with you. I’m sorry. I’m eating. I’ll respond in a bit.

Here’s what my task manager looks like in my VM and I’m getting full speed emulation. The only thing i haven’t enabled yet is static hugepages.

Here’s the thread for the audio fix but before you do this you need to type sudo virsh NAMEOFVM and then edit the domain line to this <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>.

Once you do that you need to edit /etc/libvirt/qemu.conf and change the user option to your username.

If you aren’t passing though a GPU and using LookingGlass you’re done. You might want to enable static hugepages as I mentioned earlier as you get a pretty dank performance boost but besides that there isn’t much else.

Finally, if you aren’t passing through a GPU do not expect to be able to play video games at all. I’d rather try to play HL2 on a gameboy than the QXL driver and software emulation.

Ha lol I see I made a typo in my last post. I meant I: “I’m NOW trying to pass it through”. (I’ll edit the post later)
So I was actually asking what to do to make that happen :slight_smile:

Is the audio via that qxl interface working in another way than after passing through the gpu and using looking glass? (I also saw looking glass wont work without passing through the gpu since it can’t find a device to read from when you don’t)
The sound is working at the moment though…

About the cpu thing, I will look into it later today. When I compare your task manager against mine I see one difference. I only see the chart of one core. But it do says it has 3 virtual cores… Well that’s odd…

EDIT: I got the gpu working now. Driver is installing as we speak. I changed the first domain part and added <qemu:commandline> <qemu:arg value='-cpu'/> <qemu:arg value='host,hv_time,kvm=off,hv_vendor_id=null'/> </qemu:commandline> before closing the domain part on the end of the file.

EDIT2: I got lookingglass working now. For now i just pass trough a set of mkb directly to the guest so i can test the setup. Later i will try the ssh method as described in the wiki. But when running lookglass its not very snappy. If i switch over to the direct output of the gpu to the monitor i get the snappiness i want. But through looking glass not so much. There is a huge delay between input and actually seeing it. probably 200ms or something. Or is 4k simply too much to handle for looking-glass at the moment? I needed to set the amount of memory to 128 according to the formula… since i was a bit over the 64mb mark…

I also saw that gnif added 2 releases yesterday to the looking-glass git. B1-r1 and r2. Should i build myself the latest one (with visual studio after i figure out how that works exactly) and use the latest lookingglass build from the git?
Ah, i see that there are also precompiled executables (for windows) downloadable from his website… Well lets try that then.

EDIT3: Tried newest version. It seems a bit faster. but same “problem” is still there. Does the QXL driver installed in windows has something to do with this perhaps? Another interesting thing is that if i start the client in windowed mode it seems faster than fullscreen. Is the integrated gpu not fast enough to render those frames?

Above 1080p with looking glass is YMMY, 4k is definitely tough. The iGPU can’t handle it though, check with intel_gpu_top (if installed) and you’ll probably see it’s at 100%. I was initially using the iGPU on my 7700k but it couldn’t do 3440x1440 properly. I added a Vega 64 since it was a good price and it’s much better, not perfect but very playable for me. Also it can be a little sluggish if the CPU isn’t turbo’d all the way up but that may be due to the kernel I’m using, if I force to performance governor it’s smoother.