GPU passthrough quick questions

I’ve 3 issues I’d like to get cleared up before I attempt this again. Previously I did it on fedora and got it up and running but it was really unstable but I’ll not get into that. I’m trying on ubuntu this time. So:

  • When my virtual machine is not in use, can I use the secondary monitor with the host? Previously I couldn’t and I’m guessing it’s because host has no recollection of passed through gpu. This is a big factor for me as it restricts me to one monitor during downtime.

  • This one should be straight forward, I have a usb switch, do I pass the device or the usb controller it connects to? Previously I couldn’t get the guest to detect the mouse and keyboard after switching.

  • Finally the biggest issue I had before was the whole passing through a physical ssd and using virtio. Previously I couldn’t get it to work with an already installed windows 10 ssd. Also installed virtio drivers but hadn’t much success. WHat is the steps for this virtio driver part? (Step by step instructions would be helpful at this stage as I’ve over-thought the whole passing through block device part).

  1. Yes, but the second monitor needs also to be hooked up to the host-gpu and you need to switch the input to that. I have two monitors, both in use when no VM is running. One monitor is connected to host and guest GPU (different inputs) and i switch to the different inputs depending on if i want to use guest or host.

  2. I’m passing through the mouse and keyboard directly. And for hotplug i pass through a controller. I’m not passing a USB-hub because hot-pluging and detecting does not quite work reliably with those.

  3. I do not know about passing through a whole disk. I use a partition with “raw” for the HDD in the guest. As for using VIRTIO in the guest i passed through an additonal virtual disc-drive with the virtio iso in it. When choosing where to install windows to, hit the add dirvers button and go to the iso-drive and install the virtio drivers from there. After installing the drives should show up in the list.

Oh so for the monitors you just change the input source on the actual monitor? This could be an issue for me as my secondary monitor only has DVI and VGA so there is no extra port I can use. Could I use a KVM switch in this case?

I will not be passing the mouse and keyboard through this time as I want to be able to switch between the two with the one set of keyboard and mouse. Synergy did not work with games the last time I tried so I will have to tinker with the usb switch and hopefully get it working.

I already have windows installed on an ssd, I’m hoping I can pass through the physical ssd using SATA or IDE and then try the virtio drivers after.

Anyway I am going to get stuck into this.

Yes, in fact, this is encouraged.

My recommendation for this: pass through a PCIe USB card or a secondary onboard controller. That way you can just run USB leads from the KVM to the VM.

Install the virtio drivers on windows, while it’s booted bare-metal, before doing the passthrough, then just use virtio. SATA/IDE are terribly inefficient.

Yes this sounds promising. I’ve been going through the Ubuntu tutorial and ran into my first problem. Initially I had a standard Ubuntu install with an rx480. I connected my nvidia GPU and removed dvi cable from rx480. Nouveau installed fine and I went through the initial steps of declaring the iommu ids in the various files. Unfortunately the boot process hangs at vfio when I’d expect nvidia to be loaded first as I declared it to load first in bios but it gets overridden every time.

Okay, so you’re passing through nvidia or amd?

You’re going to have to tell me exactly what you’ve done, otherwise it’s going to be hard to diagnose.

Have you loaded your vfio modules into the initramfs? have you rebuilt your initramfs?

Yes

Passing through RX 480, 750ti host.

Edit: Booted up live CD and recovered system back to default before vfio changes. Will try get it working tomorrow.

1 Like

Hmmm, You’ll have to share your configs with me. I’m not entirely sure what’s gone wrong.

/etc/default/grub:

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=1 amd_iommu=on"
GRUB_CMDLINE_LINUX=""

/etc/initramfs-tools/modules:

softdep amdgpu pre: vfio vfio_pci

vfio
vfio_iommu_type1
vfio_virqfd
options vfio_pci ids=1002:67df,1002:aaf0
vfio_pci ids=1002:67df,1002:aaf0
vfio_pci
amdgpu

/etc/modprobe.d/amdgpu.conf:

softdep amdgpu pre: vfio vfio_pci

/etc/modprob.d/vfio_pci.conf:

options vfio_pci ids=1002:67df,1002:aaf0

After adding all these lines and files I will run:

  • sudo grub-mkconfig
  • sudo update-grub
  • sudo update-initramfs -k all -c

I’ll get back to you on how the boot process went.

Update: Same happens again, hangs at vfio as shown in previous attached image.

Sorry i’m on my phone so i might have overlooked sth in your posts.
Dumb question, but have you enabled virtualization stuff in the bios?

Yes I have, SMT is enabled and IOMMU is enabled.

I am missing the vfio ids in your grub. Do they not have to be there too?

EDIT:
in my grub it says sth like:
GRUB_CMDLINE_LINUX=“vfio-pci.ids=xxxx:xxxx,yyyy:yyyy”

I added the ids to grub but to the default param as that’s the one used in tutorial but I don’t think it should matter which one, @SgtAwesomesauce may be able to confirm that. Unfortunately this still didn’t help. I’m starting to wonder is it related to nouveau drivers? I’m not blacklisting it…

I’ve had trouble with them working when they’re not in the cmdline, on a few distros, but I would only change it if it’s not working, like this situation.

And yes, your syntax is correct.

1 Like

Did you solve it?

No unfortunately as I can only work on this at the weekends so I’ll be looking at it again tomorrow. I’ll probably start fresh again and use proprietary drivers for nvidia host GPU and go from there. I’ll update this post when I do, thanks for keeping tabs on it :smiley:

1 Like

3rd Question:
Step 1) Add a random Virtio Drive which is not the one your booting from, then add a Virtio SCSI Controller. (This is for testing, it should mount the drive automatically after installing the driver later on)
Step 2) Mount your virtio iso to a virtual ODD using IDE/SATA Bus.
Step 3) Boot into safe mode and install Virtio SCSI Controller, again the test drive should be mounted at this point.
Step 4) Your bootdrive can be changed to Virtio without having the Inaccessible Boot Device Error.

1 Like

Looks like you are passing through your primary graphicscard. You can tell that where the UEFI/BIOS splashscreen appears. If you do this on the other card my guess is it would work. Correct me if I’m wrong

I can confirm that passing through the primary graphics card works provided it supports a PCI reset correctly. It works fine for NVidia, but AMD Vega have reset issues that prevent it from being used as the primary boot device.

I’ll try this at the weekend and get back to you.