Ubuntu 18.04 VM Host adventure

[raun0]

Hi,

I am going to run virtual machines using Ubuntu 18.04 at a host. I have two R9 Nano graphic cards on my setup. I am looking for third gpu to boot and to control host.

I spotted Gigabyte GC-IVA graphic card that is PCIe 1.0x graphic card for light use using Aspeed AST1300. I suggest that is going to boot in full UEFI mode but i haven’t find any proof for that.

My X99 Strix motherboard do not have option to select GPU to boot but I am looking for bios mod for that.

I do not managed yet override amdgpu driver with vfio-pci driver. What is the most possible way to success with Ubuntu? Initramfs-tools/hooks, initramfs-tools/modules or something else?

There is also some kind of difference with vfio-pci before kernel 4.18.16 and after.

Is there even any simple way to tell linux kernel which GPU to use to boot and disable the rest?

Do I need double vfio-pci IDs in options when I have two cards with same IDs?

[Cromern]

If you read this thread, it will show you how to blacklist and such. Myself i’m kind of stuck on virt-manager at the moment

Ubuntu 17.04 -- VFIO PCIe Passthrough & Kernel Update (4.14-rc1)

DRAFT of the article for ubuntu passthrough setup from the live stream. Images are missing for right now. Any comments? A video is being edited that goes through the how-to step by step. Step 1 install ubuntu step 2 install ukuu sudo apt-add-repository -y ppa:teejee2008/ppa sudo apt-get update sudo apt-get install ukuu Use ukuu to install the latest kernel. In our case we want Kernel 4.14-rc1 because that is the latest kernel as of the time of this writing and this kernel is known to work on this hardware configuration, even though it is a pre-release kernel. This kernel, or later, is also important for AMD Ryzen or Threadripper users as it contains a number of updates for those platforms. The update utility will often fail the first time due to the cache folders not being created automatically. Just run it again and it will work the second time. Notice that as it updates the kernel, it also automatically runs update-grub and update-initramfs and that it complains about missing microcode. It isn’t clear if the microcode is required, but the microcode is non-free. It can be found on the internet, however. We must also add some kernel parameters and re-run the update commands above, but that can be done later. step 3 (optional, skip) find microcode and copy to /var/lib/firmware step 4 (doing the vfio stuff) We are ready to setup a virtual machine and pass through real graphics hardware to the virtual machine. To start, we need to make sure the host platform has been configured correctly: That hardware virtualization extensions are enabled, and that the IOMMU groupings are suitable. First, make sure that vt-d (or SVM on the amd platform) is enabled in the UEFI. This varies from platform to platform but typically Level1 Techs motherboard review cover where these options are and what the IOMMU groupings are for particular motherboards. Most Linux distros do not enable IOMMU by default. You will need to update your grub bootloader config to support IOMMU. F…

Also, after editing /etc/initramfs-tools/modules , do a “sudo update-initramfs -u”

[raun0]

I have tried almost every thing.

dmesg is still:

raun0@x99:~$ dmesg |grep -i vfio
[ 10.349057] VFIO - User Level meta-driver version: 0.3
[ 10.376555] vfio-pci: probe of 0000:03:00.0 failed with error -22
[ 10.377072] vfio-pci: probe of 0000:01:00.0 failed with error -22
[ 10.377545] vfio_pci: add [1002:7300[ffff:ffff]] class 0x000000/00000000
[ 10.378053] vfio-pci: probe of 0000:03:00.0 failed with error -22
[ 10.378568] vfio-pci: probe of 0000:03:00.1 failed with error -22
[ 10.379035] vfio-pci: probe of 0000:01:00.0 failed with error -22
[ 10.379552] vfio-pci: probe of 0000:01:00.1 failed with error -22
[ 10.379993] vfio_pci: add [1002:aae8[ffff:ffff]] class 0x000000/00000000

Biggest problem is that I can’t change the boot GPU.

[raun0]

Is there anything that can be done to wool the mb? What if I delete UEFI drivers from tha VGA bios? Then maybe MB select the only card with the uefi drivers to boot that will be used for host machine.

[raun0]

I have made some progress. I succesfully booted Ubuntu server and desktop headless.
Both GPUs running with vfio driver. What I didn’t realise at first was that Ubuntu server do not use
“GRUB_CMDLINE_LINUX_DEFAULT=” - line to options, only “GRUB_CMDLINE_LINUX=”.

I also blacklisted both nvidia and nouveau in /etc/modprobe.d/blacklist-nvidia-nouveau.conf
“blacklist nvidia”
“blacklist nouveau”
“options nouveau modeset=0”

Also I add With graphical output it is much more diffucult to handle display output drivers so I am ending to use vm host headless over ssh or/and web interface.

[raun0]

Could somebody paste here example of libvirt xml table with gpu passtrough and ovmf uefi boot?

Actually I do not fully understand how to get gpu passtrough get work right away with virt-install, I got strange coloring in the monitor but nothing more.