Return to Level1Techs.com

Trouble loading vfio-pci drivers into GPU

#1

There is a bit of bloat in the article to show where this project is going and why I’m not choosing certain paths I’ll try to hide this as much as I can.

I’ve been working on this project for the past week and the goal is:

  • To run my steam library (I don’t mind losing some of my games but I would like to keep the VR ones)
  • To run the enterprise version of office that my university gives me (that includes onedrive)
  • The ability to easily control and share files with my home server and my VPS (both run vanilla Debian with OMV installed)
  • Remote access to my pc
  • Having all my monitors working when I play games like quake 3, that includes G-sync
  • Development for Moto Mods
  1. Windows 10

So far windows can do a lot of that however remote access, unless going though team viewer has been a bit of a pain, also docker has a bit of a pain to work with on and you cant work on moto mods without linux.

  1. Unraid

Unraid is out of the question, I’m not up for buying more software at this moment and I would like some flexibility for storage.

  1. Ubuntu

I’ve tried Ubuntu until 3 days ago, because no matter how many times I use it, it always freezes, changes audio devices at the worst times and crashes right in the middle of an essay saving. (I have been using Ubuntu for more than half my life It’s my go to, and yet it still hates me :disappointed_relieved:)

  1. Fedora/CentOS

This is a tricky one, I love a lot of the things that these distros do, and I don’t mind change. However I’ve always had issues with CentOS’s firewall and when I started using Fedora I gave up fixing the same issue after 30 minutes

  1. Debian/OMV/#!++ (final choice)

Debian has always been my best friend since I was allowed to have my first computer, my raspberry pi. Never had a crash on anything wacky that I’ve done. Debian on android phone, no problem, tv? never crashed, how about a ds works like a charm. Debian has been quite digital for me, if it installs it works, if it doesn’t don’t bother. I like the idea of having a lightweight computer as possible, because then i can add all the bloat i want and blame my self later on. Until the beginning of this year I’ve been using cruchbang plus plus for my servers, and for my desktops. After giving away some hardware I wanted a more noob friendly solution that isn’t locked down, that’s why i choose OMV. Crunchbang has updated to Debian 9 so I’m moving my desktop to that now.

Where are the issues
SMB bugs out on windows - still broken, not using windows
Not all VR titles are ready for linux - fixed with proton
Some VR titles bug out with proton - broken
Windows instance cant be moved - broken
Cant restore windows image made with clonezilla - broken
G-Sync cant work with looking glass - broken, however still works with direct pci-e passthough
G-Sync doesn’t work on linux - fixed with proprietary drivers
Enabling intel_iommu=on for ubuntu grub causes the system to hang on boot - broken??
Moving encrypted ubuntu partion to a seperate drive, booting in then decrypting corrupts all the files including system files - broken
The installer/live disc for ubuntu can handle multiple screens over multiple graphics cards but the installed version cant - broken
Having multiple xscreens for different graphics cards, in ubuntu, displays a black screen at all times with some sort of debug cursor on it when the mouse is in that window - broken but only in the default GNOME unbutu desktop manager
In Ubuntu all desktop managers go into a login crash loop until the power is removed - broken but some how i got it working in xfce
The XFCE desktop manager in ubuntu only opens apps in the centre of one screen, all other screens “work” however all applications are glued to the one spot and the computer only reacts to the first one opened, also none of the screens close and the system won’t go to the login screen until a reboot - broken
iommu wont enable on debian - fixed, the system wasn’t parsing intel_iommu=on because i forgot to add GRUB_CMDLINE_LINUX_DEFAULT= before it
iommu will be enabled 90% of the reboots - broken, i have no idea about this one i run virt-host-validate every reboot to check this
In debian when running the kvm with the pci-passthough libvirt crashes, system resources behave normally but the system encounters weird bugs like chrome not opening tabs, until libvirtd is started or the system is rebooted - broken, I’m guessing because vfio isn’t bound to the card.
In debian no matter what happens the proprietary nvidia drivers are bound to the gpu, even when i disable it and enable the open source drivers - fixed, i didnt know that you can cram as many “scripts” into /etc/modprobe.d/ as you want
In debian the vfio drivers never load, not even into kernal modules - broken

here is what i have run to get it working
virt-host-validate for testing iommu (this boot it fails)
virt-host-validate.txt (3.2 KB)

lspci -vnn    for finding device id and iommu group

it comes out with this result
lspci.txt (31.2 KB)
That gives me the group of 02:00.x and the id’s [10de:1b81] and [10de:10f0]

here are the “configs” that I have made/edited (in .txt form for the forum)
blacklist.txt (94 Bytes)
vfio.txt (76 Bytes)
grub.txt (1.4 KB)
modules.txt (250 Bytes)

before i reboot i always do sudo update-grub and sudo update-initramfs -u
none have come up with any errors unless i have nouveau in the initramfs modules file
I still don’t have any boot problems
I have updated the Linux kernel to the latest “non-broken” version in synaptic, with the experimental repo installed same issue
I have changed gpu’s, from a quadro to a raedon
I have followed these guides
https://wiki.debian.org/VGAPassthrough



https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF


I am in a bit of a pickle
If anyone has any advice that would be really helpful

0 Likes

#2

QEMU: Checking if IOMMU is enabled by kernel : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)

You need to just add the additional parameters to the grub file with a space between the "quiet intel_iommu=on vfio " etc , at the moment you have ‘GRUB_CMDLINE_LINUX_DEFAULT’ declared 4 times

also, go into terminal and type ‘sudo update-grub’ and then ‘sudo update-initramfs -u’ and reboot

0 Likes