Fedora 29 and GPU Passthrough Permission Denied

Issue

When running a VM session in virt-manager aka Virtual Machine Manager, an error pops up as

Error starting domain: Failed to open config space file ‘/sys/bus/pci/devices/000:01:00/config’:
Permission denied

And the details shows up as:

Error starting domain: Failed to open config space file '/sys/bus/pci/devices/0000:01:00.0/config': Permission denied

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1420, in startup
    self._backend.create()
  File "/usr/lib64/python3.7/site-packages/libvirt.py", line 1080, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: Failed to open config space file '/sys/bus/pci/devices/0000:01:00.0/config': Permission denied

Context

In virt-manager VM session configuration window:

Add Hardware > PCI Host Device > the 2 NVIDIA devices in the list.

When adding the PCI and running the VM, the error message as mentioned above in the Issue section, pops up. If PCI devices aren’t added, VM will run fine.

Procedures

IOMMU appears to be enabled:

# dmesg | grep IOMMU
[    0.176553] DMAR: IOMMU enabled
[    0.219395] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1

Grub had been updated:

# cat /etc/default/grub | grep CMDLINE
GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet intel_iommu=on vfio-pci.ids=10de:1c02,10de:10f1"
# grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

Even dracut was used (correctly?):

# cat /etc/dracut.conf.d/vfio.conf
add_drivers+="vfio vfio_iommu_type1 vfio_pci vfio_virqfd"`
# dracut -f --kver `uname -r`

# lspci -nnk output indicates vfio-pci kernel driver is in use:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] [10de:1c02] (rev a1)
	Subsystem: eVga.com. Corp. Device [3842:6160]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)
	Subsystem: eVga.com. Corp. Device [3842:6160]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

I’m missing something here. Whelp!

Brief background

I have had an successful attempt to passthrough my PCIe device in the past on a different distro, but can’t seem to figure the stuff out on Fedora 29. I’m new to Fedora.

Thank you guys in advance.

Is selinux disabled or set to permissive?

Are you doing this as root? Or as a regular user, have you added yourself to the qemu, kvm and libvirt groups? There might have been a vfio group too?

That’s about the only things special I recall doing when I was experimenting with passthrough in f29.

2 Likes

All of my examples shown were ran in root. I just redid them all using my primary user and have yet to reboot my machine. But in the meantime, I attempted to add myself to libvirt group (using a regular user with sudo when necessary) like so:

$ sudo usermod -a -G libvirt $(whoami)
$ newgrp libvirt

Uncommented these lines from /etc/libvirt/libvirtd.conf:

unix_sock_group = "libvirt"
unix_sock_rw_perms = "0770"

virt-manager now runs without password prompt (which is nice, so thank you for that!). But since the primary issue remains, I added myself manually to qemu and kvm as well:

$ sudo usermod -a -G kvm $(whoami)
$ newgrp kvm
$ sudo usermod -a -G qemu $(whoami)
$ newgrp qemu

Admittingly, I’m not too sure what newgrp qemu is doing, but just following steps here.

I’m rebooting now and see what happens.

Unfortunately no changes. Still digging.

Run

setenforce 0

As root, does it work then?

1 Like

Just rebooted after your suggestion. I’m grateful, but it didn’t do the trick. Thanks.

Don’t reboot, it nullifies the command - it’s a temporary setting

Yeah, I tried running it before rebooting. Just in case though, I setenforce 0 again and tried it to be sure. And nope. This is so frustrating.

Check that the file permissions are correct as well, they might have changed if you were first running virt-manager as a priviledged user.

/Then/ set selinux unto permissive mode with setenforce 0

sudo chown ‘username’ /path/to/image

I’m sorry, I’m gonna need help with that one. As of right now I dropped all storages from the VM to see if it would boot at all with the PCI devices inserted. I’m assuming that we’re talking about the file permission on the VM’s actual path? If so, where would I find them?

The VM images reside in whatever path you’ve saved them in when you set up the virtual OS.

But for now run virt-manager using sudo and see if it fixes the issue so we get certainty what’s causing it.

$ sudo setenforce 0

$ sudo virt-manager

$ sudo virt-manager 
$ No protocol specified
Unable to init server: Could not connect: Connection refused
No protocol specified
Unable to init server: Could not connect: Connection refused
No protocol specified
Unable to init server: Could not connect: Connection refused

(virt-manager:6130): Gtk-WARNING **: 22:20:58.216: cannot open display: :0

Same when I sudo -i and do them. I don’t think it makes much of a difference. But nothing pops up. No Virtual Manager application window is appearing. Is that bad?

Log in on xorg instead of wayland and try again

I received a bunch of connection error messages, I should’ve paid better attention, but basically clicked through and added new connection to see what happens.

Otherwise the thing is running but I can’t seem to create a new vm:

Error: No active connection to install on.

By the way @Baz, truly appreciate you sticking around man, thanks.

Is the service running?

sudo systemctl start libvirtd

$ sudo systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor pr>
   Active: inactive (dead) since Mon 2019-04-22 22:27:46 CDT; 15min ago

So I $ sudo systemctl start libvirtd. Now it runs still without connections.

Check that you have the package ‘qemu-kvm’ installed and retry.
Sorry I can’t be more verbose but I’m on my phone and it’s a pia to write long posts.

That being said I gotta get back to work so I’ll be afk a while.

No problem, you’ve been plenty helpful.

$ sudo systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor pr>
   Active: inactive (dead) since Mon 2019-04-22 23:10:47 CDT; 3s ago

Eventhough I stopped and restarted the process, it stays inactive (dead). Also, I sudo dnf install qemu-kvm as suggested, didn’t had it installed after all. However, retrying with the same result. Still no connection.

I’m going back to the beginning and see if I may have made mistakes somewhere.

The libvirtd service is loaded and active, what it’s telling you there is that the ‘vendor preset is ‘inactive’’, meaning it’s disabled at boot.

Do you have an option to create a new connection from the drop-down menu in virt-manager?