No longer able to pass USB controller to VM

So I use to be able to pass all my USB 3.0 ports to the vm by passing Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller to the vm via virt-manager. Something changed recently in the updates that borked it. Using spice usb redirection works but when it comes to a headset you need complete passthrough so there is no audio issues.

MSI Computer ATX DDR4 Motherboard Z170A XPOWER GAMING TITANIUM EDITION
Intel Core I7-6700K 4.00 GHz 8M Processor Cache 4 LGA 1151 BX80662I76700K

Errors:
Error starting domain: internal error: qemu unexpectedly closed the monitor: 2017-03-21T13:59:52.598605Z qemu-system-x86_64: -device vfio-pci,host=00:14.0,id=hostdev2,bus=pci.0,addr=0x6: vfio error: 0000:00:14.0: group 4 is not viable
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 88, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 124, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 83, in newfn
ret = fn(self, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/domain.py", line 1479, in startup
self._backend.create()
File "/usr/lib/python2.7/site-packages/libvirt.py", line 1037, in create
if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error: qemu unexpectedly closed the monitor: 2017-03-21T13:59:52.598605Z qemu-system-x86_64: -device vfio-pci,host=00:14.0,id=hostdev2,bus=pci.0,addr=0x6: vfio error: 0000:00:14.0: group 4 is not viable

is it in its own IOMMU group? are you using an ACS patched kernel?

Let me know if you can tell based on the picture I attached. Else I can try to run a command. I just need to go digging for that command. Been a while since I set all this up. No I am not using the ACS patched kernel "linux-vfio" because I didn't need to when starting this project. Everything just worked.

throw this in a .sh

for d in /sys/kernel/iommu_groups/*/devices/*; do 
    n=${d#*/iommu_groups/*}; n=${n%%/*}
    printf 'IOMMU Group %s ' "$n"
    lspci -nns "${d##*/}"
done;

and run it, copypaste the output

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Skylake Host Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU Group 10 00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #13 [8086:a11c] (rev f1)
IOMMU Group 11 00:1e.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Serial IO UART #0 [8086:a127] (rev 31)
IOMMU Group 12 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC Controller [8086:a145] (rev 31)
IOMMU Group 12 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H PMC [8086:a121] (rev 31)
IOMMU Group 12 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD Audio [8086:a170] (rev 31)
IOMMU Group 12 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus [8086:a123] (rev 31)
IOMMU Group 13 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8] (rev 31)
IOMMU Group 14 02:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller [1b21:1242]
IOMMU Group 15 04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16) [8086:1901] (rev 07)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:1912] (rev 06)
IOMMU Group 3 00:08.0 System peripheral [0880]: Intel Corporation Skylake Gaussian Mixture Model [8086:1911]
IOMMU Group 4 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
IOMMU Group 4 00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
IOMMU Group 5 00:15.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Serial IO I2C Controller #0 [8086:a160] (rev 31)
IOMMU Group 5 00:15.1 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Serial IO I2C Controller #1 [8086:a161] (rev 31)
IOMMU Group 6 00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31)
IOMMU Group 7 00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] [8086:a102] (rev 31)
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #1 [8086:a110] (rev f1)
IOMMU Group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #9 [8086:a118] (rev f1)

looks to me like the the ASMedia controller is properly isolated but not the intel one. You have to pass through everything in a group typically

Is there a way to correct this or do I need to apply that ACS kernel. Also what are the negatives of doing the ACS patch.

just add both [8086:a12f] and [8086:a160] to your vfio.conf and see if it lets you pass it then. if not, try the ACS patch.

1 Like

Passing both worked. Thanks tkoham. Now I just need to figure out why all of a sudden they are in the same group. Didn't use to be like that.

1 Like

there are no real day-to-day negatives to the patch other than being slightly behind on your kernel version (and having to recompile if your distro doesn't offer a prepackaged one.) The security tinfoils will swoop in and crush me with their massive intellects I'm sure, but if having to pass both is that much of a nuisance, you can always just use the patch