Passing USB controller - requirements?

Im running a Windows 10 VM using qemu version 2.10.1 on kernel 4.14.1 with a GPU passed through. I can also pass through an Intel-based USB controller. So far so good.

My Intel USB controller does not support resetting which is causing issues. I have another onboard USB controller - ASMedia based. Supposedly it supports resetting:

IOMMU group 12
[RESET]	04:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:2142]

But when i try to pass it, instead of my intel one, the VM is unable to boot. ‘dmesg’ gives me:

[  389.368990] device vnet0 entered promiscuous mode
[  389.369151] virbr0: port 2(vnet0) entered blocking state
[  389.369153] virbr0: port 2(vnet0) entered forwarding state
[  390.360933] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[  390.361265] vfio_ecap_init: 0000:01:00.0 hiding ecap [email protected]
[  391.444984] pcieport 0000:00:1c.4: AER: Uncorrected (Non-Fatal) error received: id=00e4
[  391.444999] pcieport 0000:00:1c.4: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=00e4(Requester ID)
[  391.445007] pcieport 0000:00:1c.4:   device [8086:a294] error status/mask=00100000/00010000
[  391.445012] pcieport 0000:00:1c.4:    [20] Unsupported Request    (First)
[  391.445017] pcieport 0000:00:1c.4:   TLP Header: 34000000 04000010 00000000 17ff17ff
[  391.445027] pcieport 0000:00:1c.4: broadcast error_detected message
[  391.445034] pcieport 0000:00:1c.4: broadcast mmio_enabled message
[  391.445036] pcieport 0000:00:1c.4: broadcast resume message
[  391.445041] pcieport 0000:00:1c.4: AER: Device recovery successful
[  391.445153] vfio_ecap_init: 0000:04:00.0 hiding ecap [email protected]
[  393.781020] pcieport 0000:00:1c.4: AER: Uncorrected (Non-Fatal) error received: id=00e4
[  393.781033] pcieport 0000:00:1c.4: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=00e4(Requester ID)
[  393.781041] pcieport 0000:00:1c.4:   device [8086:a294] error status/mask=00100000/00010000
[  393.781047] pcieport 0000:00:1c.4:    [20] Unsupported Request    (First)
[  393.781052] pcieport 0000:00:1c.4:   TLP Header: 34000000 04000010 00000000 17ff17ff
[  393.781061] pcieport 0000:00:1c.4: broadcast error_detected message
[  393.781079] pcieport 0000:00:1c.4: broadcast mmio_enabled message
[  393.781082] pcieport 0000:00:1c.4: broadcast resume message

Both of the USB controllers are in their own IOMMU group which they dont share with anything else

Why is this happening? Is the ASMedia missing some required hardware feature i dont know about?

vfio-pci sometimes doesn’t like USB controllers. It’s more for GPUs.

You could try using pci.stub kernel arguments for USB controllers instead. I’ve had to do that with a Fresco Logic controller to make it unbind to the host OS.

1 Like

I just tried doing exactly that, but with no success

IOMMU groups:

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3ec2] (rev 07)
IOMMU Group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a2c9]
IOMMU Group 10 00:1f.2 Memory controller [0580]: Intel Corporation 200 Series PCH PMC [8086:a2a1]
IOMMU Group 10 00:1f.4 SMBus [0c05]: Intel Corporation 200 Series PCH SMBus Controller [8086:a2a3]
IOMMU Group 11 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]
IOMMU Group 12 04:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:2142]
IOMMU Group 13 05:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 [144d:a804]
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation Device [8086:3e92]
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation 200 Series PCH USB 3.0 xHCI Controller [8086:a2af]
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation 200 Series PCH CSME HECI #1 [8086:a2ba]
IOMMU Group 5 00:17.0 SATA controller [0106]: Intel Corporation 200 Series PCH SATA controller [AHCI mode] [8086:a282]
IOMMU Group 6 00:1b.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #17 [8086:a2e7] (rev f0)
IOMMU Group 7 00:1c.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #1 [8086:a290] (rev f0)
IOMMU Group 8 00:1c.4 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #5 [8086:a294] (rev f0)
IOMMU Group 9 00:1d.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #9 [8086:a298] (rev f0)

lspci -nnv:

04:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:2142] (prog-if 30 [XHCI])
Subsystem: ASUSTeK Computer Inc. Device [1043:8756]
Flags: fast devsel, IRQ 16
Memory at df200000 (64-bit, non-prefetchable) [disabled] [size=32K]
Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
Capabilities: [68] MSI-X: Enable- Count=8 Masked-
Capabilities: [78] Power Management version 3
Capabilities: [80] Express Legacy Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [200] #19
Capabilities: [300] Latency Tolerance Reporting
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci

But dmesg still gives me the same error when trying to start my VM:

[  119.685940] pcieport 0000:00:1c.4: AER: Uncorrected (Non-Fatal) error received: id=00e4
[  119.685953] pcieport 0000:00:1c.4: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=00e4(Requester ID)
[  119.685962] pcieport 0000:00:1c.4:   device [8086:a294] error status/mask=00100000/00010000
[  119.685968] pcieport 0000:00:1c.4:    [20] Unsupported Request    (First)
[  119.685973] pcieport 0000:00:1c.4:   TLP Header: 34000000 04000010 00000000 17ff17ff
[  119.685981] pcieport 0000:00:1c.4: broadcast error_detected message
[  119.685989] pcieport 0000:00:1c.4: broadcast mmio_enabled message
[  119.685992] pcieport 0000:00:1c.4: broadcast resume message
[  119.685996] pcieport 0000:00:1c.4: AER: Device recovery successful

Whats going on? Why is the PCH SMBus device going crazy when i try to pass a device (thats in a different IOMMU group and even uses a different driver)

It doesn’t look like pci.stub binded to the controller. Did you verify it binded to the controller?

This may be a similar situation to PCI-E Errors on Threadripper. You could check the threads related to that for the kernel boot parameter to fix it.

Hmm? lspci clearly says that the device is claimed by vfio-pci

As for confirmation, the USB ports do not work in the host

What impact does the USB bridge have? It does not seem possible to pass the bridge device

Trying booting with this added kernel parameter.

pci=noaer
1 Like

pci=noaer works! Interesting. I wonder what kind of consequences using this actually entails… Ill look into it. Thanks for the help

I was actually seeing the same but with the GPU ID instead of my USB controller that I also pass through. I found that I needed it after a kernel update, or maybe it was even after updating my BIOS, they were done so closely together that I’m not sure. Originally I thought it was my board because I happened to de-lid as well and some of the liquid metal got onto the board and I did my best to clean it but I thought it was still having issues so I tried reseating the card and I even replaced the board and still ended up needing it. It’s been a few months and no adverse affects at all that I’ve noticed.

1 Like