Linux VFIO USB Controller Help

Software & Operating Systems Linux
1

Hi there I’ve been trying to get my Linux KVM setup working. I have
almost everything working except for usb input. While I’m able to
directly bind the usb device via qemu. Unplugging the device,
or if it’s not present causes issues. I have a usb switch, and
alternate between several devices on my desk with it.

TLRD: Recommended USB2/3 Controller. I have a Renesas
But that is not working under Linux, Windows, BSD… Tried recommendations of ECHI over XHCI
Still no go. Alternatively flashing the firmware was recommended.
But again that didn’t work. I have been working with it for several days and no dice.

Put another way how do you hot swap input devices to a vm. I basically have a USB extension cable for each vm
from my Worksation. A hub on my desk, and I just want to be able to plug that into several different vms.
Without having to restart anything.

To start my hardware is

R7 1700X
Gigabyte X370-K5
RX 550
RX 560
HD3450
2 x Renesas USB 3.0 Controller

  • Host
    • HD3450
  • VM1 Linux Dev
    • RX550
    • USB3 Controller
  • VM2 Windows
    • RX560
    • USB3 Controller

***IOMMU Groups

#IOMMU Group 12 0b:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 560] [1002:67ff] (rev cf)
#IOMMU Group 12 0b:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:aae0]
#IOMMU Group 13 0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon RX 550] [1002:699f] (rev c7)
#IOMMU Group 13 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:aae0]
#IOMMU Group 11 05:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:1343]
#IOMMU Group 11 07:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller [1912:0015] (rev 02)
#IOMMU Group 11 09:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)
#IOMMU Group 7 11:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] USB3 Host Controller [1022:145c]

DMESG Errors

[   46.181767] xhci_hcd 0000:09:00.0: can't setup: -110
[   46.181840] xhci_hcd 0000:09:00.0: USB bus 5 deregistered
[   46.181894] xhci_hcd 0000:09:00.0: init 0000:09:00.0 fail, -110
[   46.181977] xhci_hcd: probe of 0000:09:00.0 failed with error -110
[   61.293790] xhci_hcd 0000:09:00.0: Resetting
[   62.317380] xhci_hcd 0000:09:00.0: xHCI Host Controller
[   62.317388] xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 5
[   93.488974] xhci_hcd 0000:09:00.0: can't setup: -110
[   93.489014] xhci_hcd 0000:09:00.0: USB bus 5 deregistered
[   93.489070] xhci_hcd 0000:09:00.0: init 0000:09:00.0 fail, -110
[   93.489117] xhci_hcd: probe of 0000:09:00.0 failed with error -110

I have already tried returning the cards, and replacing them. Removing all pci devices except for one card.
With and without molex power. Swapping PCI Slots, etc. Under the flash firmware, Windows is giving a Code10
on the devices. Which is very common. No amount of tweaks would allow the devices to properly register in
Windows.

As to the ASMEDIA controllers, and other on the board. My Host is running off of a USB flash drive. The usb hub it is
attached to is not in the above list. Trying to pass those USB controllers resulted in the group already in use error.
I also have the ACS and NPT patch installed. Tweaking from multifunction, to downstream made no change.

2

If that’s the case, you’re out of luck. Unless you can play with the slots the cards are in, or there’s some odd BIOS setting, you’re never going to get those cards in a separate IOMMU group.

This is why I don’t pass through USB controllers. I just pass individual devices. :confused:

3

Well I can pass through the cards. The problem is the cards don’t work. It seems to be from various threads I’ve read fairly common too.

I even tried them in an older mobo with 2nd gen i5. The cards still wouldn’t load properly. I’ve returned them, and tried ordering another card. But they had the same chipset which seems to be a problem.

So to expand on this. The dmesg errors are from the host. Running Windows 10 directly on the host results in a Code 10 couldn’t start cards. The answer seems to be flashing the firmware. But Windows 10 needs to be able to load the cards first. Shorter version what USB3 cards would people recommend?

4

So you have tried different slot configurations. It doesn’t work in one of the top x8 slots? I wasn’t able to get my Renesas card to initialize properly (Neither Linux host or Windows VM) in any of the chipset provided slots with the exact same dmesg errors. But it works fine in the middle x8 slot which is wired directly to the CPU.

Yes, it meant that I had to move the host GPU into the bottom x4 slot.
This is not applicable for your particular workload with that many VMs. Threadripper would be better suited for this…
USB devices can be attached to running VMs, so bash script magic might be a workaround, maybe USB over IP (host-only network) is worth a look.