I’m trying to pass through one of the USB controllers on my motherboard to some VMs I’m going to create. Well, I haven’t even gotten that far as when I try to add the vfio-pci.ids to the kernel options, the operating system fails to boot normally.
For reference, I’m running unRAID 6.7.2.
My hardware is as follows
AMD Ryzen Threadripper 2970WX
Gigabyte X399 Aorus Xtreme
64GB Corsair Dominator Platinum RGB
EVGA GeForce 2080Ti
AMD Radeon RX580
When I add vfio-pci.ids=1022:145a,1022:1456,1022:145f to my syslinux configuration, unRAID boots but not normally. Nothing seems to get initialized properly. No web gui, no networking, and I can log in as root with no password.
I can upload the syslogs if that would help.
Whats interesting is that those ids show up in two different IOMMU groups (groups 4 and 21), which is confusing. I dont know if whats showing up in separate groups is different physical hardware with the same ids, or if the same physical USB controller is showing up in two different IOMMU groups.
I’m guessing its probably best to get a PCIe USB card to pass through to VMs, but on an older x79 Intel motherboard I was able to pass through an on board USB controller no problem.
The different IOMMU groups have different controller with the same ID.
It should be possible to pass the USB from the CPU SoC, as it is working for TR 1920X. Just check which controller is for which USB ports. According to board manual, 2 of the 1022:145f USB controllers are responsible for 4 blue ports each. Those are the 8 blue ports on the backpanel. The red one, type C and internal headers are from the chipset.
You dont need to bind 1022:145a even if it is in the same group.
And your problem could be related to binding all of the 1022:1456 Encryption controllers.
Try to boot with only vfio-pci.ids=1022:145f to see if you still have the problem.
Updating BIOS could help splitting the USB controller to its own group. IIRC agesa 1.1.0.2 did that for me on 1920X and Asrock x399 taichi.
Thank you for the info! May I ask where you are finding the 1022:145f USB controller info in the manual? I was looking through the manual here but didn’t see anything relevant. So those 2 controllers go through the CPU and the other ones go through the chipset?
Your information was very helpful. I kind of had an “a ha” moment and simultaneously facepalmed for being such an idiot. The other 1022:145f USB controller (the one I’m not trying to pass through) is the top 4 USB ports on my motherboard, one USB port of which has the USB drive with my unRAID install. No wonder unRAID was failing to boot properly. I guess I just wasn’t putting two and two together last night.
Part of the reason I was confused was I was watching some SpaceInvaderOne videos on YouTube about IOMMU groups and USB/GPU passthrough where he said something about having “duplicate” USB controllers show up in different groups and he used the vfio-pci.ids= flag to pass through the controller just fine. Well, he must not have used the ports on those controllers with his USB boot drive. I figured it would work for me, but alas it did not.
I ended up working around this last night. I came across some information about a method called xen-pciback.hide= which can be used when there are multiple of the same vendor:deviceid. However, this was changed in unRAID 6.7.x according to the changelog wherein the correct way to accomplish binding devices via this method is to put them in the config/vfio-pci.cfg file with the format BIND=<device> <device> ie BIND=02:00.0 02:00.1 but if you bind just one of the devices in an IOMMU group it will bind them all.
After I did that for the right USB controller (BIND=0f:00.3), I was able to pass through the controller just fine.
Information regarding the USB is from product specification page (10,11).
I know that the CPU integrated USB controller can drive 4 ports and the USB section shows 8 USB ports from CPU on the back panel. X399 was build for TR with 2 Zen1 chips so it has RAM, PCIe, USB and SATA pins for only 2 of those.