Threadripper and SR-IOV

I know this thread is old, but I guess I can add some info. The SR-IOV is a technology aimed to be universal for most new PCIe devices.

In my understanding, IOMMU alone is not enough to take advantage of SR-IOV capable hardware, it requires at least 2 underlying technologies aside from Intel VT-x and VT-d or similar technologies from AMD. I believe IOMMU is the “Memory Address Translation” which is one, and two is the “ACS or Access Control Services”. The SR-IOV capable hardware has “PF or Physical Function” and “VFs or Virtual Functions” which are exposed by the “Access Control Services”. The “Memory Address Translation or IOMMU” is responsible for routing the “Virtual Functions” to the Virtual Machines thru the Memory Address Space.

So you need a motherboard that supports both “Memory Address Translation or IOMMU” and “Access Control Services(ACS)”.

Correct me if I am wrong about IOMMU and Memory Address Translation.

And if you want to DDA (discrete device assignment) a GPU, I believe you still need those supporting features.

Those features are considered as “enterprise”. But you can request gigabyte for a BIOS update. In my experience, on ASUS, their technical support and Engineering team are willing to talk about SR-IOV and create a BIOS update with ACS enabler for the motherboard chipset, since the chipset specs support SR-IOV.

1 Like

SR-IOV works for me for MSI Creator TRX40, an X540-T2 and the 3970x, on bog-standard BIOS with KVM, with a special workaround: add “pci=assign-busses pci=realloc” to the kernel command line.

No idea if any workaround is available for Windows or ESX.

MSI said they would build a SR-IOV enabled BIOS for me (not received it yet) but this will need to be repeated every time they release a new BIOS.

2 Likes

That’s pretty fascinating that MSI would build a special bios for you to enable SR-IOV. I wonder why they simply don’t integrate it into the standard BIOS build!?

Also, how is the IOMMU group separation on the MSI Creator TRX40 with the latest BIOS?
Would you run the following and share for the world?

sudo lspci -vvv
lsusb
lsusb -t
dmesg

and the script to check IOMMU group separation, either from this post on the L1 forums or on the arch wiki.

1 Like

Hello, I have just purchased a asrock wrx80 creator motherboard and I am not able to install XCP-ng due to iommu issues. I am not seeing any of these settings within the bios to change to the recommended x2apic setting to fix this issue. Does anyone have any information about this?