Hi there, I have an X99 board (Asus X99-DELUXE II) and an Intel X520-DA1 add-in card. When I install the card in any CPU-attached slot, I can create VFs, pass them through to VMs, etc. If I move the card to a PCH-attached slot, i get the following error:
Jan 17 14:04:48 yoshi kernel: ixgbe 0000:72:00.0: can't enable 4 VFs (bus 73 out of range of [bus 72])
Jan 17 14:04:48 yoshi kernel: ixgbe 0000:72:00.0: Failed to enable PCI sriov: -12
I did some googling and found out that some pci= kernel command-line options can help in this case. Specifically, I tried pci=assign-busses,realloc. That seemed to let me create VFs and pass them through, but I didn’t actually check to see if they actually work. Because, unfortunately, these options break my ThunderboltEX 3 add-in card (and possibly some other things):
Jan 17 14:15:05 yoshi kernel: vfio-pci 0000:37:00.0: Unable to change power state from D3hot to D0, device inaccessible
Jan 17 14:15:05 yoshi kernel: vfio-pci 0000:37:00.0: Unable to change power state from D3cold to D0, device inaccessible
Jan 17 14:15:05 yoshi kernel: pci 0000:37:00.0: Removing from iommu group 51
Jan 17 14:15:05 yoshi kernel: pci_bus 0000:6b: busn_res: [bus 6b] end is updated to 6b
Jan 17 14:15:05 yoshi kernel: pcieport 0000:01:00.0: devices behind bridge are unusable because [bus 6b] cannot be assigned for them
Jan 17 14:15:05 yoshi kernel: pci_bus 0000:02: Allocating resources
Jan 17 14:15:05 yoshi kernel: pcieport 0000:02:02.0: Unable to change power state from D0 to D3hot, device inaccessible
Jan 17 14:17:09 yoshi kernel: pcieport 0000:02:02.0: Unable to change power state from D3cold to D0, device inaccessible
Any suggestions? Thank you in advance.