Return to Level1Techs.com

PSA: There are FOUR PCI IDs to passthrough for a Turing GPU

Tried passing through a Turing 1660 Ti today and was able to successfully do so, however, it comes with a note if you’re coming from Pascal:

There are FOUR devices you need to passthrough, and to retain the multifunction trait for the libvirt XML, I had to manually add a few entries. Adding entries using virt-manager is not recommended because it will mess things up if you previously had a multifunction listing in your XML.

Here’s the 4 devices:

05:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU116 [GeForce GTX 1660 Ti] [10de:2182] (rev a1)
05:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aeb] (rev a1)
05:00.2 USB controller [0c03]: NVIDIA Corporation Device [10de:1aec] (rev a1)
05:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device [10de:1aed] (rev a1)

And I had to make corresponding changes to the libvirt XML to accommodate it

From

...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
    </hostdev>
...

to

...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x2'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x3'/>
    </hostdev>
...

Do also note the USB-C controller calls on the XHCI kernel driver so it loads before vfio-pci. You will need to sometimes use pci-stub on XHCI controllers due to the order some distros or kernel ramdisks load modules.

And the 1660 Ti still has the devices for USB-C and a Serial connection, but are hand me downs from the RTX 2060, and it does not mean they work. However, they are all in the same IOMMU group so they need to be passed through together:

Output of my improved LS-IOMMU for the 1660 Ti:

IOMMU Group 22 
Slot:   05:00.0
Class:  VGA compatible controller
Vendor: NVIDIA Corporation
Device: TU116 [GeForce GTX 1660 Ti]
IDs:    10de:2182
Rev:    a1

IOMMU Group 22 
Slot:   05:00.1
Class:  Audio device
Vendor: NVIDIA Corporation
Device: Device 1aeb
IDs:    10de:1aeb
Rev:    a1

IOMMU Group 22 
Slot:   05:00.2
Class:  USB controller
Vendor: NVIDIA Corporation
Device: Device 1aec
IDs:    10de:1aec
Rev:    a1

IOMMU Group 22 
Slot:   05:00.3
Class:  Serial bus controller [0c80]
Vendor: NVIDIA Corporation
Device: Device 1aed
IDs:    10de:1aed
Rev:    a1

That’s basically it. If you’re used to Pascal, just be aware Nvidia has more devices in the same IOMMU group that needs to be passed through.

2 Likes

Hmmm I knew there was a USB controller on some of the turing gpus, but the serial controller is new. Thanks!

4f2xyk

1 Like

Ehhh, it just means a little more configuration shouldn’t be too bad.

Yeah it’s not a big deal, just another ID added to the list. I just wasn’t sure if the star trek reference was intentional or not, so now it is. :smiley:

1 Like

doesn’t watch star trek

ohhhh