Mac Pro 2013 (MacPro6,1) GPU + Thunderbolt passthrough with display

YamiYukiSenpai · June 19, 2024, 3:51am

I have a 2013 Mac Pro I recently acquired for free along with a Thunderbolt 2 dock, and would like to passthrough one of the Mac Pro’s GPUs and a Belkin Thunderbolt 2 dock so that the dock can be used for display output. Is that possible?

MacPro 6,1 IOMMU groups:

IOMMU Group 0:
        00:00.0 Host bridge [0600]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 DMI2 [8086:0e00] (rev 04)
IOMMU Group 1:
        00:01.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 1a [8086:0e02] (rev 04)
IOMMU Group 2:
        00:02.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 2a [8086:0e04] (rev 04)
IOMMU Group 3:
        00:03.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 3a [8086:0e08] (rev 04)
IOMMU Group 4:
        00:04.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 0 [8086:0e20] (rev 04)
IOMMU Group 5:
        00:04.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 1 [8086:0e21] (rev 04)
IOMMU Group 6:
        00:04.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 2 [8086:0e22] (rev 04)
IOMMU Group 7:
        00:04.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 3 [8086:0e23] (rev 04)
IOMMU Group 8:
        00:04.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 4 [8086:0e24] (rev 04)
IOMMU Group 9:
        00:04.5 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 5 [8086:0e25] (rev 04)
IOMMU Group 10:
        00:04.6 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 6 [8086:0e26] (rev 04)
IOMMU Group 11:
        00:04.7 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 7 [8086:0e27] (rev 04)
IOMMU Group 12:
        00:05.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 VTd/Memory Map/Misc [8086:0e28] (rev 04)
IOMMU Group 13:
        00:05.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Memory Hotplug [8086:0e29] (rev 04)
IOMMU Group 14:
        00:05.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IIO RAS [8086:0e2a] (rev 04)
IOMMU Group 15:
        00:05.4 PIC [0800]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IOAPIC [8086:0e2c] (rev 04)
IOMMU Group 16:
        00:11.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Virtual Root Port [8086:1d3e] (rev 06)
IOMMU Group 17:
        00:16.0 Communication controller [0780]: Intel Corporation C600/X79 series chipset MEI Controller #1 [8086:1d3a] (rev 05)
IOMMU Group 18:
        00:1b.0 Audio device [0403]: Intel Corporation C600/X79 series chipset High Definition Audio Controller [8086:1d20] (rev 06)
IOMMU Group 19:
        00:1c.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 [8086:1d10] (rev b6)
IOMMU Group 20:
        00:1c.1 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 2 [8086:1d12] (rev b6)
IOMMU Group 21:
        00:1c.2 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 3 [8086:1d14] (rev b6)
IOMMU Group 22:
        00:1c.4 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 5 [8086:1d18] (rev b6)
IOMMU Group 23:
        00:1d.0 USB controller [0c03]: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 [8086:1d26] (rev 06)
IOMMU Group 24:
        00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev a6)
IOMMU Group 25:
        00:1f.0 ISA bridge [0601]: Intel Corporation C600/X79 series chipset LPC Controller [8086:1d41] (rev 06)
        00:1f.3 SMBus [0c05]: Intel Corporation C600/X79 series chipset SMBus Host Controller [8086:1d22] (rev 06)
IOMMU Group 26:
        10:00.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 27:
        11:01.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 28:
        11:02.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 29:
        11:08.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 30:
        11:09.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 31:
        11:0a.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8723] (rev ca)
IOMMU Group 32:
        12:00.0 USB controller [0c03]: Fresco Logic FL1100 USB 3.0 Host Controller [1b73:1100] (rev 10)
IOMMU Group 33:
        14:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 34:
        15:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        16:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]
IOMMU Group 35:
        15:03.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 36:
        15:04.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 37:
        15:05.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        39:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        3a:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        3a:01.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        3a:04.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        3a:05.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        3b:00.0 USB controller [0c03]: Fresco Logic FL1100 USB 3.0 Host Controller [1b73:1100] (rev 10)
        3c:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
IOMMU Group 38:
        15:06.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 39:
        5b:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 40:
        5c:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        5d:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]
IOMMU Group 41:
        5c:03.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 42:
        5c:04.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 43:
        5c:05.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 44:
        5c:06.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 45:
        a2:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 46:
        a3:00.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
        a4:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]
IOMMU Group 47:
        a3:03.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 48:
        a3:04.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 49:
        a3:05.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 50:
        a3:06.0 PCI bridge [0604]: Intel Corporation DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [8086:156d]
IOMMU Group 51:
        02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT / Trinidad XT [Radeon R7 370 / R9 270X/370X] [1002:6810]
        02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Oland/Hainan/Cape Verde/Pitcairn HDMI Audio [Radeon HD 7000 Series] [1002:aab0]
IOMMU Group 52:
        06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT / Trinidad XT [Radeon R7 370 / R9 270X/370X] [1002:6810]
        06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Oland/Hainan/Cape Verde/Pitcairn HDMI Audio [Radeon HD 7000 Series] [1002:aab0]
IOMMU Group 53:
        0b:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme BCM57762 Gigabit Ethernet PCIe [14e4:1682]
IOMMU Group 54:
        0c:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries NetXtreme BCM57762 Gigabit Ethernet PCIe [14e4:1682]
IOMMU Group 55:
        0d:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BCM4360 802.11ac Dual Band Wireless Network Adapter [14e4:43a0] (rev 03)
IOMMU Group 56:
        0e:00.0 Non-Volatile memory controller [0108]: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1608 (DRAM-less) [1e4b:1608] (rev 01)
IOMMU Group 57:
        ff:08.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 0 [8086:0e80] (rev 04)
IOMMU Group 58:
        ff:08.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link Reut 0 [8086:0e83] (rev 04)
IOMMU Group 59:
        ff:08.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link Reut 0 [8086:0e84] (rev 04)
IOMMU Group 60:
        ff:09.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 1 [8086:0e90] (rev 04)
IOMMU Group 61:
        ff:09.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 1 [8086:0e93] (rev 04)
IOMMU Group 62:
        ff:09.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link Reut 1 [8086:0e94] (rev 04)
IOMMU Group 63:
        ff:0a.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 0 [8086:0ec0] (rev 04)
        ff:0a.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 1 [8086:0ec1] (rev 04)
        ff:0a.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 2 [8086:0ec2] (rev 04)
        ff:0a.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 3 [8086:0ec3] (rev 04)
IOMMU Group 64:
        ff:0b.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers [8086:0e1e] (rev 04)
        ff:0b.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers [8086:0e1f] (rev 04)
IOMMU Group 65:
        ff:0c.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee0] (rev 04)
        ff:0c.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee2] (rev 04)
        ff:0c.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee4] (rev 04)
        ff:0c.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee6] (rev 04)
        ff:0c.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee8] (rev 04)
IOMMU Group 66:
        ff:0d.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee1] (rev 04)
        ff:0d.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee3] (rev 04)
        ff:0d.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee5] (rev 04)
        ff:0d.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee7] (rev 04)
        ff:0d.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee9] (rev 04)
IOMMU Group 67:
        ff:0e.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 [8086:0ea0] (rev 04)
        ff:0e.1 Performance counters [1101]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 [8086:0e30] (rev 04)
IOMMU Group 68:
        ff:0f.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Target Address/Thermal Registers [8086:0ea8] (rev 04)
IOMMU Group 69:
        ff:0f.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 RAS Registers [8086:0e71] (rev 04)
IOMMU Group 70:
        ff:0f.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers [8086:0eaa] (rev 04)
IOMMU Group 71:
        ff:0f.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers [8086:0eab] (rev 04)
IOMMU Group 72:
        ff:0f.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers [8086:0eac] (rev 04)
IOMMU Group 73:
        ff:0f.5 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers [8086:0ead] (rev 04)
IOMMU Group 74:
        ff:0f.6 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 DDRIO Registers [8086:0eae] (rev 04)
IOMMU Group 75:
        ff:10.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 0 [8086:0eb0] (rev 04)
IOMMU Group 76:
        ff:10.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 1 [8086:0eb1] (rev 04)
IOMMU Group 77:
        ff:10.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 0 [8086:0eb2] (rev 04)
IOMMU Group 78:
        ff:10.3 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 1 [8086:0eb3] (rev 04)
IOMMU Group 79:
        ff:10.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 2 [8086:0eb4] (rev 04)
IOMMU Group 80:
        ff:10.5 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 3 [8086:0eb5] (rev 04)
IOMMU Group 81:
        ff:10.6 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 2 [8086:0eb6] (rev 04)
IOMMU Group 82:
        ff:10.7 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 3 [8086:0eb7] (rev 04)
IOMMU Group 83:
        ff:11.0 System peripheral [0880]: Intel Corporation Device [8086:0eb8] (rev 04)
IOMMU Group 84:
        ff:13.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe [8086:0e1d] (rev 04)
        ff:13.1 Performance counters [1101]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe [8086:0e34] (rev 04)
        ff:13.4 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Registers [8086:0e81] (rev 04)
        ff:13.5 Performance counters [1101]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring [8086:0e36] (rev 04)
        ff:13.6 Performance counters [1101]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring [8086:0e37] (rev 04)
IOMMU Group 85:
        ff:16.0 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 System Address Decoder [8086:0ec8] (rev 04)
        ff:16.1 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers [8086:0ec9] (rev 04)
        ff:16.2 System peripheral [0880]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers [8086:0eca] (rev 04)

Domrockt · June 19, 2024, 8:42am

Hello, could you tell which OS you are using to virtualize?

But in Short, yes you can passtrough the thunderbold Controller to an VM and that will work as usual. The same goes with one of the gpu.

YamiYukiSenpai · June 19, 2024, 10:37am

Just bog-standard Ubuntu Server

I managed to successfully passthrough 1 of them (other 2 errored out). Any predictable way of knowing which Thunderbolt port was passed through?

Domrockt · June 19, 2024, 7:40pm

1 of 2 means what errored out? the Thunderbolt controller or the GPU?

YamiYukiSenpai · June 20, 2024, 12:33am

Current VM config:

<domain type="kvm">
  <name>raven-neon</name>
  <uuid>2e5236ca-9f45-4d8c-bf2b-d626e5454591</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/22.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">16777216</memory>
  <currentMemory unit="KiB">16777216</currentMemory>
  <memoryBacking>
    <source type="memfd"/>
    <access mode="shared"/>
  </memoryBacking>
  <vcpu placement="static">8</vcpu>
  <os firmware="efi">
    <type arch="x86_64" machine="pc-q35-8.2">hvm</type>
    <firmware>
      <feature enabled="yes" name="enrolled-keys"/>
      <feature enabled="yes" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" secure="yes" type="pflash">/usr/share/OVMF/OVMF_CODE_4M.ms.fd</loader>
    <nvram template="/usr/share/OVMF/OVMF_VARS_4M.ms.fd">/var/lib/libvirt/qemu/nvram/raven-neon_VARS.fd</nvram>
    <bootmenu enable="yes"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
    <smm state="on"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" cores="8" threads="1"/>
  </cpu>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/media/vm/storage/raven-neon.qcow2"/>
      <target dev="vda" bus="virtio"/>
      <boot order="1"/>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/media/vm/iso/neon-developer-20240617-1605.iso"/>
      <target dev="sda" bus="sata"/>
      <readonly/>
      <boot order="2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x8"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x9"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0xa"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0xb"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0xc"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0xd"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0xe"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0xf"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </controller>
    <interface type="bridge">
      <mac address="52:54:00:74:33:04"/>
      <source bridge="br0"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <interface type="network">
      <mac address="52:54:00:f4:08:ae"/>
      <source network="default"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <tpm model="tpm-crb">
      <backend type="emulator" version="2.0"/>
    </tpm>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="none"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x16" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x0a" slot="0x00" function="0x0"/>
    </hostdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <watchdog model="itco" action="reset"/>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x09" slot="0x00" function="0x0"/>
    </rng>
  </devices>
</domain>

Here are the Thunderbolt “devices” I can pass through:

$ lspci -nnk | grep -i thunderbolt | grep -i system
16:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]
        Subsystem: Belkin DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [1799:0092]
        Subsystem: Belkin DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [1799:0092]
        Subsystem: Belkin DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [1799:0092]
        Subsystem: Belkin DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [1799:0092]
        Subsystem: Belkin DSL5520 Thunderbolt 2 Bridge [Falcon Ridge 4C 2013] [1799:0092]
5d:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]
a4:00.0 System peripheral [0880]: Intel Corporation DSL5520 Thunderbolt 2 NHI [Falcon Ridge 4C 2013] [8086:156c]

$ cat /etc/modprobe.d/vfio.conf 
softdep drm pre: vfio-pci
options vfio-pci ids=1002:6810,1002:aab0,8086:156c
options vfio_iommu_type1 allow_unsafe_interrupts=1

When I pass through 16:00:0, I can boot it up, but I can’t seem to see anything
When I pass through 5D:00:0 or A4:00:0, I get

Error starting domain: internal error: QEMU unexpectedly closed the monitor (vm='raven-neon'): 2024-06-20T00:21:14.958378Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:5d:00.0","id":"hostdev3","bus":"pci.6","addr":"0x0","rombar":0}: vfio 0000:5d:00.0: hardware reports invalid configuration, MSIX PBA outside of specified BAR

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1384, in startup
    self._backend.create()
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1353, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: QEMU unexpectedly closed the monitor (vm='raven-neon'): 2024-06-20T00:21:14.958378Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:5d:00.0","id":"hostdev3","bus":"pci.6","addr":"0x0","rombar":0}: vfio 0000:5d:00.0: hardware reports invalid configuration, MSIX PBA outside of specified BAR

I confirmed as well that when the Mac Pro boots, I see the grub through the Thunderbolt dock

rrubberr · June 23, 2024, 11:43pm

Have you tried other Thunderbolt ports? It best well be that one GPU or the other takes precedent on certain ports (there are 6 of them, if memory serves).

YamiYukiSenpai · June 23, 2024, 11:46pm

Yes, I did. Though I think I found the issue with the GPU part:

Gonna try and see if I can find a ROM for this one, but no luck in TechPowerup.

Any suggestions for the Thunderbolt ICM firmware?

Edit: found firmware for W7000, which is a 4GB variant of D300, and 270X VBIOS, which is explicitly mentioned in lscpi. Getting this with 270X:

And this with W7000:

Display isn’t working yet, though.

FYI, I’m not using an eGPU. I’m just trying to use the FirePro D300 GPU in the Mac Pro

EDIT: about MSIX PBA outside of specified BAR, I found this on Proxmox forum. Just not sure how to apply that on the Mac Pro.