I was unable to find a OOB product that suits my requirements:
A small box with a PCIe 3.0 x4 interface to use regular non-GPU PCIe AICs with computers with a Thunderbolt 3 port. The first use would be an Intel XL710 2 x 40 GbE ethernet adapter (natively PCIe 3.0 x8, but am fine. with it being limited through TB3).
Can I DIY-build such a thingy with an external enclosure for M.2 PCIe NVMe SSDs and actively powered M.2-to-PCIe adapters?
I’m asking since I don’t know if there are any limitations to TB3 that the manufacturer can specify so that it can only be used for NVMe devices…?
As I’ve experienced, getting the boot sequence correct on a Thunderbolt device is neigh impossible. This is a bad idea to begin with using Thunderbolt 3. Using U.2 to relocate a AIC is much better solution. Since it’s raw PCI-E though, make sure it’s shielded and short.
The USB 3.0 cable based solutions are all typically x1 which won’t work if bandwidth is a concern.
The clarify for a normie: Why is the boot sequence relevant for an ethernet adapter (not using to boot off of network)?
I wanted to make such a “box” to be able to get stuff faster to/from laptops and a desktop computer that has TB3 but no more PCIe lanes available.
But do I understand you correctly that there shouldn’t be a reason why this shouldn’t work by design?
Why I asked this question:
Experience with a non-TB3 ASUS Pro WS X570-Ace motherboard:
Motherboard has a built-in U.2 port (from X570 chipset) that can be switched between (PCIe, UEFI default) and SATA (option))
PCIe NVMe SSDs work fine on that interface with PCIe 3.0 x4 link
Now I wanted to use this port to attach a regular PCIe AIC with such an adapter
Devices in the adapter don’t show up at all
On the other hand: I can use M.2-to-U.2 adapters and a chain of PCIe AIC-to-M.2 slot, M.2-to-U.2 adapter, U.2-U2 cable and the first adapter mentioned above to operate regular AICs
(with both, PCIe lanes coming from the CPU and from the X570 chipset)
Yeah, it’s quirks like that, which I’m finding out doing Thunderbolt devices, now knowing not everything is equal. U.2 on a motherboard can restrict it to NVMe Protocol only. Same with NVMe enclosures for Thunderbolt 3, in that the controller only accepts NVMe Protocol and not PCI-E.
Best bet is adapters. Take a M.2 Port, turn it into U.2, then turn that U.2 into PCI-E. This is the best bet since there’s no special treatment of those lanes like with onboard motherboard U.2 ports.
You still have the PCI bridges (The Thunderbolt controller itself) and boot initialization issues to deal with there. If you send PCI-E signals down a U.2 cable similar to a riser, this is not an issue.
*Haven’t had time yet to test this AIC at all so I don’t know its potential issues yet. Drivers work fine and Blackmagic’s management software recognizes the AIC without any problems.
Since PCIe 3.0 x4 from Thunderbolt 3 isn’t enough bandwith for four HDMI 2.0b inputs I’m expecting issues when using the AIC with multiple ports set to 2160p. 1080p should be fine.
Configuration:
Ryzen 7 PRO 4750G
2 x 32 GB DDR4-3200 ECC UDIMM
Gigabyte B550 Vision D with UEFI F11p (Virtualization ON, IOMMU ON, Windows’ “special” virtualization features ON, Above 4G decoding ON, CSM disabled, Secure Boot ON)
The motherboard’s integrated Titan Ridge Thunderbolt 3 controller is connected to the B550 chipset
Win 10 Enterprise x64 20H2, booted from an external USB 3.1 (10 Gb/s) SSD, also connected to onboard Thunderbolt controller
Used the PCB of the external M.2 PCIe NVMe enclosure mentioned in my initial question and connected a M.2-to-PCIe slot riser cable with a separate power input.
This opens the door to many fun projects of mine, no need to buy expensive eGPU boxes with low-quality PSUs any more when you just want that damn Thunderbolt-PCIe slot PCB.
