Anyone looked at TRX40 IOMMU Groups and Passthrough yet?

Not having the board you asking, but I can answer two question2 s:

2 Not tested but for my board it is in separate IOMMU group. Maybe it needs FLR patch, but only to find out is to try it out.

4 Yes FLR patch does work. It even work for the Audio Controller, which are disguised as starship USB controllers, due to how audio is configured on TRX 40.

I watch tech news every day and I haven’t seen anything like that. Zen3 is not a completely new architecture that requires a totally different socket from everything I’ve seen. In fact it’s supposed to be the last release for the AM4 socket, and I’d bet, without seeing anything to the contrary, that the Zen3 Threadripper will ALSO be compatible with TRX40.

What AMD has said though is that Zen3 could be considered something like a new architecture because of improvements they’ve made. But, Zen3 comes out at the end of 2020, and for Ryzen it will be the last one on the AM4 socket. Zen4 will have PCIe5 and DDR5 and will require new sockets of course. Since I don’t know for a FACT information about Threadripper and TRX40, I can’t say anything definitive, but if there has been any definitive news released by AMD, could you provide a link to it? It’s contrary to what I understand. I find it hard to believe that with the expense that went into making these new sockets and the cost of these boards, which are a bit better than their older gen boards, that they would make it for only one gen with the changes I’ve seen so far on tech news, which from everything I’ve seen doesn’t require new MBs, since both X570 and TRX40 both have an improved architecture to deal with these faster CPUs. I’m pretty sure there will still be a MB that comes out with Zen3, but that doesn’t mean the X570 and TRX40 won’t be capable of running Zen3 CPUs.

There are some boards that have 5 PCIe slots, with one being an X1. But, wouldn’t a USB card be X1, or at least allow running at X1?

Also, some boards have 10Gbps LAN, so that would take care of the last bit.

From what I see the boards do X16 to 2 slots and X8 to two others, didn’t bother looking to see what goes through the chipset and which are direct to the CPU. I think though that the chipset to CPU link is PCIe4 X8, so it give plenty of bandwidth for whatever does go through the chipset.

Maybe one day instead of needing a USB card, there will be breakout boxes that you can connect to one of the ports in the back and gives you multiple ports. That would eliminate the need for a USB card. :slight_smile:

Isn’t that SAS controller PCIe2.0? When I look it up it’s PCIe2.0 X8, which means 4GB/s max. That’s like one high quality NVMe drive. I do understand though that’s plenty of bandwidth for dealing with RAIDed mechanical drives. The problem though is if your system is doing a few things at once, the transmission time (meaning a single data transmission) is slower than a gen3 device and substantially slower than a gen4 device. At some point that slower transmission time, if you’re doing a lot of data transfers with that SAS will create latency for the system, so at some point you probably want to bump that up to PCIe3. But, this is workload dependent and only you know exactly how your system is being used.

Could you explain more about the audio on TRX40? Which TRX40 board you have?

To quote anandtech:

It turns out that AMD did not include any onboard passthough audio in the chipet. That means that all the motherboard vendors that are using a Realtek HD audio codec for the rear audio have had to include the ALC4050H audio codec to bridge the USB to the I2S. The Realtek ALC4050H is used to integrate a bridge between the chipset and USB 2.0 to give the Realtek ALC1220 all the necessary connectivity to the onboard audio. Most of the boards in the stack use the ALC4050H codec to power the 3.5 mm microphone input on the rear panel, as well as the front panel audio as not to waste its capability.

As for TRX40 board. I use Gigabyte Aorus Xtreme.

Basically, even if the audio is a physical PCIe device, the interface is on the USB bus. Specifically it’s on the 47.00.3 Controller (Which is also the front USB-C ports) on the Zenith II. I am working my way through checking what each port is connected to, and will posts a list when I have it completed.

@DJ_Datte Well that is exactly what I am thinking too, except that 47.00.3 is just an USB controller, to which the board manufacturer attached an USB sound device (actually two of them).
I asked about this since I have seen some misunderstandings about this, e.g someone said something about the SPDF and the HDA controller, also I see that the mobo has HDA disable/enable option, which seems pointless since HDA controller is not connected to anything.

TL;DR:

On all the motherboards, except the TRX40 line, we always had HDA controller in the CPU or chipset, and it was connected using a special bus (HDA bus) to a sound device (a codec) which provided all the sound output (back and front ports)

On TRX40, we still have that HDA controller, but it appears not be connected to anything. Still strange that bios has HDA enable/disable option.

Instead, we have two USB sound devices (one for back ports and one for front ports), plugged in the chipset usb controller, basically in the same way you can plug such device to a back port.

It is to some sense better this way since the user can do USB pass-though of both USB devices, even to different VMs. (this is not the same as pci passthrough).

However if you pass the whole usb controller (that 47.00.3), the both usb sound devices will be passed with it, whether you like that or not.

Excellent explanation!

On my board those usb controllers were together in one IOMMU group with one more device.
Luckily I was able to pass through all 3 of them with no issue.
So I wonder if its a recurring pattern on all TRX40 boards.

2 Likes

Here is the Zenith II USB Topology:

Keep in mind the addresses can move slightly, but the ordering will stay. All of the ports can be passed to VMs, but this was tested with ACS and FLR patches (with FLR being specifically activated for the StarShip controllers, so your milage may vary).

The port numbers (Port3-12) relate to the downloadable M/B Manual.

Not figured out:
MidBoard Internal USB3 M/B Connector (I can’t fit a cable there atm, getting an angled connector soonTM).

03:00.3 - StarShip
Back Red USB Port - Port11 (2 Ports)

22:00.0 - Asmedia Controller
Back Bottom USB-C Connector - Port13

24:00.3 - StarShip
Back Red USB Port - Port 8 (2 Ports)

47:00.1 - Matisse
Back Red USB Port - Port 9 (1 Port)
Back Red USB Port - Port 12 (1 Port)
Back Top USB-C Connector - Port 10 (1 Port)
Internal USB 2.0 Connectors
Intel BlueTooth Controller
Aura Led Controller Wired Here

47:00.3 - Matisse
Back Blue Ports Port 3 & Port 5 (4 Ports)
Bottom Internal USB3 M/B Connector
Internal USB-C Connectors (x2 Connectors)
Both Internal RealTek Sound Cards Wired Here USB 2.0

This took me months…

I’m glad to see the convo’s in this thread happening give me hope that when I take the plunge and go Threadripper 3 I can really make this system what it deserves to be.

4 Likes

Awesome setup! Remember to treat your seating as well as any other interface, considering your bottom will touch the chair for a long time. It doesnt have to be fancy like a gaming chair, it just has to be soft and comfy.

+1 on the Lenovo Wireless Keyboard + track point: - I can also see you are a man of efficiency. :+1:

1 Like

lol the “DEF NOT Koolaid!” label made me laugh haha

Man that’s a nice set up. What OS and configuration do you have on there?

Interested to know the providence of your chassis/test bench/trolley as I’ve been considering making something similar but I’d rather not have to make it from scratch.

I love the concept. Don’t like color mix, but it’s really a cool setup. I’d have to get a divorce to be able to try something similar! :slight_smile:

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.