I’ve been benchmarking USB4/TB networking on a pair of Framework Desktop Strix Halo machines. Focused initally on bandwith, but more recently on latency.
Anyway, the usable bandwidth per link seems rather disappointing relative to the published 40Gbps specs. On Strix Halo specifically, I’ve been unable to get much over 10-11Gbps per link, per direction (Tx or Rx). I’ll get into the settings and tests in a followup-post, and I sure hope someone can point out a solution - but for nowit seems closer to a 10Gbps link than a 40Gbps one.
The latency side, however, seems quite a bit more promising. I had heard that USB4/TB networking was far inferior to Ethernet due to latency, but I’m measuring it as slightly faster than a direct P2P ethernet link, and a lot faster than connecting over Ethernet going via a shared switch (let alone if it goes >1 network hops).
Anyway, all of this only applies to Strix Halo. And maybe also varies based on OS (I used Fedora) and even kernel (I used 6.17 for these tests). But figured it might be interesting to folk.
import socket
import time
import sys
PORT = 5005
ITERATIONS = 1000
def run_server():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# 50 matches your sysctl net.core.busy_read
try:
s.setsockopt(socket.SOL_SOCKET, socket.SO_BUSY_POLL, 50)
except Exception: pass
s.bind(('0.0.0.0', PORT))
s.listen(1)
print(f"Server listening on {PORT}...")
conn, addr = s.accept()
with conn:
for _ in range(ITERATIONS):
data = conn.recv(1)
if not data: break
conn.sendall(data)
def run_client(server_ip):
latencies = []
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
# TCP_NODELAY is vital to avoid buffering delays
s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
try:
s.setsockopt(socket.SOL_SOCKET, socket.SO_BUSY_POLL, 50)
except Exception: pass
s.connect((server_ip, PORT))
for i in range(ITERATIONS):
start = time.perf_counter_ns()
s.sendall(b'x')
s.recv(1)
end = time.perf_counter_ns()
latencies.append((end - start) / 1000) # Convert to microseconds
avg = sum(latencies) / len(latencies)
print(f"\nResults for {server_ip}:")
print(f" Avg Latency: {avg:.2f} µs")
print(f" Min Latency: {min(latencies):.2f} µs")
print(f" 99th Percentile: {sorted(latencies)[int(ITERATIONS*0.99)]:.2f} µs")
if __name__ == "__main__":
if len(sys.argv) > 1:
run_client(sys.argv[1])
else:
run_server()
Test scripts for throughput
This side is a bit more involved, I’ll share in a bit. The results aren’t that interesting - bigger pipe == better - the only really surprising thing was that AFAICT, Strix Halo USB4 seems unable to get over ~11Gbps in Tx or Rx.
I am expanding my test matric with 10GbE and 25GbE, just need to finish the testing.
From observations of other systems and thunderbolt networking, try turning the MTU up to stupid levels, try 65536. There were some other changes needed too… Thunderbolt isn’t the best for TCP. Which is a bit strange considering.
Thanks for the reminder, yeah I did test MTUs of 1500 (default), 9000 (most common recommendation) and 65522 (max supported value). For each setting I validated the MTU had taken effect on both ends with a ping -s <<MTU-28>> -M do <<remote-IP>.
The larger MTU did give a consistent but very small increase in throughput (~1.4%). Ran warmup + 3 tests at each MTU, using Claude to collate and summarize:
MTU
Aggregate (2 links)
Per-Link Bidir
Per Direction
1500
45.45 Gbps
22.72 Gbps
~11.36 Gbps
9000
45.77 Gbps
22.89 Gbps
~11.44 Gbps
65520
46.09 Gbps
23.05 Gbps
~11.52 Gbps
I realize that I neglected to mention the other tuning parameters in place for the tests apart from the kernel parameters mentioned above. I also have these settings active in all tests to date:
I really wish I remembered where I read on this recently. I think it was something about a CEO mini homelab or mini rack? There were three or four MS-01s and a mountain of tweaking to the thunderbolt mesh. I will say that if I’m remembering correctly he was getting 15gbit/link.
I am in the process of developing my own rdma driver for the Strix Halo - (GMKtec EVO-X2 128Gig) and I started out at about 7.5Gig transfer from one box to the next (I have 4 of them) using just the off the shelf software. I was planning on running them in a ring configuration.
I eventually got all the way down to the tunderbolt_net driver and came to the conclusion that it was the bottle neck. It looks like it only supported 4K frames. I hacked it up, tossed all the IP stuff, and started to make the frames bigger, then went to near zero copy.
It didnt take me long and I got to 41Gig? I couldn’t believe it? I was thinking maybe I was running in a 3 X 1 fat mode as I was only sending packets in one direction. I added a loop back mode so that the packets would come right back, was expecting the 1 lane to bottle neck everything it I was running in a 3 X 1. It didn’t. I got a full 41 Gig round trip.
I still didn’t believe it.
Long story short I now have a driver that is achieving 78Gb/s with a latency around 1.5uSec.
This PHY is USB4 V2 capable (I got to think it didn’t make specs somewhere).
Turns out that the firmware is reporting the wrong capabilities. If you ignore it and just let the phy go full tilt (at least to another Stix Halo box) it works great at 80 gig (or 40 gig a lane).
I am now at a point where I am trying to find somebody out there that is capable of confirming it or telling me where I am wrong?
If it is of any use, I am happy to help test this out in a couple configurations. I’m using an Infiniband link between my Strix Halos currently but have compared a pretty wide range of P2P connections including some not mentioned here including some more oddball ones like PCIe tunneling over USB4 and WiFiDirect. IB wins but its a giant PITA to get working especially on my Framework Desktops (their BIOS is flaky specifically w. Mellanox cards).
Your 78Gbps seems high given what PCIe 4.0x4 can do, and the USB4 ports only get 4 PCIe lanes each. Are you bonding the 2x USB4 ports, or is this counting traffic in both directions? 78Gbps in one direction would be stellar, that exceeds what 100Gbps IB/RoCEv2 can do, given the Gen4x4 limit.
Hey just wanted to drop a quick note. I’m on vacation right now but when I get back I’ll put a package together for you.
I am running chachyOS and have it setup to stay off of one of the processors so I can use the Rama driver to that one processor with no Linux in the way.
Let me know if you need os settings and such.
I don’t have the numbers off the top of my head but I am at around 50gb/s hooking into the rocm and am writing an IP layer so that I can load the silly deepseek model. Currently it take like 17min across the 2.5 gig Ethernet and I am hoping to get down to a couple min.
When you say makes sense, what do you mean? v1 is two lanes of 20gig in each direction? How do you get to 41 gig round trip (and that includes overhead)?
Even if you assume fat mode (3 lanes of 20gig) you would only get 1 lane back and should only get 20gig round trip?
You mean you are actually claiming USB4 v2 support in AMD’s USB4 implementation? Intel’s discrete chip would certainly be bottle-necked at 64 gbps (before even subtracting encoding overhead).
Gen4 is 16 Gb/s per Lane and Gen5 is 32 Gb/s per lane and direction.
The USBv2 implementation of Minisforum MS-S1 (AMD Strix Halo) and MS-02 (Intel 285HX) is using Intel JHL9480 TB5 Controller (Barlow Ridge). As it is connected to PCIe Gen4 x4 it is limited to 64 Gb/s in each direction concurrently. As this is a limitation of the controller 80 Gb/s will never be reached in real world TB5/USB4v2 connections. You can this also see when testing transfer bandwidth to and from TB5 eGPU docks.
After loading this version (modprobe thunderbolt throttle=0), ping -f reports:
93590 packets transmitted, 93588 received, 0.00213698% packet loss, time 2283ms
rtt min/avg/max/mdev = 0.018/0.019/0.684/0.003 ms, ipg/ewma 0.024/0.020 ms
That’s <20µs round trip time, still using the very inefficient thunderbolt_net (which copies all data, calculates checksums and fragments in software). RXE (software RoCE) runs on this.
I’ve found my 7840u (I only have one 395, a Bosgame M5, which keeps me from testing for that 80Gbps) had one core saturated during bandwidth tests with irq throttling disabled. Still, setting it to something moderate like 64 does pretty well, compared to the stock 500. Also, that 0.7ms wakeup time can be reduced to about 0.2ms with /sys/devices/system/cpu/cpu*/power/pm_qos_resume_latency_us (should have the same effect as processor.max_cstate=1 mentioned above). Clearly more latency sources are around, but making slight progress on variables to tweak.
My tests as yet have stayed around 10-11Gbps, which is below the 20Gbps the firmware reports and the 40Gbps it should be capable of. I haven’t yet deepened the queues (thunderbolt_net hardcodes 256 frame deep rings), tried multiple rings, removed copying, etc. But I have confirmed the IP layers are costly.
Looks useful, but at first glance like it doesn’t strip out the copy, holding device dedicated kernel side rings of DMA buffers. It would be possible to improve for e.g. io_uring_register_buffers, sendfile or p2pdma if you plan your frame capacity, i.e. page sized datagrams, not streams. Perhaps the best match would be io_uring_prep_read_multishot with 4KiB buffers.
PS. Thunderbolt has a mode called raw mode, which has less hardware assistance. For each transfer descriptor, frame mode supports a 4KiB frame in 16 packets, while raw mode supports 256 bytes of payload in one transport layer packet. It’s a bit like control or interrupt vs bulk endpoints.
