Windows 10/11 and 10/25/100 GbE Networking Performance

High-Performance Computing Workstations & Servers
1

Hello All,

So as the title lists this is about higher than 1 GbE networking on Windows 10/11.

Prefix: Yes, I have 10,25 and 100 GbE on Linux and it works great for that use case (which I prefer) due to a project constraint we need Windows 10/11, not my decision or preference or desire, getting that out of the way.

This is mainly to move Terabytes of data off a workstation to a storage server in a data center (local but in a different building) via 10 GbE using Cisco switches (9K series).

Questions: (relating to Windows 10/11 Only)

  1. Does anyone have a working setup with Windows 10/11 with higher than 1 GbE networking that as close to Linux bandwidth performance numbers?

  2. The closest we’ve gotten is with robocopy as using rsync is terrible on Windows 10/11.

  3. 100 GbE testing even on the same switch as the storage server (also on 100 GbE) only performs as good as 10 GbE.

I am working on getting my numbers in a presentable format. Will be back later today/evening.

Thanks,

2

To not have N edits.
I forgot, system is AMD Threadripper and we tested the same workstation with Debian v12 and it performed well like expected, swapped drives to test Windows 10, so same workstation and all components and BIOS settings. BIOS firmware is updated to the latest, same for drives (NVMe).

3

What motherboard? What switch?

4

From CPUz:

vendor ASUSTeK COMPUTER INC.
model PRIME TRX40-PRO
revision Rev 1.xx
Name AMD Ryzen Threadripper 3960X
Codename Castle Peak
Specification AMD Ryzen Threadripper 3960X 24-Core Processor

Switch is Cisco 9407 Using the 10 GbE Line Card for the 10 GbE test and for the 100 GbE test a Cisco N9K-C93240YC-FX2.

5

I can’t personally help, but that will help others help you better.

6

I’ve noticed that out of the box, Windows network speed is very dependent on the performance of a single thread/core.
Modern high-end consumer desktop Intel and AMD systems can comfortable saturate 25GBe on Windows, but a Zen 2 based threadripper is about half the speed of the top of the line consumer Intel CPUs so I wouldn’t expect it to reach those speeds without lots of network stack tweaking.

2 Likes
7

StorageReview covered Tuxera. I think its probably the best solution for that right now. If you need linerste smb this is probably it. Wish Wendell would review this…

Heres the product,

Heres the StorageReview video,

You need to roll your own Linux storage server, so not sure how it fits in your case, but this is one of the best looking supported solutions.

You could also try ksmb but not sure how stable that is.

1 Like
8

Sounds like you are using basic SMB over TCP connections and not using the RDMA functions. Linux will usually enable RDMA by default, Windows usually will not. So check if thats enabled.

Also are you using basic Windows 10/11 Home or Pro? Those do not support RDMA as that is considered higher end business stuff. Only server editions and “Windows X for Workstation” allow RDMA use.

1 Like
9

So some more background.

I’ve set the “Recieve Window Auto-Tuning Level to Experimenta” ref: A word about AutoTuningLevel - TCP Receive Window Auto-Tuning Level explained | Microsoft Learn

I have updated every firmware possible, BIOS, NVMe OS Drive, Windows Motherboard Drivers, Latest NIC Driver (Intel E810-C-Q2).

I’m on the trail of SMB and specifically version being used. I will look at these SMB options you list as well.
Thanks for the info. I’m still getting my numbers together.