Although the NIC is between two high-end GPUs, I don’t think it’s the problem, since the GPUs at idle shouldn’t cook it and I have some airflow coming at PCI-E area.
But turned out the main problem was MTU size not getting changed on the interface when I changed it in the adapter settings GUI.
I have Disabled almost everything in the Adapter Properties:
- Downshift retries
- Energy-Efficient Ethernet
- Flow Control
- Interrupt Moderation
- Interrupt Moderation rate
- Large Send Offload (all version and protocols)
- Receive Side Scaling
- Recv Segment Coalescing (IPv4/6)
And now I see a 1second burst of ~950MB/s and then it can be around 500 to 300s and can go as low 200s.
When I had some or all of these features enabled, it was rarely above 400, usually between 200s and 300s.
No Memory or CPU shortage on the NAS, or on the desktop.