I posted this on Mikrotik forum also.
I can’t tell if there’s something I don’t know or if this Mikrotik is faulty. if it’s me and if anyone knows what I should be doing, please do tell.
Below I’ll just paste what I already wrote as it took a good 30 minutes to write.
How do these CRS*s play with other vendor equipment in your experience?
Here’s the setup we have.
Cat8 Cables.
SwitchOS or RouterOS in bridge mode give the same results.
CRS309-1G-8S+IN with S+RJ10 Revision 2
- Supermicro. Ubuntu. Marvell AQtion AQN-107 10 GbE NIC.
- Desktop. Windows 10. Two 10Gbps NICs. One Gigabyte and one Marvel.
- Laptop. Windows 11. Realtek 2.5Gbps NIC.
RouterOS reports 10G on all ports, that’s false.
Windows machines do report 10G and 2.5G on NICs.
Ubuntu reports 5Gbps NIC saying partner advertised only 5G.
iperf3 gives results that are all over the place.
Supermicro → Desktop == 4.5Gbps. Desktop → Supermicro == 93Mbsp. (When using ISP router its stable at 1Gbps upload and download).
Supermicro → Laptop == 1.7Gbps. Laptop → Supermicro == 2.3Gbps.
Laptop → Desktop == 2.3Gbps. Desktop → Laptop 500Mbps.
Editing MTU sizes does nothing. Well it does make it worse in some cases.
All drivers updated from Microsoft to Realtek and Marvel.
Driver on Ubuntu I didn’t check what that is, it’s Ubuntu 22.04 with kernel 5.15.
Did not try to turn off autonegotiation on Ubuntu NIC yet.
Sometimes the Link to desktop is paused as seen from SwOS.
What I am doing wrong or is this Mikrotik broken? Or is the all other 4 NICs broken but Mikrotik is fine?
Ethtool on Ubuntu/TrueNAS.
user@ubuntu:~$ sudo ethtool enp132s0
Settings for enp132s0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
2500baseT/Full
5000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: No
Link partner advertised FEC modes: Not reported
Speed: 5000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
Supports Wake-on: pg
Wake-on: d
Current message level: 0x00000005 (5)
drv link
Link detected: yesiperf3 client running on Laptop connected to Desktop.
Download 478Mbps
Upload 2.09Gbps
iperf3.exe -c 192.168.1.75 -R
Connecting to host 192.168.1.75, port 5201
Reverse mode, remote host 192.168.1.75 is sending
[ 4] local 192.168.1.23 port 55064 connected to 192.168.1.75 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 42.2 MBytes 354 Mbits/sec
[ 4] 1.00-2.00 sec 69.8 MBytes 583 Mbits/sec
[ 4] 2.00-3.01 sec 64.0 MBytes 536 Mbits/sec
[ 4] 3.01-4.01 sec 54.0 MBytes 452 Mbits/sec
[ 4] 4.01-5.00 sec 64.1 MBytes 540 Mbits/sec
[ 4] 5.00-6.01 sec 60.1 MBytes 501 Mbits/sec
[ 4] 6.01-7.01 sec 61.6 MBytes 517 Mbits/sec
[ 4] 7.01-8.00 sec 63.4 MBytes 538 Mbits/sec
[ 4] 8.00-9.01 sec 49.5 MBytes 411 Mbits/sec
[ 4] 9.01-10.00 sec 41.4 MBytes 351 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 570 MBytes 478 Mbits/sec sender
[ 4] 0.00-10.00 sec 570 MBytes 478 Mbits/sec receiver
iperf Done.
C:\Users\m\Documents\Supermicro-stuff\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.75
Connecting to host 192.168.1.75, port 5201
[ 4] local 192.168.1.23 port 55106 connected to 192.168.1.75 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 246 MBytes 2.06 Gbits/sec
[ 4] 1.00-2.00 sec 247 MBytes 2.07 Gbits/sec
[ 4] 2.00-3.00 sec 252 MBytes 2.12 Gbits/sec
[ 4] 3.00-4.00 sec 253 MBytes 2.12 Gbits/sec
[ 4] 4.00-5.00 sec 249 MBytes 2.09 Gbits/sec
[ 4] 5.00-6.00 sec 251 MBytes 2.10 Gbits/sec
[ 4] 6.00-7.00 sec 250 MBytes 2.10 Gbits/sec
[ 4] 7.00-8.00 sec 250 MBytes 2.09 Gbits/sec
[ 4] 8.00-9.00 sec 249 MBytes 2.09 Gbits/sec
[ 4] 9.00-10.00 sec 250 MBytes 2.10 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 2.44 GBytes 2.09 Gbits/sec sender
[ 4] 0.00-10.00 sec 2.44 GBytes 2.09 Gbits/sec receiver
iperf Done.iperf client on Desktop goint to Supermicro
Upload 88Mbps
Download 4.74Gbps.
C:\Users\M\Downloads\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.110
Connecting to host 192.168.1.110, port 5201
[ 4] local 192.168.1.75 port 65401 connected to 192.168.1.110 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.01 sec 11.1 MBytes 92.3 Mbits/sec
[ 4] 1.01-2.00 sec 10.4 MBytes 87.9 Mbits/sec
[ 4] 2.00-3.00 sec 8.75 MBytes 73.2 Mbits/sec
[ 4] 3.00-4.00 sec 11.1 MBytes 93.5 Mbits/sec
[ 4] 4.00-5.00 sec 11.4 MBytes 95.2 Mbits/sec
[ 4] 5.00-6.01 sec 11.5 MBytes 96.3 Mbits/sec
[ 4] 6.01-7.01 sec 8.12 MBytes 68.2 Mbits/sec
[ 4] 7.01-8.00 sec 11.1 MBytes 93.6 Mbits/sec
[ 4] 8.00-9.01 sec 11.1 MBytes 92.3 Mbits/sec
[ 4] 9.01-10.01 sec 11.0 MBytes 93.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.01 sec 106 MBytes 88.6 Mbits/sec sender
[ 4] 0.00-10.01 sec 105 MBytes 88.4 Mbits/sec receiver
iperf Done.
C:\Users\M\Downloads\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.110 -R
Connecting to host 192.168.1.110, port 5201
Reverse mode, remote host 192.168.1.110 is sending
[ 4] local 192.168.1.75 port 65414 connected to 192.168.1.110 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 563 MBytes 4.73 Gbits/sec
[ 4] 1.00-2.00 sec 565 MBytes 4.74 Gbits/sec
[ 4] 2.00-3.00 sec 565 MBytes 4.74 Gbits/sec
[ 4] 3.00-4.00 sec 566 MBytes 4.75 Gbits/sec
[ 4] 4.00-5.00 sec 566 MBytes 4.74 Gbits/sec
[ 4] 5.00-6.00 sec 564 MBytes 4.74 Gbits/sec
[ 4] 6.00-7.00 sec 565 MBytes 4.74 Gbits/sec
[ 4] 7.00-8.00 sec 566 MBytes 4.74 Gbits/sec
[ 4] 8.00-9.00 sec 566 MBytes 4.75 Gbits/sec
[ 4] 9.00-10.00 sec 566 MBytes 4.75 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 5.52 GBytes 4.74 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 5.52 GBytes 4.74 Gbits/sec receiver
iperf Done.Link paused
