No network with X553 NIC, TrueNAS Scale, and 2.5Gbe switch

Networking Networking Hardware
,
1

Hi there!
I am new to networking and using networking equipment. Very new.

I recently purchased an Supermicro A2SDi-H-TF motherboard and two 32GB Samsung DDR4 2400 ECC RDIMMs. The board has an onboard X553/X557-AT Intel NIC and provides dual 10Gbe RJ-45 sockets.

However, I don’t have any 10Gbe switches or routers. I have a 2.5Gbe Hasivo S1100P-8GT switch.

When installing TrueNAS-SCALE-22.02.4, I cannot connect to the web GUI and TrueNAS cannot connect to the network through the Hasivo 2.5Gbe switch. I get this:

image
image724×782 91.1 KB

Here are the things I have tried to get this working with the 2.5Gbe switch:

  1. I tried installing the Intel ixgbe kernel drivers suggested by Patrick from STH for a similar Supermicro A2SDi-H-TP4F motherboard with the same Intel X553 NIC. The driver version Patrick mentioned as working (5.0.4) is no longer available from Intel, so I tried installing the latest version (5.16.5) from Intel’s website. I had to follow the README instructions in the tarball instead of Patrick’s instructions because Patrick’s instructions didn’t work at the sudo insmod ./ixgbe step. The executable in Patrick’s instructions did not exist. Even so, these Intel Linux drivers did not help.
  2. I tried installing TrueNAS-13.0-U2 (TrueNAS Core). This is the BSD version, and it supposedly had the Intel X553 drivers first. However, when I installed TrueNAS Core, I got the following:
    image
    image586×671 41.6 KB

    This effectively turned out to be the same result as with TrueNAS Scale (which is the Linux version of TrueNAS). I don’t know how to use BSD or install BSD drivers, so I did not try that. I would like to stick with TrueNAS Scale for the Docker containers anyway.
  3. I tried using a different Cat-6 cable that I know works on another TrueNAS Scale 22.02.4 box. This box uses an ASUS ROG B550-I motherboard with an onboard Intel I225-V 2.5Gbe NIC, and that box is able to auto-negotiate with the Hasivo 2.5Gbe switch at 2500Mbs:
    image
    image1977×69 47.5 KB

    Even so, this cable did not work with the A2SDi-H-TF box and TrueNAS Scale. I have the same result.

Finally, I pulled out my TP-Link TL-SG108E 1Gbe switch and tried that. With this switch, TrueNAS Scale and TrueNAS Core are able to connect to the network, get a valid IP address, and I am able to access the TrueNAS Scale web GUI.

I have many questions. Am I doing something wrong? Is the X553 Intel NIC not capable of auto-negotiating 2.5Gbe? Do the Linux ixgbe drivers not support 2.5Gbe? Am I the silliest nutcase you have ever heard of?

I would love to hear your comments. Thanks for reading!

2

I should mention that I also tried to 2) Configure network settings in TrueNAS Scale. I specified my router/gateway IPv4 address, but this operation failed:

image
image1676×423 67.7 KB

This operation succeeds with the TL-SG108 switch.

Because I am a nutter, I also tried letting TrueNAS Scale auto-negotiate with the TL-SG108 switch, and then physically disconnecting the Cat-6 and reconnecting it to the Hasivo 2.5Gbe switch. Yeah, that did not work.

3

2.5GBe is a consumer standard, you bought a “server” board.

at least you need something like this

4

Thank you for the reply! I didn’t realize that 2.5Gbe is not a standard supported by servers.

Sounds like I need to invest in a 10Gbe switch.

5

It depends on the network cards/chips used, x550 supports 2.5/5 Gbps (in addition to 1G/10G), that motherboard does not.

Some older 10Gbps switches also don’t support NBASE-T, some SFP+ switches do support NBASE-T as long as you use an appropriate transceiver (to convert SFP+ to RJ45)

1 Like