Which SFP+ Cable?

Networking Networking Hardware
1

I have an SFP+ expansion card, a T520-CR 2PORT 10GB PCI-E HBA LP (CHELSIO) - which I got because I was able to confirm it was seen by TrueNAS.

Which it is - but I have “no carrier” - and I think it is because the SFP+ cable that I have is potentially incorrect or just plain broken…

I have a 10GTek SFP+ DAC cable.

When I boot up TrueNAS the Chelsio lights up - and then as TrueNAS is booting, I can see the activity lights on the card attempting to send data - but it fails and the card powers down.

My Router the AX89X - doesn’t recognise that an SFP+ cable is plugged in - i.e. the LED for the SFP+ port does not light up, but in testing, it doesn’t recognise a a cable in the 10gb T-Base port either - unless the cable is plugged in and data is travelling on the cable - otherwise the LED stays off (I tested with a cable only plugged in on the Router and not plugged in anywhere else).

So I leaning towards this just being the “wrong type” of SFP+ cable.

Wondered if anyone could confirm and help me out with what I actually need or confirm that this cable should work and I should return it for a replacement.

2

Looking at it and digging a little deeper, I am not sure it is the cable…

TrueNAS reports the following, as this is a two SFP+ port card:

cxl0: Media: Ethernet None
cxl1: Media: Ethernet 10GBase-Twinax <full-duplex…>

Which to me says the card is seeing the cable - which is what I thought, due to the way the lights on the card looked like they were trying to send data…

My router on the other hand has this:

image

It doesn’t matter how I configure this even without the other 10gb:

image

I have raised a support request with ASUS but thought I would post here, in case someone had some insight so I don’t have to wait a week for a response.

3

This is all very odd, I dug into the logs a little and I found this:

Nov 1 23:09:25 kernel: [19549.805504] sfp_phy_link_change_notify[161]:INFO:SFP+ nolink.
Nov 1 23:09:35 kernel: [19559.805501] sfp_read_status[119]:INFO:SFP+ plugged-in.

Which is the Router recognising that a SFP+ cable is being plugged in, but still reports that no cable is plugged in…

Infuriating! lol

4

For my switch I have to set the correct cable typ, either DAC10G or SFP+.
Same with your nic, it may be that the card does not support DAC cable.
I don’t know Chelsio, but at least Emulex cards distinguish between cards for SFP+ or DAC10GB

5

I found a QVL for the AX89X here - well someone else did, and I found their post referring to it:

The cable I bought, is the one listed on here.

As noted above as well, the Chelsio is recognising that a cable is present in TrueNAS - it is the Router that doesn’t seem to see that a cable is plugged in…I think the router port may be duff?

Checking Chelsio’s website that cable should work with their product as well, which seems correct given that TrueNAS recognises the cable and type.

6

have you tried to configure an IP address in Truenas?
FTLX8571D3BCL, F418S27413D and FTLX8571D3BCV-IT are all Finisar Transceiver Modules, you can buy them for $20-30 a pair on Ebay, pls a cable like this

7

Sometimes, you have to explicitly set the duplex and the link speed on the NIC and the switch. Have you tried that?

1 Like
8

These are the SFP+ DAC cables on the Asus AX89X qualified Vendor List:

For an SFP+ tranceiver that you use your own fiber cable for, id go with this Intel one that is on their QVL:
https://www.google.com/search?q=Intel+FTLX8571D3BCV-IT

https://www.asus.com/support/FAQ/1044066/

The Chelsio should work fine with most SFP+ type things. I have used both Mellanox coded and generic coded products on a T520-CR before in my network and it worked just fine.
One option though is to use the Intel SFP+ transceiver in the Asus that is on their QVL, and buy a CHelsio SFP+ transceiver from their web store in the NIC, then connect it with an LC to LC fiber optic multi-mode cable. That way each device has its specific hardware in it. Only do this if you cant get your other cable working though.
Chelsio brand SFP+:

9

It is very odd, and I have raised it with ASUS support, but they are taking their time…

My router log states the following:

Nov 5 09:25:28 kernel: [ 258.896207] sfp_phy_link_change_notify[161]:INFO:SFP+ nolink.
Nov 5 09:30:34 kernel: [ 564.896226] sfp_read_status[119]:INFO:SFP+ plugged-in.

Truenas shows the following:

image
image704×552 40.9 KB

I tried to adjust the settings on the Router as suggested above, and I tried to assign an IP address to cxl1 - but didn’t really change anything.

The cable I purchased was this one:

And I know it says H10GB, but I think that is referring to that which it is compatible with, as if you look at the model number of the part at the bottom it says under technical details:

Item model number: CAB-10GSFP-P2M-30(Cisco-1pc)(UK)

I don’t know if that is different from:

10Gtek, CAB-10GSFP-P2M-30-Ubiquiti

Which is on the QVL list, I would have thought it was the same, but who knows at this stage.

I have just searched for the Ubiquiti cable - and seen it for sale with that particular part number - though not really in the UK…I will try and buy one of those and see how that goes…

Well ordered an AWG30 cable that is Ubiquiti compliant by part number…annoyingly, it didn’t say -30 on the part number, but the description of the cable referred to it as an AWG30 cable - so hoping they just omitted that part from the model number which is: CAB-10GSFP-P2M-Ubiquiti

I think the P2M is merely the length, so hoping that the -30 just refers to the AWG.

10

Ok, the new cable arrived - is seen by both the Chelsio card and the Router.

The link is currently down, but I think I need to assign it an IP address as mentioned earlier in this thread.

Will need to look at how to do that.

11

Yay!

image

That was a pain…and thanks to @Janos for the hint about configuring the IP - that sorted it straight away, so appreciate that!

12

They are different. SFP transceivers have coding in them to identify as a specific device. The majority of networking hardware out there works with “generic” code devices, such as Ubiquiti, Mellanox, Chelsio, etc. However certain “big name” enterprise network companies use their market positions to take advantage and require specially coded transceivers. Companies like Cisco and Juniper for instance. So a Cisco-coded device will not always work in other hardware. That is why your new cable with “Ubiquiti coding” is seen by the router and the old one was not. This sort of thing is annoying but is only really an issue if you are doing things with Cisco and Juniper hardware.