Dell OEM Mellanox ConnectX4 in Non-Dell Server: Do I need to flash fIrmware?

I’ve asked elsewhere and gotten some contradicting answers, so I was really hoping to get some more feedback here. :slight_smile:

Hello!

I snagged one of these on eBay for $19, here: Dell Mellanox CX4121C ConnectX-4 Dual 10/25GBE SFP28 Low Profile | eBay The exact model number as listed was Dell Mellanox CX4121C ConnectX-4.* I put it in my test Linux Mint Debian Edition box, and LMDE not only saw it, but auto configured it for DHCP, and it auto negotiated to 2.5 Gbps on my multigig switch via a 10GTek RJ-45 transceiver.

NVIDIA’s Debian management software sees it just fine.

I couldn’t be happier so far with how it’s working out.

I did have a question about the firmware, and I’d really appreciate some help. It clearly identifies as a Dell device.

$ sudo mlxfwmanager --query 
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX4LX
  Part Number:      020NJD_0MRT0D_Ax
  Description:      Mellanox 25GBE 2P ConnectX-4 Lx Adapter
  PSID:             DEL2420110034
  PCI Device Name:  /dev/mst/mt4117_pciconf0
  Base MAC:         98039b993a82
  Versions:         Current        Available     
     FW             14.32.2004     N/A           
     PXE            3.6.0502       N/A           
     UEFI           14.25.0018     N/A           

  Status:           No matching image found

The part number and PSID are Dell variant identifiers. From this thread here-- Mellanox ConnectX-4 CX4121C firmware | ServeTheHome Forums – it’s apparently a Mellanox MCX4121A-ACAT with custom Dell firmware.

Now, as reluctant as I am to mess with something that’s working, I’ve had bad luck putting HP firmware flashed SAS drives into a non-HP server, so now I’m a bit nervous about just leaving the HP firmware on there, even though Debian seems to be okay with it so far. I’m going to be putting this card in a server that will be hard to offline long enough to cross-flash it later, if that becomes necessary.

I promised questions.

  1. Did I actually read that STH thread correctly? I’m a bit nervous that I’m not ID’ing the actual card correctly.
  2. Is there some reason not to flash the actual Mellanox firmware onto the card? See: Firmware for ConnectX®-4 Lx EN
  3. Do I have the latest firmware already? I’ve got 14.32.2004, but the release notes for the last version are for 14.32.1900 ( NVIDIA ConnectX-4 Lx Adapter Cards Firmware Release Notes v14.32.1900 - NVIDIA Docs ). I suspect the OEM versions have slightly different numbers…

Specifically, here’s what I’m seeing on NVIDIA’s site:

*Spectacular price and rock solid packaging and shipping speed. Would absolutely recommend buying from here.

I have one of these Dell cards. From what I can gather cross-flashing to “real” Mellanox f/w may be beneficial if you run into issues with transceiver compatibility.

I keep hearing conflicting info for ConnectX-4 products regarding a vendor-lock for DAC’s, optical modules, AOCs, and the like. Some say anything goes while others have run into problems with off-brand hardware.

I suspect both might be correct – original Mellanox cards don’t care but the OEM “branded” cards might.

FWIW I’ve found no compelling reason to cross-flash at the moment and plan to remain on the latest Dell firmware.

I flashed mine based on instructions here: Mellanox ConnectX-4 CX4121C firmware | ServeTheHome Forums

I went ahead and posted my experience there with a bit more detail. If you’ve never done it before, some of the outputs and commands are a bit confusing, and part of it does make you wait.

I was getting an amber (warning?) light with the Dell firmware, that went away with the stock Mellanox firmware. It definitely wasn’t vendor-locked and seemed to work fine with the Dell firmware, but the server I’m going to put it in is a pain to disassemble and even to offline for troubleshooting, so I went ahead and flashed it to avoid even the possibility of trouble.

(That is to say, I’m not good enough with networking gear yet to easily troubleshoot esoteric firmware problems that I may or may not encounter six months from now after I’ve forgotten that there’s Dell firmware on my NIC that might not like the whatever-future-me-has-just-done. :stuck_out_tongue: )

Once you’ve got good instructions, it’s a surprisingly easy process. As a bonus, the stock Mellanox management tool seems to be able to check for updates once you put the stock firmware on the card.

It also cleanly handles resetting the card and bringing it back up so you don’t even have to reboot after the flash. Very impressive.

$ sudo mlxfwmanager --query
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX4LX
  Part Number:      MCX4121A-ACA_Ax
  Description:      ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6
  PSID:             MT_2420110034
  PCI Device Name:  /dev/mst/mt4117_pciconf0
  Base MAC:         98039b993a82
  Versions:         Current        Available  
     FW             14.32.1900     14.32.1900
     PXE            3.6.0502       3.6.0502  
     UEFI           14.25.0017     14.25.0017

  Status:           Up to date
2 Likes