I’m currently at the point where I think these cards only support 1.5 watts per port, and I think all these optics have been 2.5 watts and 3.5 watts.
I don’t know if there any way to override the max power per port?
I have very active cooling on the card.
I have some ConnectX-3 cards (314A) and they work fine with optics. Never had an issue at all. The Mellanox transceiver you tried also has a 1w max power draw on its spec sheet. I also use some RJ45 transceivers in the cards and they have a power draw of about 3w and it works just fine.
It’s got me wondering if there’s a power setting somewhere I need to change.
I’ve tried both CX314A-BCBT and CX354A-QCBT cards and the optics are detected but don’t turn on.
In hindsight, it looks like I needed a CX354A-FCBT card, but I’ve seen multiple people get the previous two working with similar optics.
That’s a really good point. I only looked at the Cisco optics. Having the same issue with this 1W optic.
I didn’t know that ConnectX-3 cards are so picky.
I have Lenovo CX3 Pro and CX3 from Oracle and HP and tried DAC cables and transceivers from Oracle, HP, HUAWEI and Cisco they all worked, even a breakout cable from FS works, although only one of the four SFP+ has a signal, but basically it works.
HUAWEI transceivers
Oracle DAC works with 56GbE
PN:7354328
FS Breakout cable,was intended for something else, but works to connect to a switch with 10Gb SFP+
I sold the Cisco Transceiver but as soon I am back in Office next Monday I can give you the HP PN too.
edit: Interesting, that answered a question that I had myself
That’s the Oracle DAC PN:7354328
[manja-02 ~]# ethtool -m enp5s0
Identifier : 0x11 (QSFP28)
Extended identifier : 0x00
Extended identifier description : 1.5W max. Power consumption
Extended identifier description : No CDR in TX, No CDR in RX
Extended identifier description : High Power Class (> 3.5 W) not enabled
Power set : On
Power override : On
Connector : 0x23 (No separable connector)
Transceiver codes : 0x88 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 40G Ethernet: 40G Base-CR4
Transceiver type : 100G Ethernet: 100G Base-CR4 or 25G Base-CR CA-L
Encoding : 0x05 (64B/66B)
BR, Nominal : 25500Mbps
Rate identifier : 0x00
Length (SMF,km) : 0km
Length (OM3 50um) : 0m
Length (OM2 50um) : 0m
Length (OM1 62.5um) : 0m
Length (Copper or Active cable) : 1m
Transmitter technology : 0xa0 (Copper cable unequalized)
Attenuation at 2.5GHz : 2db
Attenuation at 5.0GHz : 4db
Attenuation at 7.0GHz : 4db
Attenuation at 12.9GHz : 7db
Vendor name : Volex Inc.
Vendor OUI : 14:1b:bd
Vendor PN : VQ2830LP100L
Vendor rev : 01
Vendor SN : Q28PL1020161S341
Date code : 200611
Revision Compliance : SFF-8636 Rev 2.5/2.6/2.7
Module temperature : 0.00 degrees C / 32.00 degrees F
Module voltage : 0.0000 V
[manja-02 ~]# ethtool enp5s0d1
Settings for enp5s0d1:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseKX4/Full
40000baseCR4/Full
40000baseSR4/Full
56000baseCR4/Full
56000baseSR4/Full
1000baseX/Full
10000baseCR/Full
10000baseSR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10000baseKX4/Full
40000baseCR4/Full
40000baseSR4/Full
56000baseCR4/Full
56000baseSR4/Full
1000baseX/Full
10000baseCR/Full
10000baseSR/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 40000baseCR4/Full
56000baseCR4/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 56000Mb/s
Duplex: Full
Auto-negotiation: on
Port: None
PHYAD: 0
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000014 (20)
link ifdown
Link detected: yes
Appreciate the time you put into this! My 10Gtek DACs have worked perfectly. Only had a problem with the QSFP and SFP transceivers.
Tried the CX314 and CX354 cards in a Supermicro MBDH12SSLNTO Motherboard with an EPYC 7763 and still had the same exact issue.
Went ahead and returned all the CX3 cards and bought some CX4 cards. Going to see if they make a difference, especially since I’m using BIDI transceivers.
I know necro posting is frowned upon, but OP did eventually figured this out, or you gave up and went with the CX4?
I had a similar problem where the 10G BiDi transceivers (in QSA adapters) are detected by the MCX354A-FCBT, but the lasers doesn’t fire.
I figured it had something to do with TX_DISABLE.
Module and fibre was verified to be working on a switch.
sudo ethtool -m enp36s0
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x07 (LC)
Transceiver codes : 0x20 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 10G Ethernet: 10G Base-LR
Encoding : 0x06 (64B/66B)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 10km
Length (SMF) : 10000m
Length (50um) : 0m
Length (62.5um) : 0m
Length (Copper) : 0m
Length (OM3) : 0m
Laser wavelength : 1270nm
Vendor name : SE
Vendor OUI : 00:90:65
Vendor PN : 10G-BX10-1270U
Vendor rev : A
Option values : 0x00 0x1a
Option : RX_LOS implemented
Option : TX_FAULT implemented
Option : TX_DISABLE implemented
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : SE2401GU125
Date code : 240318
Optical diagnostics support : Yes
Laser bias current : 0.002 mA
Laser output power : 0.0001 mW / -40.00 dBm
Receiver signal average optical power : 0.0001 mW / -40.00 dBm
Module temperature : 11.30 degrees C / 52.34 degrees F
Module voltage : 3.2727 V
Alarm/warning flags implemented : Yes
Laser bias current high alarm : Off
Laser bias current low alarm : Off
Laser bias current high warning : Off
Laser bias current low warning : Off
Laser output power high alarm : Off
Laser output power low alarm : On
Laser output power high warning : Off
Laser output power low warning : On
Module temperature high alarm : Off
Module temperature low alarm : Off
Module temperature high warning : Off
Module temperature low warning : Off
Module voltage high alarm : Off
Module voltage low alarm : Off
Module voltage high warning : Off
Module voltage low warning : Off
Laser rx power high alarm : Off
Laser rx power low alarm : On
Laser rx power high warning : Off
Laser rx power low warning : On
Laser bias current high alarm threshold : 100.000 mA
Laser bias current low alarm threshold : 0.000 mA
Laser bias current high warning threshold : 100.000 mA
Laser bias current low warning threshold : 0.000 mA
Laser output power high alarm threshold : 1.7783 mW / 2.50 dBm
Laser output power low alarm threshold : 0.0955 mW / -10.20 dBm
Laser output power high warning threshold : 1.1220 mW / 0.50 dBm
Laser output power low warning threshold : 0.1514 mW / -8.20 dBm
Module temperature high alarm threshold : 95.00 degrees C / 203.00 degrees F
Module temperature low alarm threshold : -50.00 degrees C / -58.00 degrees F
Module temperature high warning threshold : 85.00 degrees C / 185.00 degrees F
Module temperature low warning threshold : -40.00 degrees C / -40.00 degrees F
Module voltage high alarm threshold : 3.6000 V
Module voltage low alarm threshold : 3.0000 V
Module voltage high warning threshold : 3.5000 V
Module voltage low warning threshold : 3.1000 V
Laser rx power high alarm threshold : 1.7783 mW / 2.50 dBm
Laser rx power low alarm threshold : 0.0158 mW / -18.01 dBm
Laser rx power high warning threshold : 1.1220 mW / 0.50 dBm
Laser rx power low warning threshold : 0.0251 mW / -16.00 dBm
Edit: It is in fact, TX_DISABLE’s doing.
I bridged TX_DISABLE (pin 3) with VeeT (pin 1) and the laser started up.
SFP pin out in this wiki page:
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x07 (LC)
Transceiver codes : 0x20 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 10G Ethernet: 10G Base-LR
Encoding : 0x06 (64B/66B)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 10km
Length (SMF) : 10000m
Length (50um) : 0m
Length (62.5um) : 0m
Length (Copper) : 0m
Length (OM3) : 0m
Laser wavelength : 1270nm
Vendor name : SE
Vendor OUI : 00:90:65
Vendor PN : 10G-BX10-1270U
Vendor rev : A
Option values : 0x00 0x1a
Option : RX_LOS implemented
Option : TX_FAULT implemented
Option : TX_DISABLE implemented
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : SE2401GU125
Date code : 240318
Optical diagnostics support : Yes
Laser bias current : 36.720 mA
Laser output power : 0.6637 mW / -1.78 dBm
Receiver signal average optical power : 0.0001 mW / -40.00 dBm
Module temperature : 10.86 degrees C / 51.54 degrees F
Module voltage : 3.2317 V
Alarm/warning flags implemented : Yes
Laser bias current high alarm : Off
Laser bias current low alarm : Off
Laser bias current high warning : Off
Laser bias current low warning : Off
Laser output power high alarm : Off
Laser output power low alarm : Off
Laser output power high warning : Off
Laser output power low warning : Off
Module temperature high alarm : Off
Module temperature low alarm : Off
Module temperature high warning : Off
Module temperature low warning : Off
Module voltage high alarm : Off
Module voltage low alarm : Off
Module voltage high warning : Off
Module voltage low warning : Off
Laser rx power high alarm : Off
Laser rx power low alarm : On
Laser rx power high warning : Off
Laser rx power low warning : On
Laser bias current high alarm threshold : 100.000 mA
Laser bias current low alarm threshold : 0.000 mA
Laser bias current high warning threshold : 100.000 mA
Laser bias current low warning threshold : 0.000 mA
Laser output power high alarm threshold : 1.7783 mW / 2.50 dBm
Laser output power low alarm threshold : 0.0955 mW / -10.20 dBm
Laser output power high warning threshold : 1.1220 mW / 0.50 dBm
Laser output power low warning threshold : 0.1514 mW / -8.20 dBm
Module temperature high alarm threshold : 95.00 degrees C / 203.00 degrees F
Module temperature low alarm threshold : -50.00 degrees C / -58.00 degrees F
Module temperature high warning threshold : 85.00 degrees C / 185.00 degrees F
Module temperature low warning threshold : -40.00 degrees C / -40.00 degrees F
Module voltage high alarm threshold : 3.6000 V
Module voltage low alarm threshold : 3.0000 V
Module voltage high warning threshold : 3.5000 V
Module voltage low warning threshold : 3.1000 V
Laser rx power high alarm threshold : 1.7783 mW / 2.50 dBm
Laser rx power low alarm threshold : 0.0158 mW / -18.01 dBm
Laser rx power high warning threshold : 1.1220 mW / 0.50 dBm
Laser rx power low warning threshold : 0.0251 mW / -16.00 dBm
how eactly did you do this?
like, a smol solder job?
insulating tape over pin2, then conductive tape linking 1&3? over the top of the insulation?
I just bridged Pin 1, 2 and 3.
Pin 2 is Tx_Fault whish disables the transmitter upon a fault. I just said screw it and bridged both Tx_Fault and TX_DISABLE to ground. 
I did it using a small piece of aluminium foil so everything is reversable. Insulated the foil with a piece of kapton tape so nothing else will blow up.
