I need a ≥25GbE NIC that doesn't have suspend/resume issues (on Arch)

Hi,

Starting to get a little frustrated at my luck with high-bandwidth NICs.

Dodgy AliExpress ConnectX-3 40G causes a no-boot on my NAS. No good, not using any of those anywhere.

(Dell) E810-XXVDA2 2x25G takes a shit on resume in my workstation, I read it is a thing with those cards, they have some bus-conflict-type issue if you run them through a sleep-wake cycle and won’t recover until you restart the PC. (obviously unacceptable for home PC/workstation use)

Hence currently running an Intel XXV710-DA2 2x25G in my workstation and one of the E810-XXVDA2s in my NAS (as that doesn’t need to sleep).


But, the XXV710-DA2 has its own issue with suspend and resume!

The link is always dropped after a sleep cycle, and doesn’t automatically reconnect. In fact, the link drops to “unavailable” status in NetworkManager GUI (and disappears from the list in the NM applet). However manually connecting with nmcli connection up $LINKNAME does restore the link and I can continue using the computer as normal.

(I tried to make a systemd unit to do this for me as a workaround, but it never worked, so I made a short alias for the command and have to run it every time I wake the workstation, which is obviously suboptimal)

So, I am wanting to find myself a NIC that plays nice, and wondered if y’all had some ideas. Target OSs are Cachy/KDE and Debian

I am pondering ditching TrueNAS (due to their enshittification) on my main NAS, so the ability to use RDMA may open up, it’d be nice if the NICs were capable.

Not really interested in anything faster than 100G or slower than 25G.

Any more you need to know, please ask

TIA!

I’ve had pretty decent luck with a Mellanox ConnectX-4 LX (CX4121A) from eBay; only tested with TrueNAS (Debian) though, not Arch. Worked well enough with a direct connection between my TrueNAS server and Windows PC with two ConnectX-4 LX’s, two DELL SFP28 transceivers (0M14MK), and some Cable Matters 40/100Gb OM4 LC fiber cable.

2 Likes

I would second ConnrctX4-Lx, using them in most of my machines.

But just to add, a non-OEM Intel XXV710-DA2 with up to date firmware shouldn’t have such issues either. Are you sure its not something with your transceivers or switch, etc?

1 Like

No switch here, these are point-to-point links. Transceivers are nice Flexoptix ones, P.8525G.01, don’t expect any issues there…

(I have no clue on firmware though. I bought the cards off eBay and straight into the computers they went..)