25Gbps NAT on commodity HW

Hey everyone,

Just got my Internet connection from my ISP upgraded to 25Gbps (There is no typo there), and now I’m facing rich kids problems. It seems I’m getting some BW limitation, most likely due to hardware. That’s why I’d like some of the communities views on that one.

My hardware/software setup is : Elitedesk G2 i3-6100 + Mellanox ConnectX-4 + Proxmox - Interface bridge to a Debian VM.

I suspect that my limitation is my CPU clock/speed, and I’m considering upgrading to I7-6700 (that’s what the Elitedesk G2 is supposed to sustain), and didn’t tried my Ryzen 5600X yet to see what was the efficiency comparison.

Otho, I know that most of my tests (speedtest/iperf3) were not optimized for that kind of loads (multicore load- Any ideas there ? ). My end target is to NAT on my GW (I’m platform agnostic, but it’s still a " home " setup, so Intel / AMD is an option, noisy fan aren’t.)
I didn’t find yet an option for fanless SFP28 25G/10G switch (I know there’s a Mikrotik out there in the wild that will accept SFP28 fanless., but it’s categorized as " Router " ).

Ho and for reference, I was able to speedtest up to 12Gbps and Iperf3 up 17Gbps (multiple servers) with this CPU.

Any thoughts ?

PS: The uplinks/path toward the speedtest/ipref are multiple 100G and known path, so I should have a decent metric for measurements, I’m open to debate this around PMs.

1 Like

Jealous.

What country/ISP? and how much / month?


(Re technical stuff, I’m thinking alderlake with xdp… Not sure)

1 Like

Switzerland, Init7, 777CHF per year (roughly 840USD on today’s exchange rate). Also note that there I had to pay a fee for the upgrade to 25G, roughly the cost of the optic in the POP. No CGNat, IPv6 prefix delegation, CPE agnostic. And I’m not working for them, they are friends, I work for another ISP also in Switzerland (just so it doesn’t look like advertisement :wink: ) .

Perhaps relevant: 25 Gigabit Linux internet router PC build (2021) - Michael Stapelberg

1 Like

Thanks for the link, already was aware of Michael’s blog post, I just wanted alternative feedback.

In the interval, I took a chance to drop the Connectx4 in my ryzen, and could directly reach 23 gbps download test with a low cpu load from the gateway without any major tweaks. So that’s a quite good sign that the poor i3 is a bit low power to handle those 25gbps.
I’ll continue my exploration and see what are the best settings for efficient NAT.

2 Likes

Maybe see if you can get a i7 6700t or or 6400t?

Either way, even if you did a 6500 you should still be able to use your skylake box I reckon.

Good point, I also thought of that option, as the i7-6700 is supported by the ElitedeskG2. I looked up the used market prices (available around here) and the delta with a more recent AMD Ryzen 5 new is so low that I’ll probably build a new machine and reuse the old thing elsewhere.