I’m looking for any help on the topic below. Just saw Wendell’s recent post about adding 2.5 G NICs via M2 connectors to the inexpensive UM450. My asks are a bit above the spec.
What I’m looking for is fundamentally a router platform with the following characteristics:
Being reasonably small (no tower PC, rack machine, and the like)
Packs a punch to support packet processing up to 10 Gbit\s bidirectional with NAT enabled
Swappable NICs to adapt to changing environments (2.5 GbE, 10 GbE, SFP+, SFP28, and the like)
I’m looking for 10G because of the Inter-VLAN routing of existing 10G circuits in my home lab.
My favorites so far:
(A) Minisforum Elitemini B550
Benefits:
Powerful CPU (5700G)
PCIe Slot to plugin any NIC of choice
Open Questions:
A.1. Does anybody know the specs for the attached PCIe connector? Which PCIe standard and how many lanes?
(B) Minisforum UM690
Benefits:
Powerful CPU (6900HX)
USB4 Port (40 GBit\s interface)
Open Questions:
B.1. Anybody has experience with the platform supporting PCIe tunneling through USB4?
B.2. Do Anybody have recommendations for PCIe enclosures? Or experience with USB4 C / Thunderbolt 3 to 10 Gbit NIC adapters?
(C) Modern Intel Small Form Factors (NUCs, Minisforums)
Benefits:
Hi,
I have been looking at the same issue for a while, here are my findings (not the holy truth) that might help you.
1 - Someone will correct me if I am wrong, because I can’t find the documentation where I had found it but: to do proper IDP/IPS filtering of the WAN, the rule of thumb is roughly 4Mhz of CPU speed per 1 Mbps filtered
2- The NIC of choice would be an intel X710 but 10GBps cards tend to get hot
Based on that: the processors A) and B) are fine but
A.1, B.1, B.2 => fitting a full size external pci NIC doesn’t seem very reliable in the long run (I wouldn’t do it for the router on which my entire house (life?) relies, but I might not have understood your plan correctly)
C => I had initially considered a nuc with a small 1U passive case from Akasa (Galileo TU3) that supports a proper PCI card, but it requires a thin mini ITX mb and is virtually impossible to find.
For reference, in the end I bought a Supermicro x10sdv-12c-tln4f which has a Xeon D-1557 12 Cores 24 threads 1.5Ghz base clock / 2.1Ghz Turbo 45W with 128Gb of DDR 4 with a very small 1U case that allows having the pci card in the front (SC 505-203B).
I initially wanted to use it to do a forbidden router (before it was actually called that) so to filter 8 Gbps I needed to dedicate a VM with 8 cores and 16 threads
I KNOW wendell would say that “an alderlake would run circles around my Xeon D”, which is true, but I got it for 250 EUR, and if I had the money for an alderlake platform, I wouldn’t “waste” it on a router.
Anyway, I kind of toasted it while doing some “fanless” testing, so work still in progress…
Hmm, if we’re talking ultra SFF routers, then there are two big questions:
One network card, or two?
External or internal power?
See, it’s mostly a question of how small you feel like going. Do you want to go extremely SFF, or are you okay with just slightly bigger? My take here with one network card and internal power would be the J-Hack Pure X:
As for your traffic handling, a potato CPU can do most of the lifting; all heavy action happens in the network card itself. The CPU only programs the card registers and perhaps do some brief firewall inspections of really suspicious packets that pass the fourth screening.
Here is a latest-intel build that should fit the Pure X:
As with all my build suggestions, they are supposed to be just that; suggestions. Feel free to change to something cheaper or more reliable here. AMD with a 5600G could fit the bill neatly here, too.
To be fair though, DIY when going this small will almost always be more expensive, take up more room and be perhaps louder than you’d like. That said, that’s the price you pay for upgradability, and the upgrade may be less expensive.
(1) There is no plan to enable IDS/DPI inspection on those east-west links. I probably do it for north-south, but those are 1 Gbit\s connections.
(2) X710 will be the series of choice, but for my use case, a T2L is my preference, as my 10GBit is all good old copper. That hopefully reduces the heat issue a bit.
What are your reliability concerns regarding an exposed PCIe nic?
I think you hit an essential spot. Maybe I’ve too narrowed down already on the super small form factor machines.
A B550 M-ITX mobo in a big tower (DUAL PC case) is still idling with an AMD 3400G and a few RAM sticks. The fractal design node 202 with a built-in power supply and a NOCTUA cooler could provide me with a small form factor machine.
