Ok so I recently went down the path of darkness, The Forbidden Router. In theory it is great but I am having some issues in practice. Currently everything seems to be set up fine but I get terrible throughput. I can run a speedtest from the router directly and get 1Gbps+ speeds but then when I use a client on the LAN I am getting capped at 150Mbps roughly. I am not running out of CPU or memory. Lan is Router>SFP+ DAC>Layer 2 SFP+ Switch>10G Copper SFP+>Gig ethernet switch>Laptop. Everything seems to be negotiated at Gig or better speeds. Where should I start looking to try and troubleshoot this thing?
Ok, so I had this issue… its really stupid…but this simple fix ended up working for me.
I messed up and either had a broken, miss wired, or cheap miss labled cable that was only rated for fast ethernet… it needless to say ruined my 10g speeds on LAN.
Im sure you probably checked…but I thought I had checked mine too till the cable that was labled correctly didnt perform as it should have.
Are you going out to the Internet to get your speed? Or, two computers on the same router? If the internet, you might be only 150 Mbps!
1 Like
When I run a speedtest from the CLI on the WAN port it is 1180Mbps. When I run a speed test from a client on the wan it is 150Mbps. I swapped it our with my old router which only has Gig ethernet ports and I am getting 700-800Mbps. Same cables same everything just a different router.
1 Like
- Do you have IDS/IPS or anything else enabled on the OPNsense that could impact performance?
- Is your WAN Connection PPPoE? If so, you might have to adjust settings according to your ISP
- Your Router is virtualized - did you pass through the NIC or emulate one on a Bridge?
- What does an iperf test between your laptop and the Router result in?
1 Like
No
DHCP from the cable company
Passthrough
I did not do this.
2 Likes
Please give this a try.
There could be all kinds of issues arising from the passed through NIC and other factors within the setup. If the iperf test doesn’t perform well, it’s not a routing issue.
But if it does perform well, it’s most likely a software/configuration thing and we’d need more info on your configuration.
WAN Connection PPPoE
Not sure if it is better to start a new thread but I’m gonna shoot my shot right now and do that later if needed.
Long story short. ISP using trashy ONT that give around 8Gbps instead of 10Gb symmetrical (those speeds are reported from users on local forums). Now, I used pfsense sandboxed but that’s all, and I’ve heard about PPPOE HW bottlenecks and at 1Gbps got less performance than with their ONT. So I want to build one of the infamous “forbbiden routers”. Is there any HW configuration to get better speeds for PPPOE? Heck, I’m even down to learn VyOS from ground up to do it.
This has some good info.
Optimizing MTU and a few other factors can get you speed improvements.
Generally speaking, having a good Intel NIC and powerful CPU should yield good performance.
I’ve recently worked with a Xeon D-2123IT Supermicro Box and without any additional tweaking was able to get the full 1.5Gbps expected via PPPoE.
Welp. That means I’m gonna do some thinkering I guess. Some guys talked about this on a local forum after the announcement was made by the ISP about the new plan but they quiet down when the PPPOE issue appeared.
Do you recall what CPU usage have you seen while testing at that speed? I’m trying to figure out if I can use some “legacy” hardware aka DDR3 stuff that I have around.
Did it once. It adds complexity. I don’t like it.
To the people using bridges/virtual switches to do this: on Linux this is normally dependent on CPU. Thy loading a few popular torrents with hundreds of peers and watch the CPU spiking along with the number of open connections.