Should I learn Vyos/tnsr if basic linux is working well for me now? No problems with my current setup, I just occasionally get vyos/tnsr curious.
I run linux on four routers, five if you include my home. The largest location being 1000’ish devices, a half dozen vlans, a few software bridges, four LAN trunk ports and one WAN link. External links are handled by wiregaurd running inside containers on the router. The other typical router services all run through individual containers. DNS secondaries run on and off the physical router. Domain DNS is forwarded to the DC subdomain. Pretty basic stuff.
My router’s host OS basically just does routing and filtering. That bare metal host can all be rebuilt with about a dozen simple text file configs. Pretty easy stuff for any linux admin. Yet everybody seems to suggest vyos or tnsr.
I’ve read the highlights and spent a little time looking the configs over, but I don’t really get it. I guess that is why I haven’t spent the time (and future money) learning this stuff. I’m more of a linux admin who knows moderately complex networking. And I guess I just like my simple text files. Am I missing something?
Tnsr uses a vector packet processing architecture and is the future of how router processing is done as it is far more efficient than the old method of packet processing. It would be useful to learn, and is interesting to have such a workflow implemented in software mostly as Pfsense does with their tnsr version. Though on the OS side of things it really isnt much different than you are used to, as it is more the hardware and driver side that has the changes.
Vector Packet Processing is an order of magnitude faster than the way we have been processing packets in the past because it can skip a lot of copy processes. However, it does have more initial overhead so it is really meant for higher bandwidth connections rather than small bits of traffic.
The nice thing is a lot of newer router SoCs have VPP hardware blocks built into them, and can push things like 50+ gbps using only a few watts and only a single ARM core now days. But only if you have the software to use the hardware properly. Many vendors also include software for integrating things like Suricata inspection into a VPP workflow as well.
Who is sugesting vyos and why. The only thing it provides is an abstracted cli interface to set up firewall networking and vpn stuff. This is obviously more appealing to networking people that dont want to get into sysadmin, and is probably better for businness setups with hundreds of rules, ipsecs and whatnot
If your setup is simple enough to be a handful of configs, vyos is not worth it unless you are doing it for fun and learning
How do you implement high availability?
How do you implement WAN load balancing/failover?
Who validates your scripts are correct and do not introduce security flaws?
Where do you keep these scripts backed up ?
Do you update with apt-get update and how do you know that something won’t break because of an update?
Who knows how to manage your custom created router when you go on vacation?
Are you able to create an ISO that auto-delivers a functioning version of your setup?
How much does it take to find an equally linux savy person to resolve issues when you are not available?
If none of these potential answers make you think it would be worth exploring a solution that provides these functions for you and in a package that doesn’t need you to be responsible for making it work then no, you don’t want to ‘explore’ vyos
As for tnsr, can’t say, never found a use for it/lost interest since the pfsense guys decided to monetize the whole ecosystem (not complaining, it’s their IP and sweat and blood, just does not make sense for a home enthusiast anymore, and it is way out of the big names for the paid services they offer …)
let’s be real, I don’t think there are so many people trained in VyOS even in the actual industry for this to be a selling point. It’s already hard enough to find someone that knows Mikrotik CLI lol, and they sell a ton of devices (comparatively speaking)
I get what you are saying. The question of what happens if I get hit by a bus is a real one. They’d have to bring in a linux admin who could read a few systemd-netword files and one relatively elaborate (but well noted) nftables file. No scripts, just simple text files. Plus a few smaller obvious things that I have well noted. Oh, and they’d have to know about containers too. Pretty basic stuff, I think.
I guess I just take this further than anyone I know. Which has me wondering if I am being too self-reliant and should bite the bullet and pay for something more popular. Would my boss have better time finding a Vyos admin over a linux admin? I’ve always thought finding a linux admin would be easier but I can’t say I know anybody who wouldn’t have a little learning curve to get up to speed. But I don’t know any Vyos or tnsr guys either. Most guys just know the big name providers, basically all the simple big-name GUI platforms.
Burning my budget on something “simple” like this has always rubbed me the wrong way, but maybe I’m wrong. If my goal is to reduce the cost of replacing me maybe I should look to big name providers. That cost has to outweigh the savings we get by me just doing it myself. Vyos and other big name providers are expensive. In my defense, our router configs don’t really change that much and we have cold backups, so my company should have plenty of time to replace my setup before the hardware dies - or a security issue mandates an update. My attack surface is about as small as it gets, so this setup could coast for a long time with no urgent intervention.
TNSR totally bypasses Linux kernel networking stack. It builds the network stack from scratch using open-source projects and runs it user space (just like an application).
Usual network utilities are not compatible. You have to use its own utilities for configuration & administration..
Unless you’re working for data center kind of workloads, I wonder people ever want to touch TNSR.
***
VyOS is just Linux with a special CLI like Juniper for configuration & administration. Underneath, all just wrappers around usual utilities. The best selling point is one single config file determines all functionalities. If you have a router failed, it takes a few minutes to start a new one with the exact configurations.
People who like NixOS perhaps will be fond of VyOS. Personally I’ve passed that point and am happy with OpenWrt. lol.
That is kind of cool. Instead of pulling over a few config files I can pull over one. I think that would save me maybe 5 minutes rebuilding my host router OS, then another 15 or so pulling over the containers. Add another 5-10 minutes for miscellaneous work.
So this could save me around 30-60 minutes on a job that happens once every 3-7 years. That is pretty slick, but is it worth the license fees? Unfortunately, 30 minutes of my time is not as valuable (I wish it were). The next time I update my cold backup at the main office, I should time myself rebuilding from scratch.
you don’t need to be trained in vyos to operate it, you need to understand how a firewall/router that is configurable through a cli works, cisco/checkpoint/dell, whatever, the syntax will be different, the concepts will be the same
For a homelab, who. cares, run your scripts, run opnsense, run on a raspberry.
If you are running an appliance that makes a company lose money when they cannot work because you or someone else edited a file …
I am not saying you should do it, I am just making a point that in certain situations it is a risk going custom
I’m currently looking at setting up something like tnsr/vyos for performance reasons. Firewalling at 25gigabit is serious business and most products run into walls around 1, 3, 5, 10 gigabit depending. 25 is just not happening without very beefy hardware or VPP. Even the “enterprise fortress” thing from unifi drops to about 12 gigabit when you start doing things that involve actually looking at the packets.
Yeah no. The same concepts apply if you configure the services manually with their own config files. The syntax is different, the concepts are the same. The issue is the syntax.
Unless you have experience in that specific cli your ability to manage that appliance will be limited to what you can read from the docs or the integrated help.
A lot of companies dont let anybody touch their network gear unless they have vendor certifications for a reason.
Anybody can figure out a basic setup, but businness often have complex things that require more deep knowledge of the cli or web interface and the concepts used in its abstractions.
That is not my point. My point is that VyOS is going to be as alien as any other appliance if you are not trained in it.
While any other linux sysadmin can probably handle the same basic setup as he has.
That ‘probably’ in you phrase does not sit well with most companies that need the internet connectivity service for making money. That is the only point I wanted to try and make. I am not advocating vyos or cisco ios or whatever, what I am trying to say (poorly, at this point) is that once you move away from the single admin/single host, even if custom scripts are going to be easier for you than learning some standard cli, they will not be seen as an acceptable implementation.
Once you start dealing with security reviews/internal policies/risk assessments (and that is 99% of businesses that need a cyber insurance nowadays) you will find out that you are not allowed to deploy ‘your’ solution
For your homelab, your business,your startup, your one man only managed services company, you do you
People can compile from VyOS source code which doesn’t require buying a license.
For x86-64 devices compilation should be straight forward.
For other devices like Banana Pi BPI-R4, there is a community build shared on user forums.
Though any non x86-64 builds are as good as what Linux Kernel is baked in. The user space which is the meat of VyOS and what VyOS differentiates from other router distributions is not that important IMO.
Yeah, editing some files remotely carries some risk. This is one benefit for the traditional networking save/commit concepts that I don’t have. If I fubar a systemd-networkd or nftables file remotely and I’ve got problems.
Beyond those two areas though, this isn’t an issue. And those two areas almost never change. I could probably fall asleep on administering those two functions for a few years and nobody would notice.
I’ve always thought any linux admin who knows vim and can read well noted config files can replace me by noon. Maybe this isn’t the case. I am training my support guy to learn this basic stuff, but some of the concepts are probably still fuzzy for him. I just assume most linux admins are smarter than me and could quickly see what I’m doing. So maybe I am undervaluing the replace-me cost.
I didn’t read the entire thread here, but wanted to reply to the original topic/post as I have experience with all of this.
So, firstly TNSR and VyOS are different but similar, both are great, but I’d recommend both if you are just doing it for the learning experience.
Notably, VyOS DOES have Vector Packet Processing but it’s a bit of a pain to get setup and it’s more plain than TNSR’s implementation of it. TNSR on the other hand is VPP natively and also has the new VPF (Vector Packet Filter) which Netgate developed and is specific to TNSR, it’s much better than your normal ACLs.
Notably, the CTO of Netgate himself has said, and I quote, “VPF was developed with a new pfSense in mind”, so we don’t really know what this means, but chances are VPP and VPF are either coming to pfSense (maybe with pfSense moving to Linux, though VPP is available on BSD as well now) OR TNSR is becoming the new pfSense (I think this is more likely).
All in all though, TNSR is not open source the way pfSense is and you can’t get a license without paying anymore, it’s $1k a year minimum just to get it installed and play around with. I’m considering doing it on my Netgate 6100 though since pfSense is pretty slow on that hardware given my 8 gigabit WAN connection.
I’m nowhere close to hitting my hardware performance wall. Correct me if I’m wrong, but using modern hardware and efficient nftables rules (flowtable), then that performance wall is north of 50 Gbps.
I hear you on the time component. I am cautious about that but looking back I’ve never wasted my time learning to roll my own. Everybody seems to think that is dangerous or risky, but I just think everybody wants to have a company behind them. Rolling your own is an individual sport. You win or lose all by yourself.
Does your linux router’s packet filter (nftables) hit this kind of wall? Are you using flowtables rules in you nftables config? Of course, “it depends” is a qualifier at some level.
I haven’t gotten to play with network speed at 25 Gbps yet, but from what I have seen at my level and the stuff I’ve read about with higher end configs, it seems like this shouldn’t be a big deal.
