Hello All, I very recently got a job as the IT guy for a smallish company, they recently needed to expand into a second building in the complex, and now their satellite office is connected to the primary office’s network via a chain of ethernet cables and switches, which at one point exits and spans a field.
I am looking to use the fiber internet line from the satellite office to build a site-to-site vpn with openVPN,
I am at a bit of a loss however, the boss wants to use the fiber, doesn’t want to be strung out across the field, however I am trying to go for reliability and cost effective-ness. at the moment the best solution to this mess is either:
-Ubiquiti UDM pro and AP in the primary office, Edgerouter and ubiquiti AP in the satellite.
-Ubiquiti Edgerouters on both ends and some other APs.
I believe I understand how to configure the openVPN but I am struggling to find the best configuration of hardware to restore order to the mess I have found came with the job.
I would stay away from the UDM Pro, its a beta product at best. The interface is nice and shiny but is missing a ton of real low level features.
I have heard halfway decent things about the Edgerouter lineup, but there are rumors on the Ubiquity forums that they are trying to kill off that series of products so don’t buy if you plan on needing to support them.
As for fiber, just grab a few of these and stick the fiber in a trench and call it a day.
The fiber is already run into the building, and in the suite I have just regular ethernet, but the goal is to use the fiber that was paid for and installed.
I believe that a site to site vpn is the best solution, as the boss man wants to avoid running and burying more cables, when there is a way to avoid it,
but I will be insisting upon a more sane approach to wiring the next complex we build.
I would go with Pfsense on the gateway and wireguard for the tunnels. Unifi is probably fine to manage the switches and APs. Going all in on Unifi with their gateway, you can cause yourself some headaches. You run into situations where the config is broken but you need a functional network to change the config, so you end up having to tear down everything. With Pfsense (or anything you can manipulate directly), you inherently don’t have this problem. You can still lock yourself out of the switches and APs I guess, but if the controller is attached directly to the pfsense gateway (avoiding circular dependency between unifi controller and unifi switch), you should have a much better time if/when things break.
The future of the Ubiquiti Edgemax series is in question as they seem to be pivoting it to “UNMS” which is controller-based like Unifi.
To expand on this post, since wireguard isn’t included with pfSense, I would suggest getting something that doesn’t need to reboot and allows for updating once a month or every few months without issue like RHEL8.
You can set the routing settings in pfSense to point to the tunnel & RHEL is free for prod for upto 16 devices per account.
So the goal is to prevent someone from tapping into the fiber?
What throughput are you hoping for?
Reason I’m asking is precisely as @Novasty mentioned, if you’re expecting gigabit or faster - it’s hard to do, you need to burn a lot of CPU on both ends to get close.
As an alternative, there exist these $200-$400 rack mountable, redundant power supplies Mikrotik routers, that can do 200-300k packets per second of IPsec. (good enough to saturate a gigabit connection).
If you need 10Gbps IPsec, they also sell the CCR1036 for about $1300 a piece.
All in all - this is probably easier to support in the long run (and to get support it needed) than some custom built supermicro boxes.
Any chance wireless networking would work for you? Could get a pretty fast connection between the two buildings at very little cost if you can manage line-of-sight, such as putting up poles on the two rooftops with antennas mounted on them.
Long-term, digging a trench and installing conduit to run fiber through is the most reliable, fastest, and with lowest ongoing cost.
Try a linode VPN server in middle of 2 pfsense boxes.
Each site independant internet, wiregaurd to keep watch.
or fiber card for each pfsense, direct link???
If your boss wants to use the fibre look at running it properly. Get a quote and flick it to him. As in, use a company to do it properly.
If he’s willing to run it properly do it.
Wireless is more vulnerable to lightning, line of sight issues (someone parks a truck in front of it etc), lower bandwidth and you’re relying on vpn or other security devices to purchase, maintain, patch etc.
Float doing fibre properly first (contractor) and see if the cost is acceptable. Run at least a 12 core - the trenching etc is the expensive bit, the cable is relatively cheap per metre.
If it is, do that and move on. You won’t regret it. Multiple pairs for multiple 10 gig networks vs some shitty wireless link is a no brainer if your boss will stump for it. Speed wise, maintenance wise and reliability wise.
Also means you don’t need servers at both ends as the fibre will be fast enough that both buildings are as good as local to the servers, storage etc. those purchase and maintenance costs (patching, licensing etc) add up fast and scale up fast.
Alternatively with a multiple 10gig capable network from point to point you can incorporate it into DR plans and do backup over it effectively in case the hq building burns down.
Beats internet plus vpn at both locations too. Assuming the run is a few hundred metres or so?
I’ve been through this to replace wifi point to point with fibre. It was $70k to run (we had to do traffic management etc as it crossed a road in an airport campus ). But even so it saved a heap as per above. Long term easily paid for itself in savings and better user experience, less IT babysitting etc.
YMMV but do the sums and take all the above into account. Cheap up front isn’t the full story.
Even better get someone who does it for a living to handle it.
“IT GUY” doesn’t mean you have to DO everything yourself. Its more your job to make sure the jobs get done properly. Slight difference.
Engage pros to do this sort of thing (they will handle permits, OHS, etc and be insured) and spend your time doing things that you’re qualified to do.
Doing it yourself if you don’t do it for a living and know what you’re doing leaves you and the company open to all sorts of calamity. Using a contractor, neither you nor your employer are liable for whatever gets dug up.
Don’t go there.
At the very least get a proper pro quote first so your boss has an idea of the actual costs involved for a proper solution.
If that doesn’t fly then look at wireless or doing it cheap yourself whilst articulating the risks of doing so to your boss.
Doing it yourself and fucking it up looks way way worse if you didn’t do the above (get quotes for the fibre option, especially as the boss indicated preference for it - if your wifi link isn’t perfect you’ll never hear the end of it either) and articulate the risks of doing it in house first.
I just spent the better part of two months getting UniFi stuff to work for a small business. Similar situation to what you are describing. I wouldn’t recommend using UniFi/Ubiquity in a serious business application after my experience.
My work currently uses OpenVPN to connect 32 remote sites. Using Edgerouter ER-4 as the client devices.
I cannot recall any issues with OpenVPN itself in the the last 3 years. After power outages and loss of internet the tunnels come right back up. It has been pretty solid for us.
Yeah if you have the option for direct fibre though bandwidth will be the difference between no need for remote servers or standing up a bunch of local infrastructure depending on how the business operates.
Having a second site up the road via fibre is also a great DR option.
). But even so it saved a heap as per above. Long term easily paid for itself in savings and better user experience, less IT babysitting etc.