I am taking over the network for a hotel based out of a rather large condominium in the Philippines. The owner wants me to set us up for internet reselling to the condos and I would like to even setup some PtP CPE for so we can resell to near businesses as a WISP. Within the first year we could have several hundred customers. I have most pieces of the puzzle filled in, but I was hoping I could use our current PF-Sense firewall to handle the translation of our public non CGNAT IP to the Internal CGNAT 100.64.0.0/10 subnet. I really don’t want to purchase appliances like the A10 Thunder ect… Would PF-Sense be okay for this or does anyone know of better(Open Source) systems for this like VyOS, OPNSense, ect. I am really just looking for input, as I have actually not deployed a RFC6598 style networks before. Hoping someone here has more insight as to what might be the best performing system or has any recommendations for best practices or hardware/software.
CG-NAT is just NAT with a address range, no reason why pfSense wouldn’t work.
The more interesting part is:
- how you want your provisioning to work (radius or PPPoE or …)
- how much total bandwidth are you dealing with
- were you imagining a more ghetto setup with a single pfSense box to handle all this, or would you have more than one for fail-over and/or load balancing.
- in the meantime, what are you doing with IPv6
- what other gear is involved, switches and wifi things?
At the moment
- Q: how you want your provisioning to work (radius or PPPoE or …)
- A: Radius
- Q: how much total bandwidth are you dealing with
- A: In the long term lots, but right now we are going to be dishing out 40-100MB connections to customers. This is based in the Philippines. People here really don’t need that much bandwidth and only real options are Globe, Smart/PLDT and a lot pay 1000PHP+ each month for half decent internet. So our plan is to sell 500-100php plans at 40-100mbos We have around 3gbps right now and can increase that as needed. Most Filipinos are going to be happy being with that as long as it works consistently, and they save money.
- Q: were you imagining a more ghetto setup with a single pfSense box to handle all this, or would you have more than one for fail-over and/or load balancing.
- A: I will be using a variety of technologies but I would really like to keep PF-Sense for the load balancing, DHCP, routing, ect. I am considering taking some Poweredge servers and putting 10GB nics in there and doing a load balancing HA configuration with pfSense on those. Should be more than fast enough and reliable. I have other solutions for a lot of other things.
- Q: in the meantime, what are you doing with IPv6
- A: A work in progress.
- Q: what other gear is involved, switches and wifi things?
- A: Right now we are using Ubiquiti switches and I intend to keep it that way and just use Port Isolation and ACLs to make sure customer routers cannot intercommunicate on the same switch. I am still trying to decide what our options for CPE routers will be, but probably a combination of TP-Link and Ubiq
All that sounds reasonable, I’d do a proxmox cluster + pfSense in VMs - for reliability.
Also, 3Gbps of traffic isn’t that much and with a VM setup you can use the hardware to host other things, like that radius and DNS and whatever else you need.
I intend to use Proxmox for a lot of stuff related to this and have used it for years in other areas though I will for sure being leaving PFSense as physical dedicated boxs. I understand the rationale for visualizing them, though.
Do you have a legal requirement to maintain metadata? Many countries mandate this so that you end users can be identified based on source IP, source port and time. Part of CG-NAT involves making this data collection scalable, for example with port blocks (RFC 7422). For a smaller install like yours capturing Netflow of all the traffic may be an alternative.
Also just for clarity, how many public IPs are you working with here?
The only software / lower end routers that I know of which have some implementation / documentation on RFC 7422 are the following. Though you can likely set this up with a script in Vyos, FRR or Linux/NFTables/IPtables.
https://help.mikrotik.com/docs/display/ROS/NAT#NAT-CGNAT(NAT444)
https://github.com/alexk99/the_router
I am still reading into this, but based on what I have read thus far, the Philippines has no real laws on the books for “metadata” retention on customer traffic, but they do have laws stating an ISP cannot release or sell customer data of any form unless ordered by certain government agencies. I intend to keep a minimal amount of customer data.
I used to think less-data->less-hassle, but I’m not sure if it’s true except on e.g. Switzerland/Scandinavia and a few other places.
Both pfSense (freebsd) and vyos (linux) can provide you with a log of NAT state changes as just a basic plaintext feed you can stick into files yourself with not too much hassle.
The tools to help you get that feed are built on top of infra that’s generally used to sync connection states between two or more nat firewall instances (so that you can do a failover without breaking people’s TCP or UDP streams).