In my case, I currently don’t have a switch/router sitting between the OPNsense instances and my WAN connection.
My question is: using a regular dumb switch between OPNsense and my WAN connection… can I use my 1 allocated static IP from my ISP as the VIP on the WAN side, and then 2 arbitrary public IPs (same subnet of course) for each WAN interface on the firewalls? My thought is that the fiber ONT from the ISP will only see the VIP (right?).
Or should I just toss a router there and do it as the diagram has it?
I would not bet on that, I have the same setup with Pfsense, my CARP-IP is 10.0.60.230 and master IP is 10.0.60.231.
We have both configurations, WAN01 has a router behind, hence only one public IP is needed and WAN02 has only a switch but three public IPs.
Here how a traceroute looks like
root@pve01:~# traceroute google.de
traceroute to google.de (172.217.18.99), 30 hops max, 60 byte packets
1 10.0.60.231 (10.0.60.231) 0.255 ms 0.650 ms 0.642 ms
2 192.168.100.1 (192.168.100.1) 0.479 ms 0.514 ms 0.502 ms
…
and nmap
Nmap scan report for 10.0.60.230
Host is up (0.00015s latency).
Not shown: 994 filtered ports
PORT STATE SERVICE
53/tcp open domain
80/tcp open http
2601/tcp open zebra
2604/tcp open ospfd
3551/tcp open apcupsd
8080/tcp open http-proxy
MAC Address: 00:00:5E:00:01:E6 (Icann, Iana Department)
Nmap scan report for pfSense-01.xxxx.intern (10.0.60.231)
Host is up (0.00016s latency).
Not shown: 994 filtered ports
PORT STATE SERVICE
53/tcp open domain
80/tcp open http
2601/tcp open zebra
2604/tcp open ospfd
3551/tcp open apcupsd
8080/tcp open http-proxy
MAC Address: 00:50:56:89:69:CD (VMware)
Nmap scan report for 10.0.60.232
Host is up (0.00017s latency).
Not shown: 997 filtered ports
PORT STATE SERVICE
53/tcp open domain
80/tcp open http
8080/tcp open http-proxy
MAC Address: 00:50:56:89:B8:FD (VMware)
Probably not.
You want to setup carp on the lan side, and enable wan only on the master node. You configure wan expecting a single ip, and use this script to manage fail over:
the problem with such a configuration is, your firewall is now HA, but you add another single point of failure with the switch, in addition, the firewall configuration is more complex, so the actual availability could be even worse than with only one Firewall.
If possible add another ISP and configure CARP on LAN and MultiWan for both nodes.
If you have incoming VPNs, there is a addon at least with PfSense, called “frr”, which adds OSPF, so you can automate routing via multiple ISPs/VPN tunnels.
yeah, i understand that. i’m doing this because opnsense lives on a VM host. and if i need to do something to that host, i have to take the internet down. this setup will be to allow me to keep the WAN connection up if i have to work on a VM host.
It’s doable, have done it, wrote wiki on it, using keepalived on Linux. Let me check CARP.
Ha, it’s in the manpage example: carp(4) , alongside tcpdump instructions.
What I don’t understand is whether that devd script is needed for host B from the example to keep the wan IP, if WAN interfaces are healthy.
…
I guess that’s what the tcpdump is for.
Ideally you have a static public /29 when using carp. However, you could try using link local addresses for the dedicated interfaces and set the public IP on the CARP interface. I believe this will result in the backup carp host having no internet connectivity though. It also may not work at all, I am not sure.
Assuming both your opnsense instances are virtualized, then you wouldn’t need an additional hardware switch or router. Just use the virtualized network on the hypervisor.
The OP has one ONT/Media converter coming from the ISP and two bare metal servers where he wants to deploy the VMs, so he needs a switch to connect both to the ISP …
