Awesome guide, I’m doing something similar with haproxy running on my opnsense router pointing to internal services.
I have avoided pointing to my Plex internally because I’m worried that video traffic will get routed through my haproxy on the router.
Is there any special configuration that allows me to use Plex.domain internal but not proxy all the video traffic?
Ok, I’ve been banging my head for a long time now.
Some information about my network:
From what I understand I need to get wireguard from pfsense to the cloud. From there do I install HAProxy on the same host where wireguard is, or I configure it on the pfsense box?
PS I know that I can forget about my problems with cloudflare tunnels but I don’t want to be that reliable on cloudflare
How do I do this?
Setting up wireguard is easy. Look at Wendell’s tutorial, he laid it down well. Install wireguard on Linode, get the wireguard package on pfSense and set them up as described.
I also think I was misunderstanding something too though. TBH, I don’t think I completely understand Wendell’s Guide here. So I set up HaProxy on both the Linode and on my PFSense box? Then I configure the HaProxy installation on PFSense to connect to my Linode over ssh? Then somehow proxy incoming connections back to the HaProxy instance that then sends it off to the PFSense HaProxy instance?
I’m not sure I understand what you said, but I’ll clarify how things should look.
pfSense <---- wireguard ----> Linode
Linode VPS has a public IP address, so you configure Wireguard on Linode to listen on a certain port and give the Peer (public) key for your pfSense on the VPS. Then on pfSense, you set to connect to the Peer endpoint VPS IP:port. Wireguard part is done.
For the HAProxy now. Suppose you have a website / web server behind pfSense, say you run nginx or apache or darkhttpd, doesn’t matter. Here things get a bit complicated.
Say you have 3 networks:
Back to the wireguard part for a bit, since I laid down the network. So, you would have your settings like this:
Wireguard on pfSense would look like:
Wireguard on VPS:
I may be doing something wrong above, keep in mind I’m doing this off the top of my head.
At this point, you have a working tunnel between the 2 sites. Now you need to set up HAProxy on Linode. On the HAProxy conf, you would point the server towards 192.168.0.20. pfSense in this case serves no purpose and you don’t need to set it as a reverse proxy, it’s just an additional overhead.
Well, you can set pfSense as a reverse proxy for your local network(s) behind pfsense, there is no harm in that, but you shouldn’t point linode to pfsense which points to the web server, that would be overly complicated. All you have to do in Linode is point the HAProxy listener (forward) to the web server behind pfsense.
I hope I explained it well enough.
I got another idea that I was thinking might be better as I went about trying to set this up. Instead of using HaProxy to tunnel traffic to my home’s PFSense router, why not just create a direct Wireguard tunnel from my VPS to home server? Here is my Linode’s Wireguard configuration:
[Interface]
Address = 10.0.0.1/30
ListenPort = 51820
PrivateKey = {SecretKey}
[Peer]
PublicKey = {HomeServerPublicKey}
AllowedIPs = 10.0.0.2/30
And for my home server’s configuration:
[Interface]
Address = 10.0.0.2/30
ListenPort = 51820
PrivateKey = {SecretKey}
[Peer]
PublicKey = {LinodePublicKey}
Endpoint = 104.237.130.46:51820
AllowedIPs = 10.0.0.1/30
So I am able to send ICMP Packets over the tunnel, but I tried to configure Nginx on the Linode VPS as a reverse proxy over the Wireguard tunnel and I keep failing. Perhaps I am doing something wrong in this regard. Here is the Linode’s Nginx Configuration file that I am using for testing:
1 │ # vim: filetype=nginx
2 │
3 │ # HTTPS PORT 443 CONFIGURATION
4 │ server {
5 │ server_name codedragon.dev www.codedragon.dev;
6 │ listen [::]:443 ssl http2;
7 │ listen 443 ssl http2;
8 │
9 │ location / {
10 │ proxy_pass http://10.0.0.2:8443;
11 │ proxy_redirect off;
12 │ proxy_set_header Host $http_host;
13 │ proxy_set_header X-Forwarded-Host $server_name;
14 │ proxy_set_header X-Real-IP $remote_addr;
15 │ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
16 │ proxy_set_header X-Forwarded-Proto $scheme;
17 │ proxy_set_header X-Forwarded-Protocol $scheme;
18 │ proxy_set_header X-Url_Scheme $scheme;
19 │
20 │ include /etc/nginx/headers.conf.d/0-security-headers.conf;
21 │ include /etc/nginx/headers.conf.d/1-codedragon.dev-headers.conf;
22 │ }
23 │
24 │ include /etc/nginx/ssl.conf.d/0-options-ssl.conf;
25 │ include /etc/nginx/ssl.conf.d/1-codedragon.dev-ssl.conf;
26 │
27 │ }
28 │
29 │
30 │ # HTTP PORT 80 CONFIGURATION
31 │ server {
32 │ if ($host = www.codedragon.dev) {
33 │ return 301 https://$host$request_uri;
34 │ }
35 │
36 │ if ($host = codedragon.dev) {
37 │ return 301 https://$host$request_uri;
38 │ }
39 │
40 │ server_name codedragon.dev www.codedragon.dev;
41 │ listen 80;
42 │ listen [::]:80;
43 │ return 404;
44 │ }
And the home server’s Nginx config:
1 │ # vim: filetype=nginx
2 │
3 │ # WIREGUARD PORT 51820 CONFIGURATION
4 │ server {
5 │ server_name codedragon.dev www.codedragon.dev;
6 │ listen 8443;
7 │ listen [::]:8443;
8 │
9 │ location / {
10 │ root /var/www/codedragon;
11 │ index index.html index.htm;
12 │ }
13 │
14 │ }
Remove your endpoint’s IP and port from the wg conf in your reply. Better to keep it secret. On the VPS, set allowed IPs to 10.0.0.2/32. Rest looks fine.
On the VPS, do a
curl http://10.0.0.2:8443
If that doesn’t work, it may be some of the weird nginx listening things. Try adding in /etc/hosts on Linode:
10.0.0.2 codedragon.dev www.codedragon.dev
Then try a
curl http://codedragon.dev
If that works, modify the nginx reverse proxy on Linode to codedragon.dev instead of IP address.
I am trying to do this between Linode and OPNsense, but with the following setup and am very lost.
[OPNsense—(HAproxy on OPNsense?)—seperate physical interface–wireguard client on pi—wireguard server on VPS— HAproxy unconfigured on VPS]
I have my WIreguard tunnel successfully setup and can communicate between the 2 peers.
I have CrowdSec setup on the VPS.
I am confused though about the following:
-Where am I terminating my SSL connections? in HAproxy with Let’sEncrypt with a wildcard cert?
-How am I directing traffic from the interface subnet on OPNsense to the services running in docker natively on my TrueNAS Scale box?
-Should I have a reverse proxy on OPNsense to direct traffic and if so, should I use the built in HAproxy or nginx?
-I have a domain with Cloudflare and want a wildcard cert
-I am confused on the firewall rules in addition to the reverse proxy to reverse proxy setup.
Any clarification on these is appreciated while I try to work through initial configs
As it turns out, you can install haproxy and acme client on your pfSense box and eliminate the need to do this in a dedicated VM.
I just got it setup and its pretty snazzy.
The only changes I did was that at my firewall I do not listen to 443 on WAN I listen to a random high port specifically coming from my VPS so that if someone does find out my IP they won’t see everything I’m running easily. Also to prevent ISP snooping.
I’ve been using HAProxy since 2016. It’s basically a OSI layer 7 router daemon IMHO. I think I just scratched the surface. Always look forward to hearing some interesting uses!
If the traffic between VPS and your router is not encrypted, your ISP will be able to see the traffic on that port and even read it. The probability of it happening is low, unless you are a target for surveillance. What random high port numbers do for you, is that script kiddies won’t try to randomly mess with your ports (like try known web server vulnerabilities on port 443, if it is ran on something like 9969, they typically won’t bother). Same port SSH port.
I’d still suggest doing a site-to-site VPN with your VPS. Problem / limitation with wireguard right now, as far as I can tell, is that, if you don’t have resolution to both directions and only use it one way like a classic openvpn setup, then it won’t attempt to reconnect if something changes (say, your VPS is the only one with static IP, you configure your router to connect to it, you reboot your VPS, your router now won’t connect back to it unless you restart the wireguard service - and I haven’t yet tested, but I think just a simple public IP shuffle on one end is enough to require a wg reconnect, I have to restart my wg quite often on my phone, but seems to not be as often affected on my router when using wg, who knows).
I do not de-encapsulate SSL from my VPS to my firewall, I pass the raw port 443 traffic over and then SSL terminate inside my network.
Traffic is encrypted 
i agree, that this is important because otherwise traffic would be sent in the clear but appear as secure in the browser. (which is why I think that little padlock is useless)
I’ve been wanting to do that but yeah I do not have a static IP at home, so I would need to use classic OVPN but the overhead on that is rough for streaming stuff.
Id use it for layer 4 on my VPS and layer 7 on my network.
I thought all HAProxy do is layer 7. Can you briefly describe one example of layer 4 usages?
I use it to forward ports 22, 80, 443 in TCP mode. When operating in TCP it does not deencapsulate the traffic so you can forward it anywhere. This is useful for load balencing traffic and sending it to a destination you know about at a different port port example.
When operating in http mode Haproxy will crack open the packets and look at stuff like request headers. This is useful for ACL where you need to figure out what traffic gets routed to what host via looking at the subdomain in the request.
I’ve using transparent TCP mode too. But never thought of using it other than HTTPS. I find your SSH use enlightening.
Is there any way that you are able to use Cloudflare DNS proxy with HAProxy and still have the traffic routed properly inside your home network?
Yes.
Also nothing to do with Cloudflare DNS proxy nor HAProxy I believe.
You just need to have Hairpin NAT setup properly in your router/firewall.
EDIT:
If that somehow doesn’t work by what you meant, you can re-define your domain name in your name server by pointing it to your local IP that runs HAProxy.
When I get a free moment, I will try this. Thanks for the info!