@Dynamic_Gravity - how do you manage to get a minecraft server (tcp) to work with mode http as shown in your example? I’m having trouble to get it to work.
I have tried but with no luck so far.
That backend is for Plan, which runs a webserver about the servers Minecraft metrics.
The Minecraft server itself should be behind a tcp LB but I had difficulty so I did SRV records instead.
Ah okay. Thanks for your reply. I will try setting up the server with a SRV record.
Would love a full walk-through on something like this. Just afraid I am going to break something bad!
Probably a little bit late to the party on this one.
the only way I can think of to get a minecraft server working with HAProxy, is to leave it until last and have it as a default backend.
have it so that all your ACL’s have to fail before it tries to connect to your minecraft backend.
You may even need a dedicated front-end for your minecraft, which checks all your ACL’s, forwards them through to your other front ends, and then if all else fails, connects to your minecraft backend.
1 Like
Hi @wendell, First I wanted to thank you. Lately you have been covering every
topic I have been working on / planning for: Workstation build, Home Server Series
Zettlekasten ( I started litterally reading about this topic 2 weeks before I
saw it on your YT channl), … I can say leaving
TS was the best thing you ever did 
Now back to the topic. I would say I’m a heavy tinkerer with home workstations
services and proxying. In the last years most of my services have been running
in the cloud and I started recently upgrading toward a full at-home hosting so
this thread was perfectly timed.
I am still in the process of the migration and trying to figure out the best
architecture. I will first describe my previous current architecture that served
me well in the last years, then the new one I am upgrading toward. Hoping this
writeup might help someone, it will probably intersect with the Home Server
Series thread.
Table Of Content
#1. Current Architecture
## V1 - Nginx Reverse Proxy + DOCKER
### DNSMASQ and Wireguard
## V2 - Caddy Proxy + Docker
## V2 - Exporting services through TOR
#2. Upgrade to home hosting
## Automated forward service proxying ?
1. Current Architecture
V1 - NGINX REVERSE PROXY + DOCKER
Simplified Diagram (see Example Scenario Below) :
Docker Services :: NG rev proxy :: Cloud Host(A) -----WG-----(WG PROXY VPN(B))----WG----- (OpenSense)| -> HOME
|dnsmasqs |dnsmasq | |dnsmasq \__ Laptops,phones ...
|______ secure machines
I was previously a heavy nginx user, especially the reverse proxy features. I
remember around 2012 using Nginx reverse proxy to automatically setup
subdomains to host git branches for testing. I also used to manually setup my
cloud services and proxy them through nginx. Then Docker appeared and I jump
on the wagon right away. Around the same time also appeared Wireguard, it
was the base to my current architecture which have been serving me for almost 6
years without a single major issue.
My setup allows me to have a safe access to my services from anywhere just by connecting to my VPN proxy(B). Almost everything runs on docker, and
all my machines including the cloud server are interconnected with wireguard.
I discovered later nginx-proxy for docker
and it turned proxying services to a whole new level. I could now provision
docker services and have them instantly and automatically proxied through nginx
by just adding the right labels to the containers. Everything is connected
through wireguard from my host server to my home and my phones/laptops and friends/family who join my VPN. I also run a speparate VPS as a pure VPN to avoid mixing my personal trafic and the rest of the trafic generated by the services.
DNSMASQ and Wireguard
I have multiple dnsmasq instances handling network translation between all my
machines as well as the docker containers that I select.
On Docker, I run a special docker image of dnsmasq
which automatically handles the network translation of my docker containers. So
I run one instance for each docker network that I use for proxying services.
Now the little magic is to name the networks the same way I name my DNS
suffixes.
There is an other dnsmasq instance on the host itself which does some extra forwarding with my VPN so that I can have my services reachable.
Then there is the VPN Host (B) which also runs dnsmasq. This allows me to have access to proxied services as soon as I join my VPN by just knowing the name of the
proxied service + the domain suffix.
The domain suffixes with dnsmasq and wireguard allow me to emulate subdomains
but for my private intranet over internet.
EXAMPLE SCNEARIO
Let’s use a calibre service for ebooks as an example. The service name is
calibre Let’s also say my DNS suffix for reaching my containers on docker is
wg.services. I create a docker network named wg.services and run the dnsmasq
instance over it. On my phone after I connect to my VPN, I can just type
calibre.wg.services topic access my ebook server.
V2 - Using caddy and caddy-docker-proxy
My next update was moving from nginx to caddy which
was much more minimal and flexible for me and comes with automated letencrypt certificates out of the box. It’s minimal size means I can run as many caddy instances as I need and I often use it to host static HTML pages as portals to my services. The
Caddyfile structure is very simple and can even be generated by code if needed.
The automated proxying is handled by caddy-docker-proxy. It has the same function as nginx-proxy and can handle any caddy directive that can be written in a Caddyfile.
Exporting services through TOR
As an extra backup option in case my VPN is down I started testing exporting
services through TOR using docker-tor-hidden-services.
Big advantage, I don’t need to worry about SSL or Wireguard. I can just setup
strong passwords just in case. Theoretically I should be the only one knowing
the onion address and the traffic is encrypted by default.
I also have portal to my services through TOR with just a few lines of Caddyfile.
2 Upgrading to home hosting
So I built this Threadripper workstation to host all my services and I started
thinking about a way to safely make them availble publicly without exposing my
IP. Since I have a lot of services that are TCP/UDP based and not only HTTP, I
need a good multipurpose proxy service.
Automated Forward Service Proxy
My goal is to automate the export of the services running home to my cloud VPS
proxy machine. To do that I need a reverse proxy solution that offers an API so
that I can add a script that updates the proxy when new services come up
or down.
So far I these are my options:
- Caddy Level 4 Module
- Nginx TCP/UDP proxy
- HAProxy: Just discovered it through this thread.
I am planning to do a progressive migration where I bring services slowly to my
home server while experimenting with different options. My choice will boil down
to the least long term effort solution.
2 Likes
Hey, I’ve recently had the good fortune to have the opportunity to have Google Fiber as my ISP (I live in Huntsville), so I want to try this; but the problem is that I need to deal with Dynamic DNS because they won’t sell me a static IP Address for some reason, and I’d like to use Wireguard as a direct connection between my Linode running HaProxy and my home server. I don’t really know how to do this given the whole DHCP problem on the server behind my network. I know how to set up DDNS on my PFSense box which serves as the primary Gateway for my network, but I don’t know how to do it from the Linux-powered box which is what I would prefer to do. Can anyone point me on how to get started?
Please note I do not need a DDNS provider as I use ClouDNS as my DNS provider and it supports DDNS.
Or better yet, could I use the Wireguard (or OpenVPN, but wireguard is preferred because imo it is easier to configure) connection to forward HAProxy traffic to the host behind my home network? There’s a few reasons why I don’t like DDNS tbh.
It’s fairly easy to set up a tunnel between you pfsense and your linode box. Set pfsense as you would set a normal client, connect it to the VPS, but for allowed IPs, only allow your home server’s IP to go through it (i.e. not all traffic, so avoid 0.0.0.0/0). After that, just configure your HAProxy domain, subdomain or simple URL to point to your internal IP address of your home server and you’re set.
You can use either Wendell’s or PLL’s guide on Wireguard setup.
Does this involve using DDNS? How do you connect PFSense to the VPS?
Awesome guide, I’m doing something similar with haproxy running on my opnsense router pointing to internal services.
I have avoided pointing to my Plex internally because I’m worried that video traffic will get routed through my haproxy on the router.
Is there any special configuration that allows me to use Plex.domain internal but not proxy all the video traffic?
Ok, I’ve been banging my head for a long time now.
Some information about my network:
- I have CGNAT.
- My router is pfsense. Its inside my network so I have my isp router before that but I can’t open ports and do anything on the router.
- For my cloud provider I’ve chosen oracle cloud because of the free tier.
From what I understand I need to get wireguard from pfsense to the cloud. From there do I install HAProxy on the same host where wireguard is, or I configure it on the pfsense box?
PS I know that I can forget about my problems with cloudflare tunnels but I don’t want to be that reliable on cloudflare
How do I do this?
Setting up wireguard is easy. Look at Wendell’s tutorial, he laid it down well. Install wireguard on Linode, get the wireguard package on pfSense and set them up as described.
I also think I was misunderstanding something too though. TBH, I don’t think I completely understand Wendell’s Guide here. So I set up HaProxy on both the Linode and on my PFSense box? Then I configure the HaProxy installation on PFSense to connect to my Linode over ssh? Then somehow proxy incoming connections back to the HaProxy instance that then sends it off to the PFSense HaProxy instance?
I’m not sure I understand what you said, but I’ll clarify how things should look.
pfSense <---- wireguard ----> Linode
Linode VPS has a public IP address, so you configure Wireguard on Linode to listen on a certain port and give the Peer (public) key for your pfSense on the VPS. Then on pfSense, you set to connect to the Peer endpoint VPS IP:port. Wireguard part is done.
For the HAProxy now. Suppose you have a website / web server behind pfSense, say you run nginx or apache or darkhttpd, doesn’t matter. Here things get a bit complicated.
Say you have 3 networks:
- 192.168.0.0/24 - pfsense and web server
- 10.20.30.0/24 - wireguard network, pfsense and linode
- 69.69.69.69/32 - your VPS public IP
Back to the wireguard part for a bit, since I laid down the network. So, you would have your settings like this:
- pfSense:
– eth0 = public IP or IP coming from ISP router, doesn’t matter
– eth1 = 192.168.0.1
– wg0 (probably named something else) = 10.20.30.2 - VPS:
– eth0 = 69.69.69.69
– wg0 = 10.20.30.1 - web server:
– 192.168.0.20
Wireguard on pfSense would look like:
- pfsense interface 10.20.30.2
- pfsense private key
- peer public key
- peer endpoint ip and port
- allowed IPs: 10.20.30.0/24
Wireguard on VPS:
- VPS interface 10.20.30.1
- VPS private key
- peer public key
- allowed IPs: 10.20.30.2/32, 192.168.0.0/24
I may be doing something wrong above, keep in mind I’m doing this off the top of my head.
At this point, you have a working tunnel between the 2 sites. Now you need to set up HAProxy on Linode. On the HAProxy conf, you would point the server towards 192.168.0.20. pfSense in this case serves no purpose and you don’t need to set it as a reverse proxy, it’s just an additional overhead.
Well, you can set pfSense as a reverse proxy for your local network(s) behind pfsense, there is no harm in that, but you shouldn’t point linode to pfsense which points to the web server, that would be overly complicated. All you have to do in Linode is point the HAProxy listener (forward) to the web server behind pfsense.
I hope I explained it well enough.
2 Likes
I got another idea that I was thinking might be better as I went about trying to set this up. Instead of using HaProxy to tunnel traffic to my home’s PFSense router, why not just create a direct Wireguard tunnel from my VPS to home server? Here is my Linode’s Wireguard configuration:
[Interface]
Address = 10.0.0.1/30
ListenPort = 51820
PrivateKey = {SecretKey}
[Peer]
PublicKey = {HomeServerPublicKey}
AllowedIPs = 10.0.0.2/30
And for my home server’s configuration:
[Interface]
Address = 10.0.0.2/30
ListenPort = 51820
PrivateKey = {SecretKey}
[Peer]
PublicKey = {LinodePublicKey}
Endpoint = 104.237.130.46:51820
AllowedIPs = 10.0.0.1/30
So I am able to send ICMP Packets over the tunnel, but I tried to configure Nginx on the Linode VPS as a reverse proxy over the Wireguard tunnel and I keep failing. Perhaps I am doing something wrong in this regard. Here is the Linode’s Nginx Configuration file that I am using for testing:
1 │ # vim: filetype=nginx
2 │
3 │ # HTTPS PORT 443 CONFIGURATION
4 │ server {
5 │ server_name codedragon.dev www.codedragon.dev;
6 │ listen [::]:443 ssl http2;
7 │ listen 443 ssl http2;
8 │
9 │ location / {
10 │ proxy_pass http://10.0.0.2:8443;
11 │ proxy_redirect off;
12 │ proxy_set_header Host $http_host;
13 │ proxy_set_header X-Forwarded-Host $server_name;
14 │ proxy_set_header X-Real-IP $remote_addr;
15 │ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
16 │ proxy_set_header X-Forwarded-Proto $scheme;
17 │ proxy_set_header X-Forwarded-Protocol $scheme;
18 │ proxy_set_header X-Url_Scheme $scheme;
19 │
20 │ include /etc/nginx/headers.conf.d/0-security-headers.conf;
21 │ include /etc/nginx/headers.conf.d/1-codedragon.dev-headers.conf;
22 │ }
23 │
24 │ include /etc/nginx/ssl.conf.d/0-options-ssl.conf;
25 │ include /etc/nginx/ssl.conf.d/1-codedragon.dev-ssl.conf;
26 │
27 │ }
28 │
29 │
30 │ # HTTP PORT 80 CONFIGURATION
31 │ server {
32 │ if ($host = www.codedragon.dev) {
33 │ return 301 https://$host$request_uri;
34 │ }
35 │
36 │ if ($host = codedragon.dev) {
37 │ return 301 https://$host$request_uri;
38 │ }
39 │
40 │ server_name codedragon.dev www.codedragon.dev;
41 │ listen 80;
42 │ listen [::]:80;
43 │ return 404;
44 │ }
And the home server’s Nginx config:
1 │ # vim: filetype=nginx
2 │
3 │ # WIREGUARD PORT 51820 CONFIGURATION
4 │ server {
5 │ server_name codedragon.dev www.codedragon.dev;
6 │ listen 8443;
7 │ listen [::]:8443;
8 │
9 │ location / {
10 │ root /var/www/codedragon;
11 │ index index.html index.htm;
12 │ }
13 │
14 │ }
Remove your endpoint’s IP and port from the wg conf in your reply. Better to keep it secret. On the VPS, set allowed IPs to 10.0.0.2/32. Rest looks fine.
On the VPS, do a
curl http://10.0.0.2:8443
If that doesn’t work, it may be some of the weird nginx listening things. Try adding in /etc/hosts on Linode:
10.0.0.2 codedragon.dev www.codedragon.dev
Then try a
curl http://codedragon.dev
If that works, modify the nginx reverse proxy on Linode to codedragon.dev instead of IP address.
I am trying to do this between Linode and OPNsense, but with the following setup and am very lost.
[OPNsense—(HAproxy on OPNsense?)—seperate physical interface–wireguard client on pi—wireguard server on VPS— HAproxy unconfigured on VPS]
I have my WIreguard tunnel successfully setup and can communicate between the 2 peers.
I have CrowdSec setup on the VPS.
I am confused though about the following:
-Where am I terminating my SSL connections? in HAproxy with Let’sEncrypt with a wildcard cert?
-How am I directing traffic from the interface subnet on OPNsense to the services running in docker natively on my TrueNAS Scale box?
-Should I have a reverse proxy on OPNsense to direct traffic and if so, should I use the built in HAproxy or nginx?
-I have a domain with Cloudflare and want a wildcard cert
-I am confused on the firewall rules in addition to the reverse proxy to reverse proxy setup.
Any clarification on these is appreciated while I try to work through initial configs
As it turns out, you can install haproxy and acme client on your pfSense box and eliminate the need to do this in a dedicated VM.
I just got it setup and its pretty snazzy.
The only changes I did was that at my firewall I do not listen to 443 on WAN I listen to a random high port specifically coming from my VPS so that if someone does find out my IP they won’t see everything I’m running easily. Also to prevent ISP snooping.
1 Like