In an effort to provide more detail on my infrastructure, I’m going to explain, in more detail, the sort of infrastructure I’ve got deployed so far.
All the Pis are running Ubuntu 20.04 ARM x64. SSH is keys only, and I’ve got my ingress unit (a pi 3) with Port 22 open to world.
I’ve got a wireguard server running on the pi3, providing access to the cluster and home network outside, this was necessary since I was building this out while away from home last month.
I have 3 Pi 4, 4GB models set up in a k3s cluster, with traefik and metallb configured as ingress and loadbalancers already. All that remains to do is configure the actual services.
Today marks the beginning of a second stage wherein I build out the rest of the infrastructure. Step 1 will be exposing a cluster dataset on the storage node via NFS. Since I don’t have high availability as an option for this cluster, I’m just going to pray it doesn’t fail in a spectacular way. The good news is that it’s RAIDZ2, so I’ll be able to tolerate two disk failures, but I’m not hoping for that in the slightest.
Once step 1 is complete, the next step will be deploying the Unifi controller. That’s got a lot of ports that need to be exposed and it’s also got a small internal mongodb database. So the challenge will be twofold, I must secure reliable and fast storage for the mongo process to connect to, NFS will have to do, since I don’t want to trust the MicroSD cards with too much. Additionally, I must export all these ports in a sane manner so the controller can connect to my plethora of hardware (3 APs, 3 switches and a gateway).
Step 3 is nextcloud. I have daily backups from my phone to nextcloud that are currently not completing. I hope to finish that off shortly. Nextcloud is a bigger challenge, since it requires a separate database instance to work with reasonable efficiency, and it has a number of server configuration options that need to be processed.
Step 4 is to set up the linode instance to handle proxying. I’ve got traefik configured as an ingress controller on my cluster, so if I set up an nginx server properly, I should be able to simply forward all the traffic to the ingress controller via a wireguard tunnel and it should be entirely transparent. The tricky part will be handling certs, so what I might do is just use TCP proxying, rather than HTTP/HTTPS proxying, so that Traefik can handle the certs, rather than nginx.
This is where I’d really like some advice. I’d like to use traefik for the ingress controller, but need a proxy since my ISP blocks port 80. Traefik will automatically handle LetsEncrypt certs for me, and configuring all the subdomain certs, so I’d like to continue using it for that. I suspect HAProxy might be an option here, along with nginx, but I don’t have experience with it.
