TL;DR : I don’t know how to set up a productive network for my self-hosted kubernetes cluster with limited hardware. I’m thinking either biting the bullet and tinkering with some ARM SBCs, or reinstalling everything on a hypervisor (which ?) and booting up a PFSense VM.
I’m a junior devops with some professional experience as a sysadmin.
I love self-hosting and this weekend, I made some progress installing kubernetes on my server.
Got a bunch of things set up, argocd, cert-manager, etc.
And then I thought it’d be cool to have a couple more extra features, more on the networking side, like a VPN to access my home lab remotely, a DNS and DHCP server… nothing a reasonable firewall like PFSense cant do.
But that being said, I have been struggling to find a fitting solution.
My problem is, all the hardware I have at my disposal are my one bare-metal server, running ubuntu and kubernetes, as well as a couple raspberry PIs and an espressoBIN (an ARM-based router/firewall SBC).
The EspressoBin can, at best, run OpenWRT, or if I’m feeling very adventurous, I can try porting OPNSense to it (PFSense images for ARM are not available AFAIK). In both cases, there would be serious setting up and tinkering necessary, and it would be hardly reproductible and scalable, in stark contrast with how powerful and fluid my production flow with ArgoCD is.
The other option I came up with would be to install a hypervisor on my server and run kubernetes nodes on that. With that, I would keep my productiveness and scalability (and even improve the latter), and be able to boot up a PFSense (or OPNSense) VM. However, I then wonder about the hypervisor : which should I use ? I have some experience with Proxmox, and i’m not trying to do anything fancy so it should work just fine… but the Proxmox devs don’t seem willing to do any integration with Kubernetes on their platform. That might not end up being a problem, I just don’t know. That, and I’m not sure how I would get my storage working for optimal performance (I’ll get more into details about that if necessary). In my experience, there’s no way to keep a flowing experience for very long with Proxmox, but that might just be me.
I would rather always use FOSS, and another option i’ve come across is Citrix, with which I have no experience and have barely just heard of before. It apparently has decent Kubernetes integration, but I’m not sure to what extent that’s going to be relevant for me.
A third option would be to just put some money on the side, and get a proper router (either a small x86 SBC or an actual cheap router). Considering the primary use for this whole ordeal would be access for when i’m on the go, and how rarely I actually AM on the go, I’d be ok waiting to get a more productive solution.
I’d like to know if you have ideas for how I could set up a productive networking environment for my home cluster. If you know something I don’t, I’m all ears !
If there is some interest for how I did all this by the way, I might make a post about it : ).