This guide will show you how you can route public IP addresses via WireGuard into your homelab, so that you can use a public ip on a VM directly without port-forwarding or 1:1 NAT. Another great use could be kubernetes and MetalLB, to expose real LoadBalancer adresses.
I am by no means a network engineer, so please correct me if there’s something that could be done better. I also have yet to work with vyOS firewall rules, so right now as it stands there is no sort of protection in place, but this will come soon when I’ve got the time to do so.
Prerequisites
You need one vyOS at Hetzner and one in your local network. I won’t go into detail on how to install vyOS and setting it up. But on Hetzner you can attach a vyOS image to an already created VM.
The Hetzner VM also needs at least one FloatingIP attached that we’ll be forwarding. The other public IP is needed to serve as a ssh and WireGuard endpoint.
Please follow my other guide to setup vyOS and WireGuard. When that is done and the peers can talk to each other you can come back.
My setup
(The 198.18.0.0/15 subnet is a RFC certified ip range used for testing, so that is perfect to not cause any routing conflicts)
Subnets:
10.0.0.0/24
198.18.0.0/30
198.18.1.0/24
1.1.1.1/32
2.2.2.2/32
+-----------------+ +------------------+ +----------+
|vyos-cloud-router| Internet |vyos-onprem-router| L2 |k8s-clustr|
|eth0: 1.1.1.1 |------------|eth0: 10.0.0.9 |------|Metallb |
| | |eth1: 198.18.1.1 | VLAN | |
|wg0: 198.18.0.1 | |wg0: 198.18.0.2 | |2.2.2.2 |
+-----------------+ +------------------+ +----------+
Step 1
Since metallb is able to announce ip addresses via bgp and vyOS can do the same, I chose to replicate the IPs between the two routers with IBGP.
To get started enter the config mode and execute the following where the local-asn is a number between 64512 and 65534 (It’s like a private IP used for bgp router identification) :
# vyos-onprem-router
set protocol bgp local-as 64514
set protocol bgp neighbor 198.18.0.1 remote-as 64515
set protocol bgp neighbor 198.18.0.1 description "BGP over WireGuard"
# vyos-cloud-router
set protocol bgp local-as 64515
set protocol bgp neighbor 198.18.0.2 remote-as 64514
set protocol bgp neighbor 198.18.0.2 description "BGP over WireGuard"
After commiting and saving these changes ( commit and save) the peers should start talking:
modzilla@vyos-onprem-router:~$ show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 198.18.0.2, local AS number 64514 vrf-id 0
BGP table version 1
RIB entries 1, using 192 bytes of memory
Peers 1, using 21 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
198.18.0.1 4 64515 135 135 0 0 0 02:11:48 0 1
Total number of neighbors 1
modzilla@vyos-onprem-router:~$
modzilla@vyos-cloud-router:~$ show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 198.18.0.1, local AS number 64515 vrf-id 0
BGP table version 1
RIB entries 1, using 192 bytes of memory
Peers 1, using 21 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
198.18.0.2 4 64514 136 139 0 0 0 02:12:52 1 1
Total number of neighbors 1
modzilla@vyos-cloud-router:~$
Step 2
To advertise subnets via BGP you need to have the route set on your router and add it to the bgp service.
set protocols static route 2.2.2.2/32 interface eth0 blacklist distance 254
set protocols bgp address-family ipv4-unicast network 3.3.3.3/32
After commiting and saving these changes ( commit and save ) the new subnet should be shared over bgp:
modzilla@vyos-cloud-router:~$ sh ip bgp
BGP table version is 1, local router ID is 198.18.0.1, vrf id 0
Default local pref 100, local AS 64515
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 3.3.3.3/32 198.18.0.2 0 0 64514 i
Displayed 1 routes and 1 total paths
modzilla@vyos-cloud-router:~$
Step 3
In order for the advertised IPs to be advertised with ARP on the local interface of the cloud router you’ll need to enable proxy arp for that interface, so that Linux allows advertising connected routes with arp.
# vyos-cloud-router
set interfaces ethernet eth0 ip enable-proxy-arp
Step 4
Now we need to add the cluster as peers.
for i in $i {2..6};do
set protocol bgp neighbor 198.18.1.$i remote-as 64513
set protocol bgp neighbor 198.18.1.$i description "k8s Cluster Member"
done
Step 5
Now you need to add your public ip address to metallb by editing the config in the metallb-system namespace.
kubectl edit cm -n metallb-system config
---
apiVersion: v1
data:
config: |
peers:
- peer-address: 198.18.1.1
peer-asn: 64514
my-asn: 64513
address-pools:
- name: default
protocol: bgp
addresses:
- 2.2.2.2/32
...
sh ip bgp summary should now also list all the k8s cluster-nodes.
Step 6
Now when creating a LoadBalancer service it will get the 2.2.2.2 address. Let’s try it out with an easy nginx pod. (Make sure you don’t already have a LoadBalancer-service stuck in the pending state ;))
kubectl get svc -A | grep LoadBalancer
kubectl create deploy --image=docker.io/library/nginx:alpine \
--replicas=3 --port=80 nginx-demo
kubectl expose deploy --type=LoadBalancer --port=80 \
--name nginx-demo nginx-demo
In a matter of seconds the IP should propagate to the router and can now be accessed from both routers.