Wireguard and podman

Veeh · January 22, 2023, 6:23pm

Hi,

I’m wondering if anyone has experience with wireguard running on podman, could help me out.
It was so easy to run on docker, I was not expecting this to be so difficult with podman.
The whole experiences has been weird. I had it running multiple time and then for some reason it stopped ! I guess I’m missing something on the networking side but I can’t figure it out what.

I have forward and masquerade enable on the host NIC.
The pod is running with:
–sysctl=“net.ipv4.conf.all.src_valid_mark=1”
–sysctl=“net.ipv4.conf.all.forwarding=1”

I also enable firewalld logs. But I’m not getting anything.

If you have any advises, I’m all hears.
Thanks

risk · January 22, 2023, 6:44pm

There’s probably someone around who is running it.

As a generic troubleshooting advice:

can you post which wireguard container setup are you working with and what you looked at so far
are there any signs of anything failing in logs, anything you don’t understand or that looks suspicious?
what kind of troubleshooting have you done so far? tcpdump inside container/outside on
how does routing and firewalling look like inside/outside the container?

Veeh · January 22, 2023, 7:08pm

Sure
I’m using linuxserver container and I followed Pro Custodibus doc to deploy it.

I have been using linuxserver wireguard container for quite some time now with Docker. That’s why I kept the same.

Failing logs are something else. The container run without any issue.

.:53
CoreDNS-1.10.0
linux/amd64, go1.19.1, 596a9f9
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.13.13.1 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.13.13.2/32 dev wg0
[#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE
[ls.io-init] done.

I enabled firewalld “denied” logs but did not get any. I might have missed something there. Because I seems weird that there is not way to enable debug log.

I did run tcpdump outside the container to be sure about the NIC. Because podman create interfaces like docker basically but it is not using them. Everything go through ens18.
I did not run tcpdump inside the container though.

This is my running conf:

ExecStart=/usr/bin/podman run \
#       --privileged \
        --cidfile=%t/%n.ctr-id \
        --cgroups=no-conmon \
        --rm \
        --sdnotify=conmon \
        --replace \
        --cap-add="NET_ADMIN" \
        --cap-add="NET_RAW" \
        --detach \
        --name="wg-full" \
        --publish="30021:51820/udp" \
#       --network="nwk-wg-full" \
        --env="PUID=1112" \
        --env="PGID=1112" \
        --env="TZ=America/Winnipeg" \
        --env="SERVERPORT=30021" \
        --env="PEERS=vincent" \
        --env="PEERDNS=auto" \
#       --env="INTERNAL_SUBNET=10.15.15.0" \
        --env="ALLOWEDIPS=0.0.0.0/0" \
        --env="LOG_CONFS=true" \
        --volume="wg_full_config:/config:Z" \
        --sysctl="net.ipv4.conf.all.src_valid_mark=1" \
        --sysctl="net.ipv4.conf.all.forwarding=1" \
        --sysctl="net.ipv4.ip_forward=1" \
#       --sysctl="net.ipv4.ip_forward_update_priority=1" \
#       --sysctl="net.ipv4.ip_forward_use_pmtu=0" \
#       --sysctl="net.ipv4.conf.all.bc_forwarding=1" \
#       --sysctl="net.ipv4.conf.all.mc_forwarding=1" \
#       --sysctl="net.ipv4.conf.all.accept_redirects=1" \
#       --sysctl="net.ipv4.conf.all.accept_local=1" \
        docker.io/linuxserver/wireguard:latest

I initially used a custom network. But disabled it because it is not working and looks like I have something stable with the default container lan.

Firewalld config:

public (active)
  target: default
  icmp-block-inversion: no
  interfaces: ens18
  sources:
  services: cockpit ssh zabbix-agent
  ports: 30000-32767/tcp 30020/udp
  protocols:
  forward: yes
  masquerade: yes
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

Additional rule:

root@dokfarm:~# firewall-cmd --direct --get-rules ipv4 filter FORWARD
0 -i ens18 -o ens18 -j ACCEPT
0 -i ens18 -o ens18 -m state --state RELATED,ESTABLISHED -j ACCEPT

Veeh · January 22, 2023, 9:37pm

My troubleshooting process was as follow:

I want to use the container as a tunnel VPN to access my home network.
So I started with this:

sudo podman run \
    --cap-add NET_ADMIN \
    --cap-add NET_RAW \
    --name wg-full \
    --detach \
    --network nwk-wg-full \
    --publish="30021:51820/udp" \
    --env="PUID=1112" \
    --env="PGID=1112" \
    --env="TZ=America/Winnipeg" \
    --env="SERVERPORT=30021" \
    --env="PEERS=vincent" \
    --rm \
    --volume wg_full_config:/config:Z \
    --sysctl="net.ipv4.conf.all.src_valid_mark=1" \
    docker.io/linuxserver/wireguard:latest

Then from one google search to the other, I added more options to the conf until what I have now.
Right now I’m running the container as root. I don’t think it’s possible to run rootless because of the network component. But please let me know if otherwise. (I tried but could not wake it work)
And like I said in my first post using my custom network setting the container lan subnet is not working.
I noticed that the container create an interface ’ eth0@if124’.
When using the default network subnet 10.13.13.0/24 this interface use this IP: 10.88.0.50/16
Not the same network.

When I use a custom network, in my case 10.15.15.0/24 gateway 10.15.15.1.
This interface instead of having an IP in 10.88.0.0/16 it’s getting an IP from the container subnet. 10.15.15.0/24 (Is it why the client cannot connect when using something else than the default network?)

At the beginning, I created another zone to set the masquerade and forward, With the podman NIC. But it does not make a difference everything goes through ens18.
The other thing I cannot make sense of are the firewall-cmd forward rule. I can take them off and reload firewald and the connection remains.

I’m basically trying to make sense of all this. I added a lot of stuff and I would like to know what basically is necessary.

Veeh · January 28, 2023, 5:01pm

Hi,

So in case this might interest someone.
Podman and wire guard is working fine.

The container needs to run as root other wise the network forwarding will not happen. I’ll keep digging with the network + rootless container, I might have miss something.
Anyway.

For a full tunnel (0.0.0.0/0), you need:

        --cap-add="NET_ADMIN" \
        --cap-add="NET_RAW" \
        --sysctl="net.ipv4.conf.all.src_valid_mark=1" \
        --sysctl="net.ipv4.conf.all.forwarding=1" \
        --sysctl="net.ipv4.ip_forward=1" \

For split tunnelling you need only this:

        --cap-add="NET_ADMIN" \
        --sysctl="net.ipv4.conf.all.forwarding=1" \

If you want DNS resolution make sure to allow the container IP (default PEERSDNS=auto point at the container IP, but you can also specify your DNS IP and pass it in the allowed IPs).

For both cases, if you want to use a custom network keep in mind that the container will use the first IP of your network, so don’t assign it to the gateway.

Finally the firewall rule on the host NIC.

firewall-cmd --permanent --add-masquerade
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i ens18 -o ens18 -j ACCEPT
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i ens18 -o ens18 -m state --state RELATED,ESTABLISHED -j ACCEPT

The default network that comes with podman is bridged to the host NIC. There might be a way to get a more secure setup with a different option and a specific zone. So that you don’t apply the rule to the default public zone.
So far I have been unsuccessful. I’ll update the post if I find something.