Due to multiple reasons, I cannot currently look into it. I don’t have hardware to test on and most likely not the knowledge and experience to do it. I think I’m close to biting more than I can chew, but given other circumstances, I think I could have pulled it off. I need to get my stuff together before I go into any more tech projects.
So, here’s the deal:
So the first issue seems interesting. It appears iptables snat works on the local subnet, but not on the remote subnet on the same router. I believe I recall a similar issue in a network I managed, but the remote network had to go through an additional hop (the VPN was a separate box from the router/firewall). To get around it, I used rinetd for some web services, but obviously, we don’t want that.
So, what can you do? Reproduce the issue Wendell described and try to identify a fix.
Then, there’s another thing you can help with if you are into BSD:
From what I know, FreeBSD already has mptcp working, but if you can set it up on a minimal OpenBSD or any other BSD, more power to you.
I never had to deal with mptcp (the protocol is pretty recent in IT years). Here’s some documentation I could gather:
I genuinely feel bad for accepting this and not doing it. But I have to deal with some other stuff for a while, until I can get back to a somewhat normal life. I won’t get into that here. Also, don’t ask, stuff on my end will get better eventually, I just need some time (maybe a few months).
socat and -j DNAT tcp port forwarding works fine for me as per below:
(I have forwarding sysctl enables and iptables default accept policy)
test setup utilizing network namespaces...
… that demonstrates DNAT port forwarding “just works” at least with socat.
I ended up opening a couple of split panes in tmux and pretended that each network namespace is a router/machine … it’s not exactly 1-1 but seems to work.
these below need “careful copy pasting” e.g. PS1 is a local bash variable, and you can’t create a route like this via unreachable ip.
# these all "require" root (or at least CAP_NET_ADMIN, for testing sudo -s is fine)
ip netns add rtr-isp
ip netns add rtr-1
ip netns add rtr-2
ip netns add host-on-rtr-2
ip link add l1a netns rtr-isp type veth peer l1b netns rtr-1
ip link add l2a netns rtr-1 type veth peer l2b netns rtr-2
ip link add l3a netns rtr-2 type veth peer l3b netns host-on-rtr-2
# rtr-isp
sudo ip netns exec rtr-isp bash -il
PS1="$(ip netns identify) # "
...
ip addr add 192.0.0.123 dev lo
ip addr add 1.2.3.1/24 dev l1a
ip link set dev lo up
ip link set dev l1a up
...
socat TCP:1.2.3.4:8080 stdio
# rtr-1
sudo ip netns exec rtr-1 bash -il
PS1="$(ip netns identify) # "
...
ip addr add 1.2.3.4/24 dev l1b
ip addr add 192.168.0.1/24 dev l2a
ip link set dev lo up
ip link set dev l1b up
ip link set dev l2a up
ip route add default via 1.2.3.1
ip route add 192.168.1.0/24 via 192.168.0.254
iptables -t nat -A POSTROUTING -t nat -o l1b -j MASQUERADE
iptables -t nat -A PREROUTING -i l1b -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.2
...
# rtr-2
sudo ip netns exec rtr-2 bash -il
PS1="$(ip netns identify) # "
...
ip addr add 192.168.0.254/24 dev l2b
ip addr add 192.168.1.1/24 dev l3a
ip link set dev lo up
ip link set dev l2b up
ip link set dev l3a up
ip route add default via 192.168.0.1
...
# host-on-rtr-2
sudo ip netns exec host-on-rtr-2 bash -il
PS1="$(ip netns identify) # "
...
ip addr add 192.168.1.2/24 dev l3b
ip link set dev l3b up
ip route add default via 192.168.1.1
ping -c 1 192.0.0.123
socat TCP-LISTEN:8080 stdio
...
ip netns delete rtr-isp
ip netns delete rtr-1
ip netns delete rtr-2
ip netns delete host-on-rtr-2
What exactly are you trying to forward?
… and what has it got to do with BSD and MPTCP ?
so … I guess the idea is to maybe run FreeBSD with a tap interfaces attached to rtr-2 ?
… and then maybe another FreeBSD or linux VM with another tap interface attached to rtr-isp