[SOLVED] Tailscale subnet router limited to only some devices?

Sawtaytoes · February 3, 2023, 10:13am

Solution: [SOLVED] Tailscale subnet router limited to only some devices? - #22 by Sawtaytoes

I want to set up a Tailscale subnet routed VPN at my parent’s house which targets only certain devices, so they can send all traffic through to my home network. These devices cannot install Tailscale clients.

By the time I implement this, they will have a UniFi router (same as I have), so there are gonna be some limitations.

I have no clue where to start.

Any device subnetted on the Tailscale network needs to change its gateway to the Tailscale device.
Somehow, this Tailscale host is either passing out IPs or routing between local IPs and the Tailscale network.
I might need VLANs to ensure DHCP works properly from those devices to the Tailscale device. The UniFi router can handle DHCP on VLANs and even assign a custom gateway (pretty sure).
To get VLANs working, I think I need managed switches. They have 1 managed switch, but they’d need more. Thankfully, I have a couple spare managed switches, but if I give them my spares, I won’t have any when I need them .
I’m not if if the Tailscale device would need 1 or 2 NICs. Can it be both on the VLAN and the main network’s subnet at the same time? That would simplify things.
I’d need to assign specific devices to the VPN or to a different subnet. This is actually really easy to do in UniFi.

I already have Tailscale running in TrueNAS both at my house and theirs, but neither is an Exit node, and both are setup to add a separate NIC to the host. As far as I understand, Exit node functionality isn’t available when adding a Tailscale NIC.

Potential Tailscale router hardware I have lying around:

Raspberry Pi 3 Model B. I have a PoE adapter as well as a USB Ethernet adapter.
SuperMicro X7SPA-H w/ RAM, a PSU, and a USB drive. The CPU is integrated. This has two 1Gb NICs.

If I can’t use TrueNAS’s Tailscale for this, my idea is to use one of these configurations with the Raspberry Pi or SuperMicro board, whichever makes the most sense:

pfSense
OpenWRT
Raspbian Lite (CLI-only)

With this hardware and the need to target only specific devices, how can I get this setup?

Lastly, I’m not familiar with Tailscale enough to know how devices on my parents’ network can use the Internet as if their public IP is at my house. Do I also need a Tailscale exit node at my house?

2FA · February 3, 2023, 4:23pm

You’re making this overly complicated, just setup policy based routing on the Unifi router so that specific devices default route over the tunnel instead of default WAN gateway. No need to change stuff on the client device.

Sawtaytoes · February 3, 2023, 8:55pm

Is one of my machines acting as a gateway?

And then I point those machines to the local IP of that box or the Tailscale IP?

risk · February 3, 2023, 8:58pm

Yes, a subnet router at your parents house to connect your parents lan to your tailnet (consisting of your Tailscale running hosts); and an exit node at your place to let other connected things use this exit nodes physical network for connecting to the internet.

Exit node counts as if it were a subnet router for billing purposes, so subnet router + exit node might not fit inside the free plan (allows 1 subnet router).

You could nat or run double wireguard, … or … similar to avoid paying

BTW what’s your use case exactly? Do you need these machines at your parents place accessing your network or parents network (alongside them accessing the internet through your home ISP?)

Sawtaytoes · February 3, 2023, 9:40pm

Exit Node is correct. Was playing around with that last night.

In my TrueNAS Docker situation, I actually need to spin up another container to get the Exit Node working since it’s either host or exit node.

I currently have my machine enabled as an exit node, and it hasn’t ticked-up the subnet router limit:

I’m not quite sure how to make a subnet router, but it looks like I need to connect to an exit node from my parents’ NAS.

On their Docker container, I can do --exit-node=NAME_OF_EXIT_NODE --exit-node-allow-lan-access.

Right now, I wanna see if I can outsmart Netflix’s new 1-home account restriction for fun. My folks are the account owner, but my upload speed is faster, so I’ve gone about it this way.

If I can restrict this down to just Netflix, that would be even better, but it’s a level above “how do I get started?” .

In actuality, I wanna be able to access machines in their network for tech support and make it so they can access SMB on my NAS from their place.

risk · February 4, 2023, 6:12am

--exit-node 100.a.b.c makes the client use one of the existing exit nodes.

--advertise-routes (doc) makes a subnet router.

I assume the subnets are different in your home vs theirs? No idea how things would work if it weren’t (might need nat).

How are you running Tailscale on their NAS? VM bridged to local network or k3s? (it’s helpful for a subnet router to have access to the subnet it’s advertising, but I’m not sure otoh how you’d do this in k3s).

Sawtaytoes · February 4, 2023, 7:04am

This works because I have a NAS at their house for offsite backups. Both NASs run TrueNAS, so all I have to do is load up some Docker images for Tailscale to work.

I got 2 exit nodes setup; one here and one there. I was able to successfully connect through them on my desktop and phone.

I also advertised routes on their NAS, and even their local DNS entries worked at my house when I connected to their exit node.

Looks like exit nodes and subnets are treated differently.

I wasn’t able to get this working yet in TrueNAS:

--exit-node=x.x.x.x --exit-node-allow-lan-access

Something I don’t understand is the --exit-node-allow-lan-access arg.

--exit-node says "connect to this exit node as a client.

What does --exit-node-allow-lan-access do? Does it setup a reverse subnet lookup for the local LAN, so you don’t need to manually advertise routes?

risk · February 4, 2023, 7:09am

It makes your client able to access its own nearby local lan, while using a remote exit node. Without it, your client loses access to nearby machines and acts more like it’s actually physically there.

Sawtaytoes · February 4, 2023, 7:26am

So that last piece is the kicker then.

Now that I have an exit node setup at my house, I need to have one of their NAS’s Tailscale instances connect to my exit node.

Once that’s done, how are machines in their subnet able to go through the Tailscale network?

I need to set the gateway on those devices to their NAS, but then what? How do I get that NAS to become a gateway; sending traffic over Tailscale?

risk · February 4, 2023, 4:00pm

What kind of network access does Tailscale have on their nas? Are you running a VM with a network bridged to LAN … or something else? How are you running Tailscale there?

Can you configure IP addresses manually on those devices, and set the gateway to the Tailscale LAN IP?

2FA · February 4, 2023, 11:26pm

The gateway is the router IP address on that subnet. You can create a policy based route on the router that equates to “if packet is from specific address, set default route to tailscale subnet router, else leave default route as ISP router/uplink.”

Sawtaytoes · February 5, 2023, 7:30am

That custom routing rule makes sense; although, Tailscale’s router isn’t on their subnet, it’s in the cloud and not publically accessible. Only machines with the Tailscale client can connect to the Tailscale router even when using subnet routing.

Tailscale subnet routing doesn’t change any IPs. All it does is let Tailscale clients access machines on another network. It does not let those machines access Tailscale clients (from my understanding).

If I do this:

set default route to tailscale subnet router

Then those machines will go nowhere as they don’t know where that network is located. It resides on a virtual interface on the NAS itself.

I could tell the router “hey, the gateway for these devices should be the NAS”. “The NAS” being a machine on their network that’s sending all traffic through to my network over Tailscale.

As soon as I do that, the NAS will receive packets saying “here’s my Internet traffic”. Then it will drop all those packets because it has no clue what to do with them. Unless I configure some sort of route table which says “all Internet/Gateway traffic on LAN needs to go to the Tailscale interface”, I can’t see a way for it to know what to do with those packets.

And I wouldn’t know how to set it up so any local SMB traffic would still work and only gateway traffic would go to Tailscale.

Sawtaytoes · February 5, 2023, 7:40am

TrueNAS is the NAS provider. Tailscale is installed via a Docker container on Kubernetes. There’s no Tailscale NAS.

For this setup, there is a tailscale0 interface on the host even though Tailscale is in a Docker container. But this is a virtual interface. There’s no physical interface for machines to point to.

The tailscale0 network is not bridged with the LAN network. This sounds like a decent solution, but wouldn’t that mean both networks are merged together, and they will have DHCP server conflicts? Also, I don’t think Tailscale gives out IP addresses willy-nilly. Those hosts have to be authenticated.

So in this case, I’d need the NAS in their network to act as a NAT router for non-Tailscale devices.

From what @2FA says, UniFi will let me change their gateway to the NAS’s IP. I haven’t yet tested this because I still don’t have a way to route inbound packages to the virtual Tailscale interface.

Sawtaytoes · February 5, 2023, 7:56am

Tonight, I tried OpenWRT w/ Tailscale on a spare Raspberry Pi and even set up a pfSense server on a spare SuperMicro server board, but I’ve run into multiple issues in both cases.

Adding the interface

If I add the tailscale0 interface to OpenWRT, then it completely kills the Tailscale connection:

I’ve tried both “Unmanaged” and “DHCPv6 Client”.

Narrowing it down, it seems like clicking “Configure” on the Devices screen is the cause:

I haven’t got it configured right now, but clicking “Configure” happens when you create an OpenWRT interface. I think OpenWRT tries to claim the interface and the Tailscale app loses access.

Surprisingly, the same thing happens in pfSense:

When adding the interface, Tailscale loses access.

Route mappings

While it didn’t add these mappings automatically as it should’ve, I can setup Tailscale mappings for outbound traffic in pfSense:

OpenWRT won’t let me because it only lets you route between OpenWRT interfaces, not physical interfaces.

Connecting to an exit node

I tried doing --exit-node=REMOTE_EXIT_NODE_NAME in pfSense, and it took it, but nothing appears to work. When trying to grab my IP in the CLI, it’s the local public IP, not the exit node’s public IP.

pfSense seems to expect you only want to use the router as an exit node; not that you want to connect your entire network to another one for outbound traffic.

When running the same --exit-node=REMOTE_EXIT_NODE_NAME command in OpenWRT, it actually shows the exit node’s public IP! But since I can’t route any traffic, this is pretty much useless.

At this point, it makes me wanna go back to Raspbian and try to mess with iptables manually (yuck). I’m not skilled enough to do any of that though.

I haven’t configured any routing from LAN to Tailscale just yet, but I really need to figure out the --exit-node situation in pfSense before I do that.

If I could get routing in OpenWRT to use the virtual interface rather than the OpenWRT interface, I’d be set!

Sawtaytoes · February 5, 2023, 8:20am

I was unable to ping the pfSense box from any machine on the Tailscale network.

This firewall rule fixed that:

I’m locking this down, but it was a firewall rule causing the issue.

Tailscale isn’t a first-class interface in pfSense, but it does work well as a second-class interface unlike OpenWRT.

Sawtaytoes · February 5, 2023, 9:18am

After messing with a bunch of NAT and other settings and failing to see any change, I thought maybe I need to create a custom Gateway.

pfSense does not let Tailscale be the gateway:

This may or may not matter. This is what I mean about Tailscale being a second-class interface in pfSense. It’s not usable everywhere.

I need to say “any LAN traffic is sent to Tailscale”. Then behind the scenes, Tailscale is sending NATing that traffic on WAN (which could technically be the same interface as LAN).

risk · February 5, 2023, 12:45pm

There’s no routing in the cloud, tailscaled is a router and your big virtual conceptual router is just a bunch of tailscaled being coordinated in a way that might make things look like you have this magical thing.

IIRC, by default it does allow such access, unless you setup ACLs in your tailscale config (ACLs are distributed to each Tailscale client which will then enforce them on either side).

For example,

… let’s say you have a Plex server that also just happens to run Tailscale and thus you have a plex server on 100.100.30.40 …

…if your parents network is 192.168.17.0/24 … and you have a Raspberry pi on 192.168.17.17 running tailscaled as a subnet router advertising 192.168.17.0/24…

… and then you have your parents TV on 192.168.17.38 running with a 192.168.17.17 as a gateway

Your parents TV on 192.168.17.38 will send all packets not for any of the other hosts on the local lan to 192.168.17.17 (the pi) and it will take them off of eth0 and send them into tailscale0, and tailscaled will grab them, encrypt them and send them over to where they’re supposed to go ie. to your Plex server without any kind of nat…
… this all assumes there’s not OS level firewalls on the TV and on the pi and assumes that “forwarding” is enabled on the raspberry pi which is what makes it not drop packets it receives destined for other networks it’s connected to.

tailscaled will create the tailscale0 tun interface and will setup the routing rules to make it possible to send stuff over.

On Plex host that just so happens to also be running tailscaled, tailscaled will install a route to 192.168.17.0/24 via tailscale0 which will let IP packets flow back to your parents TV.

If your parents TV sends a packet destined for 8.8.8.8 to pi on 192.168.17.17 … the routing rules on 192.168.17.17 will not send it through tailscale0, but will instead go to the home router your parents have running on 192.168.17.1 example, and once 8.8.8.8 replies, those replies with a src address 8.8.8.8 will go to your parents TV probably without passing through the pi (or they might pass through the pi, this all depends on how the ARP tables on the home router get updated, this part of ipv4 is a bit arbitrary), replies should get back to the TV either way.

Things get a lot more interesting once you add tailscale exit nodes … and you add VM bridges and firewalls and kubernetes in TrueNAS and docker networking and OpenWRT and pfSense which all have firewalls and try to manage routing rules and IP addresses and so on… you end up with way too many things.

… you kind of need to be patient and debug all the things one by one.

Sawtaytoes · February 5, 2023, 8:14pm

Thanks! I read your scenario and actually tried it last night, but Internet packets were dropped, not routed to Tailscale or my router.

I’m avoiding the NAS to make this simpler. I tried this in OpenWRT, connecting to Tailscale and then having my PC pointed at its Gateway. Just like this:

Your parents TV on 192.168.17.38 will send all packets not for any of the other hosts on the local lan to 192.168.17.17 (the pi)

That didn’t work even though packets in OpenWRT were being sent to Tailscale.

Today, I’m gonna try using Raspbian instead. It won’t have a separate firewall to deal with.

From what I’ve been reading, for incoming packets to redirect from one interface to another, you need to configure iptables. If eth0 gets packets it doesn’t know about, it will drop them by default.

And you’re saying Tailscale does that for me?

Sawtaytoes · February 5, 2023, 10:09pm

This is what’s in iptables just after connecting to Tailscale.

$ sudo iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ts-input   all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ts-forward  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain ts-forward (1 references)
target     prot opt source               destination
MARK       all  --  anywhere             anywhere             MARK xset 0x40000/0xff0000
ACCEPT     all  --  anywhere             anywhere             mark match 0x40000/0xff0000
DROP       all  --  100.65.0.0/10        anywhere
ACCEPT     all  --  anywhere             anywhere

Chain ts-input (1 references)
target     prot opt source               destination
ACCEPT     all  --  router.my-subnet.ts.net  anywhere
RETURN     all  --  100.116.92.0/23      anywhere
DROP       all  --  100.65.0.0/10        anywhere

Changing my PC’s gateway to this Raspberry Pi on Raspbian, all Internet traffic is dropped.

It’s not automatically routed even without connecting to an exit node.

If I do connect to an exit node, none of these iptables entries changes. What am I missing? Something should’ve changed, or there’d be no way for all my local packets to go through Tailscale.

Sawtaytoes · February 6, 2023, 12:04am

I got it working finally on Raspbian. I had to run this iptables command:

sudo iptables -t nat -A POSTROUTING -o tailscale0 -j MASQUERADE

This creates a NAT routing of anything going to tailscale0.

A few things, I don’t know how to save this. There’s iptables-persistent, but I’m not sure if there’s a proper way to save custom iptables changes and make sure they get loaded at boot.

I was wrong about IP tables. When you “list”, that doesn’t list everything. Here’s the nat table after changing it:

  $ iptables -t nat -L
  Chain PREROUTING (policy ACCEPT)
  target     prot opt source               destination

  Chain INPUT (policy ACCEPT)
  target     prot opt source               destination

  Chain OUTPUT (policy ACCEPT)
  target     prot opt source               destination

  Chain POSTROUTING (policy ACCEPT)
  target     prot opt source               destination
  ts-postrouting  all  --  anywhere             anywhere
+ MASQUERADE  all  --  anywhere             anywhere

  Chain ts-postrouting (1 references)
  target     prot opt source               destination
  MASQUERADE  all  --  anywhere             anywhere             mark match 0x40000/0xff0000

I still need to try this on a device at their place, but on my computer to the Raspberry Pi running Raspbian, it works great!

From what @risk is saying, I may not need iptables at all because it’s possible Tailscale subnet routing is already doing this for me. But because of Docker 'n such, we’re not sure what’s actually going on there.

Next Steps

I wanna try this on the NAS to see if I don’t need to add yet another machine to their network.

I still need to figure out a few more things:

I don’t want my NAS to forward any packets through Tailscale, only other clients using it as a gateway.
I want packets going to my NAS, rather than the Internet to not go through Tailscale.

Not sure if this NAT routing can be configured to exclude the NAS itself.