Strange issue with Github when routing traffic over Wireguard VPN

Hardware Hub Networking
1

Setup is as follows
ISP: Starlink (CGNAT)
Router: pfSense
VPS at Linode with Wireguard

There is a WG tunnel setup between pfSense and my linode. On pfSense, the WG Tunnel has been setup as an interface and a gateway.

I use the VPS to acess all of my external services behind cgnat and that works perfectly. I have also hosted a PBX behind cgnat using the same setup with no dramas.

I have policy based routing where all of the Traffic from a certain IP, in my case, my Unraid server goes out over the WG Tunnel. From the surface everything looks A ok and everything is functioning as it should. Where it comes unstuck is when my Unraid box tries to access anything hosted at a github domain / server.
Which in my case, is everytime i click on the community applications tab, or go to the docker tab or the Dashboard tab (i think its all the docker icons it downloads)

I can replicate it just by running
wget https://raw.githubusercontent/pathtogithubfile
it gets hung up like this and wont progress

image
image1096×150 7.92 KB

If i attempt a donwload from any other source, it works fine. like this

image
image1053×216 9.83 KB

I ONLY see this behaviour from github, githubusercontent and i don’t understand why.

I can run the same wget command directly on my linode instance and it downloads the file fine.

image
image1402×238 10.8 KB

If i turn off my policy based routing and just send my unraid box straight out my starlink wan, the same wget command donwloads the file fine.

image
image1087×273 13.6 KB

To make things even more strange i have another WG tunnel setup to another site and if i send my unraid box out that one the wget download also completed succesfully.

image
image1088×269 13.4 KB

Have i forgot to look at something obvious ?

2

Are you using any kind of DNS filtering? I’ve had problems in the past with github getting blocked (or more specifically AWS) by a DNS filter I was using. adding .aws.amazon.com and .amazonaws.com to the white list fixed it.

3

I’m not the only one then. I’ve seen TERRIBLE outbound internet via Linode. Using my own internal DNS helps, but some sites just don’t load at all

I put it down to sites blocking Linode due to bad traffic, but I highly doubt github would

Following this!

4

One of the first things I tried was changing from my pi-holes just to Google DNS… It made no difference unfortunately. I’m not running pfBlocker or anything on pfSense either.

Out of interest I just chucked my workstation through the same tunnel with the same results

5

I’m glad there are more of us … ha ha! It’s got me scratching my head.

The wierd thing is as mention in my OP, if I SSH direct to my VPS instance I can run the wget command and the download works fine directly to my instance. Got me stumped

6

Oh, well in that case something else is up! I vote DNS

7

Have you checked the firewall log?

8

It seems the traffic is allowed to pass
This is what i get when i try and hit it.

image
image1154×48 4.48 KB

And this is what i get when i wget the level1 logo from the forums page which downloads just fine

image
image1167×46 4.57 KB

I know, DNS is always in the back of the mind.
I can traceroute to the domain from both WAN interface

image
image679×653 28.1 KB

And WG interface

image
image632×702 30.8 KB

9

Interesting… when i traceroute from my unraid box it does not make it to the end, it must time out.

On the left a traceroute from pfsense using WG as the gw. On th right unraid box using the same WG gw

image
image1793×702 61.2 KB

10

Is this ipv4 or ipv6?

What MTU are you using for the wireguard interface? e.g. is your firewall dropping any ICMP?

BTW you can try using curl with --resolve option to ensure you’re always attempting to contact the same IP while debugging - it’d allow you to handily capture tcpdumps by IP address

11

ipv4

MTU is default so 1500 i believe… and yes this was the problem!
I went back and watched a Christian McDonald video on WG tunnel i remember him talking about the MTU value, which i had not set or worried about … clearly until now.

i have set MTU and MSS both to 1420. Everything that was not working is now working A Ok !

@risk Thanks again mate, you have helped me a second time

I also looked at my second WG tunnbel which was working and low and behold, the MTU and MSS values are already set to 1420! DOh!

@FunnyPossum would have a look at these options for your issue also.

12

Thanks for this! I’ll check it out

13

Finally getting around to messing with this, did you have to change the MTU on both ends or just on Linode?

14

It needs to match.

Example, if starting with 1500 and you add ipv6 and wireguard you’re left with 1420 MTU: Visual packet size calculator — Daniil Baturin

You can also find the maximum packet size experimentally using ping -s (man)

You can also try unblocking some types of ICMP

15

Thanks, I went to 1480 and things are working great! Wish I’d done it sooner