pfSense port forwarding/NAT woes

Scepterus · March 4, 2022, 2:35pm

Hi everyone, after my previous post about the pfSense install and how it saved my internet connection because my D-Link router was failing as a dialer, I wanted to update a bit about my progress with pfSense.

Right now I have pfSense as my main router, that connects with one interface to the modem and uses pppoe to dial, and the other interface to a gigabit hp switch.

Software wise I have the following packages: pfBlockerNG, squid, snort, and a ferw others for diagnostics and on demand tasks.

There are a few issues I’m still trying to fix, but everywhere I looked was unsuccessful in my search.

I’m suffering from an issue that torrents do not upload at all, and if they do, it’s in the fraction of the speed that is allowed to them.
Before the switch to pfSense, I had torrents seeding at the max upload rate of my network to the point I had to cap it in order for the rest of the network not to suffer.
After the switch (and a lot of tinkering with the NAT rules) I sometimes see torrents seed, but as I mentioned, not even a 10th of the limit that I set is reached.
Any ideas on what I missed in my configuration?

Secondly, I have squid set up as a cache with MITM SSL, and I sometimes get SSL errors, and images not loading, and after a few minutes and a refresh it’s fixed. Anyone knows what is the cause of this?

Also in the port forward category, I tried to host a game with a friend yesterday on Parsec, and I set up the port forwarding ahead of time like they suggested, and we got it working once.

Then, no matter what, we always got the “could not establish peer to peer connection” error.

Would love to hear what you think.
Thanks.

2FA · March 4, 2022, 8:18pm

Do you have your bittorrent port forwarded to the seed box and if yes have you verified it actually reaches the box? If you’re using TCP, you can use telnet from outside your network to see if that port is actually listening.

Scepterus · March 5, 2022, 5:23pm

Seed box? I have the port forwarded on my WAN to the correct PC on my network, and I’ve checked if my port is open from the outside, and it is.

Also, downloading works fine.

mcleod · March 11, 2022, 6:36am

Have you checked to see if snort is blocking?

Also, I have had off and on issues with squid breaking things over the years if you are running it in proxy mode. I actually am having an issue with it right now with windows updates. They will not download unless I disable squid.

Also, are you using traffic shaping? If not, it is amazing. But it is another one of those services that can screw things up unless configured properly.

Scepterus · March 11, 2022, 10:38am

That was one of the first things I thought. It’s not.

I tried disabling it to see if it works or not, that did not solve it either.

I have a working list of refresh lists lines that might help you. Windows update works fine for me, and has actually made me realize I need a full size hard drive with a larger cache. Because 5400RPM works great for small files, but larger ones the speed just falls after the first 16mb.

You mean the squid feature? If not, could you elaborate?

mcleod · March 12, 2022, 5:53pm

Sure, I would love to check it out.

Firewall → Traffic Shaper
That is where you can prioritize traffic, like VOIP, etc. The cool thing about setting up bandwidth limits here is you can set torrents to be very low but when there is no other internet traffic it will let them have more.

I have poor cell service and use WIFI calling, so that is always my highest priority traffic. But it is also low bandwidth.

Scepterus · March 12, 2022, 7:36pm

#Microsoft
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
refresh_pattern -i microsoft.com.akadns.net/.*\.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
refresh_pattern -i deploy.akamaitechnologies.com/.*\.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
range_offset_limit none
#git
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe)                     259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf)                  259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe)                  43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
#new refresh patterns 3
acl Windows_Update dstdomain windowsupdate.microsoft.com
acl Windows_Update dstdomain .update.microsoft.com
acl Windows_Update dstdomain download.windowsupdate.com
acl Windows_Update dstdomain www.download.windowsupdate.com
acl Windows_Update dstdomain au.download.windowsupdate.com
acl Windows_Update dstdomain bg.v4.pr.dl.ws.microsoft.com

#GOOGLE STUFF
refresh_pattern -i \.(cdn) 10800 100% 43800       # CDN CACHING
refresh_pattern -i (cdn)   10800 100% 43800       # CDN CACHING
refresh_pattern -i (.|-)(xml|js|jsp|txt|css)?$ 360 40% 1440        
refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 129600 100% 129600      
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 129600 100% 129600      
refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600       
refresh_pattern -i ^.*safebrowsing.*google                                  129600 100% 129600      
refresh_pattern -i ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk|\.il)     129600 100% 129600   
refresh_pattern -i ^https://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk|\.il)     129600 100% 129600   
refresh_pattern -i ytimg\.com.*\.jpg                                        129600 100% 129600  

#YOUTUBE
refresh_pattern \.ytimg\? 10800 90% 10800    #YOUTUBE IMAGE SERVER
refresh_pattern -i (yimg|twimg).com.*        1440 100% 129600   
refresh_pattern -i (ytimg|ggpht).com.*        1440 80% 129600     
refresh_pattern -i (get_video?|videoplayback?|videodownload?|.mp4|.webm|.flv|((audio|video)/(webm|mp4))) 241920 100% 241920        store-stale
refresh_pattern -i ^https?://..googlevideo.com/videoplayback.    10080 99% 43200        store-stale
refresh_pattern -i ^https?://..googlevideo.com/videoplayback.$    241920 100% 241920        store-stale

#Linux Updates
refresh_pattern -i .ubuntu.com 0 100% 129600 refresh-ims
refresh_pattern -i .linuxmint.com 0 100% 129600 refresh-ims
refresh_pattern -i .canonical.com 0 100% 129600 refresh-ims
refresh_pattern -i Packages\.bz2$ 0       20%     4320 refresh-ims
refresh_pattern -i Sources\.bz2$  0       20%     4320 refresh-ims
refresh_pattern -i Release\.gpg$  0       20%     4320 refresh-ims
refresh_pattern -i Release$       0       20%     4320 refresh-ims

#Images
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js|webp)(\?|$)           43800 100% 129600  refresh-ims      # JPG | JPEG | JPE | JP2 | GIF | PNG | BMP | TIFF | ICO | SWF

#End
refresh_pattern ^ftp:          1440    20%    10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?)         0      0%      0
refresh_pattern .              0      20%    4320

These are the patterns I’m using, there might be some overlap, but I haven’t gotten around to looking deeply into them.

There’s also this, which I found during my search:

github.com

mmd123/squid-cache-dynamic_refresh-list/blob/master/pfsense squid refresh patterns

# RELEVENT GENERAL INFO HERE
# 1 year = 525600 mins, 1 month = 43800 mins, 1 week = 10080 min, 1 day = 1440 min

#Optional: dont cache wordpress admin panel
refresh_pattern (wp-admin)  0 0% 0

# SPECIFIC USE CACHING HERE
# live_user: twitch preview thumbs
refresh_pattern -i (live_user) 30 60% 300  

# for malwarebytes update checking
refresh_pattern -i (mbamupdates.com) 1440 60% 10080  

#MISC FILE CACHING HERE
refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|rpm|divx|dvr-ms)(\?|$) 43800 100% 129600        # 3GP | 7Z | ACE | ASX | AVI | BIN | CAB | DAT | DEB | RPM | DIVX | DVR-MS
refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso)(\?|$)                         43800 100% 129600        # RAR | JAR | GZ | TGZ | TAR | BZ2 | ISO
refresh_pattern -i \.(m1v|M2V|M2P|MOD|MOV|FLV)(\?|$)                       43800 100% 129600        # M1V | M2V | M2P | MOD | MOV | FLV
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|$)           43800 100% 129600        # JPG | JPEG | JPE | JP2 | GIF | PNG | BMP | TIFF | ICO | SWF
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p))(\?|$)              43800 100% 129600        # MPEG STYLE CACHING, VIDEO AND MUSIC | MPG MPEG | MP1-2-3-4 | MK-A/V | MS-I-U-P
refresh_pattern -i \.(og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav)(\?|$)             43800 100% 129600        # OGX | OGV | OGA | OGG | RAR | RM | RAM | RPM | SND | VOB | WAV

This file has been truncated. show original

Traffic shapers are not enabled on my box, so that’s not the case.

Scepterus · March 21, 2022, 4:12am

Did those rules help you with your issues?

Regarding my issue, I found that Parsec was blocked by pfBlockerNG, so when I shut that down Parsec was working.

The problem remains with the torrents, and it drives me nuts. I’ve posted in 5 different places about this stupid issue, and no one else seems to have the same issue.

If I won’t be able to solve this, I’ll start looking for other solutions to replace pfSense, as it is a very clunky and a not very user friendly piece of software.

EniGmA1987 · March 22, 2022, 3:20pm

It could be your intrusion prevention system seeing these random connections coming in as threats and blocking them.

Scepterus · March 24, 2022, 3:56am

Except I already turned all of it off, and still no dice.
I’m growing tired of pfSense, and will start looking in to ipfire as a replacement.

EniGmA1987 · March 28, 2022, 5:43pm

I tried IPfire for a while. Performance was fine, but it has far less hardware support for good NICs and it was a lot harder to figure out how to do things in it as there is very little user support since it has such a small userbase. I ended up abandoning it and chose PFsense over it as PF was far superior IMO. I did have quite a few issues with PFsense as well of course, but I had even more with IPfire. Judging from your posts here, you seem smarter than I am at this stuff though so maybe your experience will be better.

Zedicus · March 28, 2022, 7:10pm

i like OpenWRT out of the router distros. i have tried a lot of them from super user friendly like Untangle, all the way to almost no gui at all environments. After several years of rolling my own, i ended up on ubiquiti stuff. but by the time you get it working, it is still a ‘roll your own’ system.

i guess my point is buy a 6 pack, and good luck.

Scepterus · March 29, 2022, 5:14am

I am running pfSense now since January, and I’m not sure intelligence has anything to do with this software, it’s a combination of luck and the need to give up on things when they don’t work. And I am not the giving up type.

I tried ipfire in a VM first, and discovered they neutered squid, and won’t allow me to run a MITM cache, because they decided it’s “not safe”. Classic case of “I’m smarter than my users, hence I know better than them what they need.” My advice, keep away from that.

That’s funny, I actually replaced the software on one of the Ubiquiti’s in my previous job with OpenWRT. I like OpenWRT, though I’m not sure if they do x86. I am not willingly using Ubiquiti’s software, it is a hot mess.

Now, I am testing clearOS in a VM now, but should that fail I will have to take the advice I was given on another forum, and start a Linux server instead and turn it into a router.
If that will be the case, would anyone be interested in me documenting the process in a dedicated post? I would also ask for any advice from people who might have tried this before.

Anyway, since this thing is clearly still interesting some of you, I’ll keep this post updated with my findings on clearOS, and whether I end up using it.

Zedicus · March 29, 2022, 12:26pm

Ubiquiti is a hot mess.
OpenWRT used to maintain an x86 port and a very nice home built router could be assembled using it. I have no idea if the x86 port is still maintained, that was years ago.

Scepterus · March 29, 2022, 2:23pm

OpenWRT does still have an x86 one. clearOS is also a bust. It’s like a proprietary router.

Scepterus · March 29, 2022, 4:28pm

Well sadly, as much as I love OpenWRT, they only support UEFI boot, and my current machine is older than that. So it’s a maybe in the future, not right now situation.

I guess the Linux server is my only real option, though I’m not even sure where to start.

2FA · March 29, 2022, 10:32pm

If you’re fine with CLI only, VyOS is pretty good. I use the 1.4 rolling release, there have been some issues do to them rewriting to use nftables instead of iptables but they’ve been pretty quick to debug and fix issues I reported on the forums. Or you can build 1.3 LTS using their iso building docker image which isn’t terribly difficult.

moriarty · March 30, 2022, 1:37am

Are you running physical or virtual pfsense router? I just switched from pf to opensense on kvm/qemu due to pfsense having trouble with virtio networking. If physical never mind.

Scepterus · March 30, 2022, 4:24am

Thanks for suggesting, but I’m not the type to pay a subscription for my own equipment.

Physical, there’s another thread where I state the entire spec of it.
I’ve looked into opensense, for me it’s the same thing only less stable.

Zedicus · March 30, 2022, 2:54pm

there is a fake eufi bootloader available that was popular on intel systems a while back. it put a bootloader on the HD that bios could boot, then it would redirect to the uefi partition table. i honestly do not remember what it was called, or if it would work with OpenWRT. it was just a random thought from the archives of my bourbon soaked brain.