Pfsense. 2 Xbox One's. Only one with open Nat?

[DastardlyMuffin]

I have 2 Xbox Ones. The first one is just the regular Xbox One original model. The second one is the new Xbox one S (white).

I run a very neatly configured pfsense firewall behind all of this, with both Xbox’s properly whitelisted from openvpn through their Mac ID’s, going through regular Comcast internet.

I’ve given both xbox’s unique static ip’s and static outbound ports as per the instructions on pfsense forums.

I’ve enabled natpnp, and upnp in pfsense upnp settings. I’ve created forwarded port range rules within upnp, 53-65535. Blocked port 3074 to prevent the 2 Xbox One’s from having conflicting ports.

The Xbox One S will sometimes work perfectly with an open port for an entire day. Other times it seems impossible to get it to open.

The original Xbox One always has an open port. Never had a problem with it.

The configuration is literally duplicated from one Xbox to another, with their Mac ID and internal IP the exceptions.

Am I missing something here? Did Microsoft change how the Xbox One S uses forwarded ports?

For the Xbox one S I even get a forwarded port on pfsense, going to status -> upnp. But the Xbox one s never aligns itself to that forwarded port. The Original Xbox one does.

Something is arie here with the Xbox One S and I want to get to the bottom of it.

Xbox One IP: 192.168.1.3
Xbox One S IP: 192.168.1.6

Thank you for your time

[mutation666]

I know that some of these (PS for sure) had issues with UPnP needing to be on

[mutation666]

Did a quick scan of this and seems like a good guide

[DastardlyMuffin]

Thank you for trying but this doesn’t help. I’ve already gone through all of this, you can see in the pictures I provided that upnp AND natpnp are enabled for both Xbox One’s. They are also being forwarded the correct port range within the pfsense upnp rules section. Default deny is on for security, but I am allowing the specific Xbox IP’s so it doesn’t apply to them.

Furthermore I’ve gotten this setup to work perfectly with both xbox’s at random times of the day. They don’t conflict with each other. The Xbox One always works. The Xbox One S works only sometimes.

I do not believe my pfsense configuration is at fault here, it almost seems like something is misconfigured on the Xbox One S, like it maybe has a different Mac address than it is showing within it’s settings?

[mutation666]

Those rules arent being hit look at your rules, there is no activity, try moving them to the top (the bottom 3)

[DastardlyMuffin]

Are you referring to the upnp rules, nat rules, or firewall rules?

[DastardlyMuffin]

Please anyone I need help here. I moved those 3 firewall rules on the bottom to the top and now neither Xbox has an open nat.

It seems everything I do does absolutely nothing. The Xbox’s just randomly decide when to have open ports. Pfsense always reports proper port forwarding.

[thro]

First up… apologies for length, but it’s the simplest way i can explain what’s going on in sufficient detail… Also, see the bottom of the post for the possible work-around.

The problem is thus (and this isn’t microsoft’s fault, or pfsense’s fault - it’s NAT brain damage):

• Game protocols use UDP protocol (for speed - packet loss is bad if it happens, but you can’t do anything to recover from it by the time the data is irrelevant as far as gameplay is concerned - just get the next update, and hope loss isn’t too bad). Because there’s no point attempting to recover data lost to packet loss, UDP is appropriate (and TCP is not because it will slow things down doing re-transmission to recover lost data - because that’s what TCP is supposed to do).
• UDP (for said speed) is connectionless/stateless (as opposed to TCP which does a full handshake that the firewall can see happen and link two endpoints together with a connection state)
• With NAT, your firewall can only determine which host UDP is meant to be receiving XBOX traffic either by knowing how the protocol works (which it doesn’t), or being told which machine needs the return traffic. This is where UPnP helps. The (working) XBOX tells the firewall (via UPnP) that it wants the incoming UDP for the UDP port XBOX protocol uses.
• with two internal XBOXes, stateless UDP and the two XBOXes doing the same traffic (both UDP, both same UDP port, no connection state because UDP), the firewall can’t discriminate which return traffic is for which console (outside of what UPnP told it - which is a single “i want all that UDP port X” rule). So it breaks.

So the bad news…
I do not think you’ll be able to get this to work due to the inherent nature of UDP traffic used by game protocols, and both internal machines using the same protocol. What you did to make the first XBOX work was what you needed to do. But it won’t work with a second one.

If gaming used TCP it could track state, but TCP is not appropriate for this application. If we had IPv6 everywhere, you’d have 1:1 IP address mappings (for private vs. public - or just public IP addresses) and the firewall wouldn’t need to do this hackery bullshit to try and make it work - for one console at least, via UPnP.

Your firewall (be it pfsense, or anything else that does connection tracking) simply has no info to work with due to the way both UDP and NAT work. The fact that one console works is due to the UPnP hackery mentioned above on the firewall’s part (in order to at least make one internal machine work - most people don’t have this situation), and this is why NAT is bad for the internet.

So. If you reboot your firewall (to clear any UDP “connection state” hackery it is doing to try make this work), the first xbox to connect out will probably work, and the second one to connect won’t.

i.e., you will only be able to use one at a time behind a NAT box like pfsense or any home user router.

edit:
You could get this to work if you can get 2 static IPs for your connection, and NAT one XBOX to one IP and the other to the second IP. But you won’t get that on a consumer grade internet connection. It would be cheaper to get a second consumer-grade internet connection for the other XBOX.

Oh… one other possible option:
Set up a VPN on pfsense for the second XBOX only, and have its XBOX traffic pop out of the VPN. That will probably (Maybe? do consumer VPNs work with UDP? i don’t use them) work. But ugh…

[DastardlyMuffin]

Very interesting ideas towards the end of your post.

I am 100% on the same page as you. I have looked in to it just about as much if not more, and was afraid that is what’s going on.

What puzzles me, and you can see this in my pic of upnp port forwarding rules; I denied port 3074 on all internal IPs for upnp. What I found this does, is both xbox’s end up resorting to alternate ports besides the default 3074, and they even end up on different ports each time.

This ironically worked for a few months. Both xbox’s had open ports always, only thing was we had to restart the Comcast modem daily because the openvpn internet connection would get stuck in a frozen state.

Fast forward and I found out the 2 Xbox one’s were somehow interfering with the openvpn service, they were sending a few dozen packets through it even though I had them piped through the regular Comcast internet connection, with the rules being loaded first.

The only thing that fixed this, and I found it on the pfsense forums from someone with the same issue, was adding those 3 firewall rules at the bottom to deny any Xbox one, Xbox one S, or Xbox 360 traffic from openvpn, after they’ve already been routed through Comcast internet.

I was able to get the black original Xbox One to work just last night by moving those 3 deny firewall rules back to the bottom, clearing the state table and restarting pfsense. Instant stable open port all night and morning. Haven’t hooked up the Xbox One S yet, but maybe it’ll work

I also wonder if consolidating those 3 deny firewall rules at the bottom to just one, would maybe solve this problem as well. It’s something I haven’t tried. It does seem a bit redundant to have 3 different rules for the same purpose.

If none of that works, I’ll try your idea of just leaving the other one out of upnp entirely. I don’t want it going through the VPN though just because that’s against M$'s terms of service

[thro]

What puzzles me, and you can see this in my pic of upnp port forwarding rules; I denied port 3074 on all internal IPs for upnp. What I found this does, is both xbox’s end up resorting to alternate ports besides the default 3074, and they even end up on different ports each time.

I think that UPnP if enabled enables the end device to “punch holes” in your firewall rule-set.

e.g., what you put into the firewall rules is essentially over-ridden by UPnP requests.

This is why most corporate routers, etc. do not enable UPnP, and why it is an option that is turned off by default in pfsense and other open source firewalls. Because you are essentially handing control over your firewall rules to the UPnP client device (and thus, a hacked or untrusted internal host would have full control of your firewall if allowed to do UPnP. Bad news!).

As to why messing with the rules works? Not sure. Also not sure why it would have worked before but not now.

Maybe microsoft HAVE changed the protocol slightly.

What may be of interest is to do a wireshark dump (or packet capture on the pfsense box to then open up in wireshark) of what happens when you try to connect to XBOX live or whatever the server is called these days. It may be educational. I’d suggest doing this with only ONE xbox connected (disconnect the other one) the first time to see what it tries to do in a “perfect world” and then repeat with both connected, doing a dump for each box.

Also what may be of interest is that there should be/maybe is(? its been a while and i don’t have a pfsense box handy right now) a state table or UPnP table that you can perhaps take a look at while this is going on. It will help you reverse engineer what the box is trying to do.

I don’t have XBOX so can’t really suggest much else - but all the info on what is going on should be there in the packet captures hopefully, it might just take some digging to figure it out.

Good luck!

[DastardlyMuffin]

Thank you! I’ve been trying to come up with some way to monitor what is actually going on. You’ve saved me some time on figuring out how to do so. I’ll report back here with my findings, as I feel this is very important for anyone else with more than one Xbox in a pfsense firewall!

And tell me about it, I can’t stand this upnp nonsense anymore than you or the corporate world can. But I have an angry uncle who needs his Xbox time after work lol. It’s also why I’ve minimized the ports to 53-65535 and have default-deny on. Minimized the threat at least.

[thro]

And tell me about it, I can’t stand this upnp nonsense anymore than you or the corporate world can. But I have an angry uncle who needs his Xbox time after work lol

Ahh, UPnP has its place. Unfortunately, given the world we live in with NAT everywhere it is a necessary evil for home user gear.

But the proper solution to all this crap is IPv6 and it pisses me off that 20 years later, we’re still screwing around with band-aids that simply shouldn’t need to exist. But that’s a whole other topic

[DastardlyMuffin]

Indeed, ipv6 makes all of this irrelevant. Soon…

[thro]

You’d hope. Vendors are still rolling out products that do not support IPv6. A lot of telcos don’t support IPv6. I’m lucky enough to have had ipv6 on my home connection since 2012 (my ISP at the time implemented it in 2008).

But most of them are dragging their feet.

Hopefully the tiny quantity of ipv4 allocated to China will get the corporate world on board to demand it, so they can access the Chinese market.

[DastardlyMuffin]

Yeesh. I even have ipv6 on my OnePlus One from 2013 with T-Mobile

[DastardlyMuffin]

I FIXED IT!!! Basically I had to create a separate vlan on the pfsense firewall with rules that allow all internet traffic to wan gateway instead of the default openvpn. I then isolated this vlan from LAN to create a ‘sandbox’ around the Xbox live network.

Setting up the VLAN was the confusing part since I am using an openwrt dumb AP coupled with another openwrt router as the repeater using wds and STP. Once I set things up properly on those, I ended up with 2 wifi connections. One for default lan and one for the new sandboxed Xbox live vlan.

I also had to change the upnp interface to the new VLAN in services -> upnp & natpnp. Selecting only the vlan interface for upnp will isolate upnp to vlan, since the upnp service itself is a security issue, even with default-deny on. I noticed my LAN network speed increase substantially after denying upnp to it.

The last piece of the puzzle; this was a surprising one, and my fault: I followed the privateinternetaccess VPN config for pfsense along with the recommended methods to prevent dns leaks and access to wan. The guide had me set the DNS server(s) for all lookups to be PIA. This was backfiring because Xbox live, even with the proper port forwarding rules was still closed, because its DNS was set to a VPN’s DNS (VPNs are against Xbox live ToS)

What I did to fix DNS leaking and proper DNS servers for each network was create a dedicated interface assignment for openvpn, then assign each DNS server in pfsense system settings -> general to each gateway accordingly (wan and openvpn). I also had to disable DNS server override, and enable the DNS resolver.

Cleared firewall state table, unplugged both the Xbox One and Xbox One S, then turned pfsense off. Restarted modem for fresh ipv4 address. Turned everything back on in reverse order with the xbox’s being last, and everything works wonderfully! Open NAT on both Xbox’s at the same time, every time, always. Without hitches. They finally just stay open!!!

Took a lot of troubleshooting but basically a vlan is necessary otherwise the Xbox live network will always find a way to figure out you’re routing other traffic around it. And make sure the DNS server is opendns or something similar. OpenDNS works for me with this setup.

IMPORTANT: Earlier I showed my upnp port rules allowing 53-65535 on both xbox one’s. I had to change that to 0-65535 in order to get open Nat on both Xbox’s with the deny 3074 rule still at the top.

Victory

[gnif]

I have a similar setup but instead of two xbox’s I have multiple windows machines and an xbox. As the Windows machines and Xbox are un-trusted I run them in a separate VLAN also (with a managed switch, so a true isolated vlan), they are all just for gaming after all. To avoid the requirement for hard configured port mappings, instead I opted to enable UPnP for the untrusted VLAN only, allowing the Windows machines and XBOX.

So while UPnP is a security issue, this does limit the possible risk by keeping it to machines that are unimportant anyway, and gives the convenience that even games like Destiny 2 and Satisfactory work without having to manually map ports.

[DastardlyMuffin]

You are right, that’s what I did when I created my vlan. No need to have upnp enabled on the main LAN interface when nothing needs to use it. upnp is what was allowing Xbox live to mess with the rest of my firewall rules

Isolating upnp to the vlan, and going to the vlan firewall rules and setting a deny rule at the top to any traffic going to the LAN network, creates a true inescapable sandbox for internet of things devices. My tplink router with openwrt flashed on it has been turned in to a true wifi capable Ethernet switch. I have a separate Ethernet port dedicated to the vlan network coming from the pfsense box to a second port on the AP. I then map this physically separate port to vlan50, and have it tagging the CPU for vlan50 packets coming from that separate untagged port

The best part of this setup is being able to have 2 WiFi networks which makes the isolation even easier for end users.

[DastardlyMuffin]

Ok I also came across another odd problem;

The nat would only remain open for one Xbox after a few days with this setup. Manually clearing the state table twice would fix it, so that lead me to believe the state table and/or the allowed amount of states was too small for whatever reason.

I fixed it by going to pfsense -> system -> advanced -> Firewall & NAT -> Firewall Maximum states & Firewall maximum table entries;

My maximum states were by default set to 249000 and maximum table entries 200000 for whatever reason, which is wayyy too small for 2 vlans, 4 WiFi networks and two xbox’s with 65535 ports available to them. I increased the maximum states to 3000000 and maximum table entries to 1000000 (I have a 2core 64bit amd athlon with 3GB of RAM)

I also found that the description under “Firewall maximum table entries” is broken; it will change the said default value to whatever you set it to. Doesn’t matter really but it is confusing for end users

[gnif]

I found this too, and the solution was to change the firewall to Conservative if you have not already done so.