PfSense + GFiber + WoW.... oh my

First thanks for the help…

ISP: Google Fiber @ 1Gbps
PfSense v2.4.1

I decided to bite the bullet and try to remove the google fiber network box from my network. To that end I bought a mini barebones PC and put a 60GB ssd and 8GB ram stick into this Qotom Q190 G4. I’ve been able to get it connected to GFiber with IPv4 @ normal 1Gbps speeds. I am unable to get a IPV6 address with the direct connection but when I was builing and install the pfsense software and had the box behind the GFiber network box it would get assigned an IPV6 address???

PfSense seems to be working well, I can watch netflix, amazon, espn and other streaming apps from my PC and nvidia shield device with no issues. I have even got my plex media server and TV server to talk to the outside world. But my latency for World of Warcraft has gone from 80ms latency to 1500+ latency. Now I realize i’m not doing something right and I have looked into the traffic shaping, but still has not helped, so I will assume I’m not doing it right.

I am very new to PfSence and advanced networking, but not new to PC’s and consumer router networking. I could really use some help with setting up PfSense to get my latency back under 100ms for WoW and online gaming.

It doesn’t sound like you’re doing anything particularly wrong. You should be able to run WoW without traffic shaping. I’d say undo all of your traffic shaping rules. Once that’s done, see if you can find the IP of the server you are connecting to, and tracert to it. That should help you find out which hop is adding the latency.

tracert

Nothing looks out of whack, but my in game latency sucks.

I think theres something wrong with they way that Blizzard games handle the port numbers in their networking. If I recall correctly they embed their port numbers instead of putting them in the TCP/UDP header.

Drop by that site and see if you cant forward those ports and help the issue.

That helped a lot, I’m now under 600ms, still not 80ms, but better than 1500+ms

When you connect to the game, look at your PFSense server statistics. See if there is high CPU load, or memory usage. (can’t see why there would, but always helpful to check)

The Q190G4 has a quad Intel card. Which ports are you using? If 1&2 for WAN & LAN, I would suggest moving LAN to either Port 3 or 4.

If it’s the NIC chipset configuration I’m assuming it is, Ports 1&2 share a chip, and 3&4 share another one. (think CPU like cache)

Are you on Wifi?
Do you have it plugged into a Switch, if so, what kind?

I ran WOW and 3 other devices ran speed test from 3 different site and the CPU never went over 50% and the memory usage is under 10%. I did move the LAN connection to port 4, and nothing changed. Adding the port forwarding from the previous reply did drop my ping to 35ms.

Tracing route to 24.105.62.129 over a maximum of 30 hops

1 <1 ms <1 ms <1 ms pfSenseRouter.localdomain [192.168.1.1]
2 1 ms 1 ms <1 ms 10.26.2.73
3 2 ms 2 ms 1 ms 23-255-225-25.mci.googlefiber.net [23.255.225.25]
4 * * * Request timed out.
5 * * * Request timed out.
6 22 ms 23 ms 23 ms 23-255-225-180.mci.googlefiber.net [23.255.225.180]
7 24 ms 24 ms 24 ms eqix-dc2.blizzard.com [206.126.237.174]
8 24 ms 24 ms 24 ms ae1-br01-eqdc2.blizzardonline.net [137.221.72.33]
9 37 ms 34 ms 34 ms et-0-0-13-br01-eqch2.blizzardonline.net [137.221.65.13]
10 34 ms 34 ms 34 ms be1-pe01-eqch2.blizzardonline.net [137.221.69.67]
11 35 ms 35 ms 35 ms chi-eqch2-ia-bons-02.blizzardonline.net [137.221.66.11]
12 35 ms 35 ms 35 ms 24.105.62.129

Pinging 24.105.62.129 with 32 bytes of data:
Reply from 24.105.62.129: bytes=32 time=35ms TTL=246
Reply from 24.105.62.129: bytes=32 time=35ms TTL=246
Reply from 24.105.62.129: bytes=32 time=35ms TTL=246
Reply from 24.105.62.129: bytes=32 time=34ms TTL=246

Ping statistics for 24.105.62.129:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 34ms, Maximum = 35ms, Average = 34ms

I am on gigabit ethernet on my PC…

Google Fiber Jack >> POE injector (for power to fiber jack) >> Qotom PfSense box >> Linksys LGS108P switch (for poe to my TP-LINK AP) >> PC

I’m really at a loss

First of, if you are running the output of the LAN into a switch, make sure the switch speed is correct, if you leave it on auto and it chooses incorrectly, you will get that kind of latency.

On the interface(lan) settings, choose speed for "Media 100baseTX " and see if that helps. In best scenario - match that setting to that of your switch.

Edit: Found that you’re using a switch before pfsense box(why not the other way around, fibre -> pfsense -> switch).
So in your case, you’d have to adjust your WAN settings, to match the PoE switch speed.

My Switch is behind the pfsense router (coming off the LAN port) and is running at 1Gbps.

I have replaced the pfsense router and put the GFiber back into the loop. My WOW latency drops to 78ms. Also I put the pfsense router behind the GFiber network box and now pfsense gets an IPV6 address. So there seems to be some issues with pfsense.

I want to point out, the path your packets are taking are not determined by your port forwards.

Your ping time out of the Google Fiber network is the same in both of your traces above.

I would be interested to see what the PFSense box is getting for traces.

Can you enable upnp on pfSense (if it’s not enabled already)?

Instead of tracert, try winmtr, you can ask it to trace a route with a particular protocol and port number.

I’m not sure about gfiber, but in Google datacenters, routing path absolutely depends on the whole 5-tuple (src/dst IP, sec/dst port, protocol), and then obviously DSCP bits set by the app and kernel. Even though gfiber is a separate network I’d be surprised if gfiber folks haven’t taken some design cues for their gfiber huts internals, unless maybe there’s very little benefit of doing that with only 1Gbps per IP.

Hey all, I got this working but I have no idea how. I wanted to remove the ISP variable and signed up for cable internet. I set up pfsense as a dual wan failover with GF as the primary and TWC as the secondary. Once I got pfsense set up I simply unplugged my GF connection and TWC_WAN took over just as I had hoped it would and then connected to WOW, and my latency was back to 70-80 ms. So I was getting a bit upset with the whole thing and played around in WOW to make sure it was stable, so after 2-3 hours of raiding, I deemed its stable.

I plugged the GF cable table in and it took over as the primary WAN and left the TWC_WAN plugged in then I went back into WOW and all was good, 70-80 ms latency. Now I figured it was too good to be true and the whole states table my be affecting this, so I unplugged the TWC_WAN and rebooted the router… Well a week later and my 70-80 ms latency over GF connection is stable… So I have no idea what did it but it works.

Btw I ditched that Qotom Q190G4 as it just couldn’t handle the 1Gbps speed and built myself an A10-8750 itx Fm2+ based system with an intel 4 port nic and 8GB of ram. I still had the same latency issues when i first put it into production but as you can see above its running hot straight and normal.

1 Like