I’m trying to connect 3 computers in a Proxmox cluster to my WAN (Arris SB6190 modem) to host a high-availability pfSense VM. I’m not using pfSense’s CARP failover that requires multicast and multiple public IPs, this is simply pfSense running on 1 host at a time and if it fails it comes up on a different host. This works 100% through a $15 Netgear dumb switch.
I am trying to set up a VLAN on my Extreme 220 managed switch so I don’t need a separate switch and I just cannot make it work. My VLAN seems to be properly set up to work like an independent “dumb switch”, does not pass traffic to the switch’s management interface or my main network, but can pass LAN traffic when devices and/or my main network are patched to the VLAN with a cable.
Except when I connect my SB6190 modem and anything else to the VLAN, it doesn’t work. Even with only the modem and 1 host attached, pfSense will sometimes see a WAN Gateway address after several attempts to release/renew WAN but it never gets an IP or sometimes gets 0.0.0.0. A Windows laptop which gets WAN access when connected direct to the modem just sees “unidentified network, no internet” when connected to the modem via the VLAN.
I needed to disable STP on VLAN member ports to get the VLAN to even pass LAN traffic, and I don’t understand why, but maybe that’s a clue to the issue I’m having. Other than that, I’ve changed every setting I can find, most of these settings either broke basic function of the VLAN or had no effect, which is what I expected. I’ve messed with:
Port set as General vs. Access and set both Native VLAN and Access VLAN ID to match intended VLAN
Set ports as Untagged vs Tagged vs Admit All
LACP Enabled/Disabled
Triple checked ports are Excluded from main VLAN and Included on second VLAN
Reset all VLAN settings and rebuilt config from scratch
Turned off STP on the port (on my main VLAN) where my Netgear router (AP mode) is connected because I was seeing BDPU errors, a known behavior related to STP that happens with Netgear routers and certain switches
Tried to set up everything with the Netgear router/AP totally disconnected from the switch
Besides the settings that made the VLAN unusable (like only accepting Tagged frames) all these changes left the VLAN usable for LAN traffic as if it was a separate dumb switch, but not working for WAN traffic.
Going back to the separate dumb switch, it works no problem. So my modem, pfSense instances, cables, etc don’t seem to be the cause.
Thanks, I’ll try setting it up that way with the console. I think I tried setting it like that in the GUI when I was tinkering with this earlier, but I’m not sure. In the meantime here is my network layout:
Unfortunately those steps in the CLI just set the same config I already had set, and they were still set as General ports in the GUI, not Access ports - though my understanding is with the right Native VLAN and PVID settings those settings basically do the same thing.
I changed to Access mode in the GUI and it had no effect. Still works for LAN traffic exactly as expected. Doesn’t work at all for WAN traffic
You really want to set the ports in access mode on vlan 101, no trunking, no general.
For testing, initially disregard dhcp, set a static ip on your pc in the same range as the modem, and use that. Once ping works with static IPs you will likely see dhcp working
Do not, for any reason, patch in vLAN 1 , it will only cause confusion
Long-shot although this sometimes works for me when I can’t get a WAN address from my Surfboard Modem.
Try booting up the modem completely to an internet connection while it is not attached to the switch. After it establishes a full bi-directional internet connection (I believe its Green Blue Blue Green for the LEDs), then pop the ethernet cable into the switch and see if you get a WAN gateway.
Thanks for everybody who chimed in here. I think the advice on rebooting the Surfboard modem might have been closest. It never ended up working for me no matter how many times I rebooted it or what order I plugged it in before/during/after the reboot process. However - I switched off cable to fiber internet and my new ISP’s ONT has zero problems going through a VLAN on my managed switch using the exact same configuration I’d been trying before.
So I still don’t know what was going on here on a technical level but it certainly seems to have been some weird behavior from the Surfboard modem.
One thought I had was if the switch’s management interface was exposed on that VLAN there would effectively be 2 things (potentially) trying to get an IP or generally talk to the modem. And of course that would be a massive security issue. But I triple checked and it was never configured like that. It really seems for untagged packets, the VLAN on the managed switch should look just like the unmanaged switch which worked fine, but there must have been some subtle difference.
LLDP? Some cable modem firmwares check for switches connected to it when in bridge mode, trying to block customers from aquiring more than one public IP.
Thanks for the input! That sounds more plausible than any other idea I’ve come up with for why a “dumb switch” would work but a VLAN on a managed switch could potentially be any different… too bad I don’t have the cable internet plan anymore because I would love to somehow confirm if that was the problem
