[Solved] Virtual PfSense, Linux bridging, and VLANs.. Oh my!

Hello again L1T comrades.

I’m updating my home’s pfSense instance to segment traffic along three VLANs. The caveat is, because pfSense is virtualized, the host’s bridge should be bridge-vlan-aware.

Context

PfSense is virtualized with two NICs:

  1. a physical NIC (eno1/igb0) passed through for the WAN
  2. a bridge (br0/vtnet0) configured by the host for each VLAN’s parent interface
    • bridges a physical NIC (eno2)

PfSense interface assignments:

* Note: I’ve created PASS rules for each VLAN interface to allow all IPv4 traffic originating within that VLAN’s net. Additionally, I’ve temporarily disabled the firewall (and consequently NAT?) to ensure webConfigurator access from the WAN.

Static IPv4 configuration:

  • VLAN 10: 10.11.10.0/24
  • VLAN 20: 10.11.20.0/24
  • VLAN 30: 10.11.30.0/24

Attempted Procedure

I’ve created and configured the host bridge, and created a sub-interface to serve as the access interface for the host on VLAN 10:

# /etc/network/interfaces

auto br0
iface br0 inet manual
        bridge-ports eno2
        bridge-vlan-aware yes
#       post-up /usr/sbin/bridge vlan add vid 10 dev $IFACE self

auto br0.10
iface br0.10 inet dhcp

I’m expecting the DHCP server to issue an address to br0.10 on the 10.11.10.0/24 net and be able to ping the gateway from the host, but neither expectation has held.

Please let me know if I can provide any additional information.

EDIT
Follow-up below

1 Like

You might have already set this up, but I’m going to ask anyways. Have you set up the DHCP server to broadcast & assign IP addresses for that interface?

DHCP servers are enabled for each interface to assign addresses from three disjoint ranges. I’ve also manually specified the default gateways for each interface to prevent any silly business there.

What about reenabling the firewall and only having the pass rules to allow all traffic between the VLANs for the time being?

I’m not entirely sure what is going on here.

I would assume br0.10 is the VLAN interface, which would mean its address is 10.11.10.1?

What about reenabling the firewall and only having the pass rules to allow all traffic between the VLANs for the time being?

Until (V)LAN traffic can correctly pass, the web interface can only be accessed via the WAN interface. It’s possible that disabling pf and NAT is contributing to the issue, but I can’t say for certain.

I would assume br0.10 is the VLAN interface, which would mean its address is 10.11.10.1?

I’m expecting br0.10 to be the host’s VLAN interface, which should be the same as any other DHCP client for the 10.11.10.0/24 net. 10.11.10.1 is the gateway and should be an interface inside pfSense (I think).

I found that it was easier to create virtual interfaces for each vlan rather than configuring vlans in pfsense for a single virtual interface.

2 Likes

For a bridged physical interface, would you also recommend creating and bridging sub-interfaces in the host or pfSense? Minimally, traffic from VLANs 20 and 30 are expected across the physical NIC.

This. We created an IoT vlan that was isolated from the others using this method. You can also restrict Internet access (for Chinese camera systems in our case) on others. The vlan can be run completely ignorant that any other vlans (or even the Internet) even exists.

Follow-up (Half-assed solution)

There is another post somewhere on the forum that links to this page (sorry for naked reference). Much of its content is over my head, so without going into detail, it is necessary for my setup to do a combination of the following:

# sysctl net.ipv4.ip_forward=1
...
# sysctl net.ipv4.conf.all.rp_filter=2
...
# sysctl net.ipv4.conf.all.arp_filter=0
...
# bridge vlan add vid 10 dev br0 self
# bridge vlan add vid 10 dev vnet0

After about a minute, the DHCP server issued br0.10 an address on the 10.11.10.0/24 net, allowing the host to ping the pfSense guest at 10.11.10.1.

A brief epilog

Without knowing any better, I assume the above solution is highly permissive and not suitable for my use case. As a note to my future self and aspiring readers, future implementations should significantly reduce the scope of the above commands (i.e. don’t use net.ipv4.conf.all. ...) and include broader considerations for ARP and RP uses.