So... quite a specific question I can't wrap my head around, concerning networking and kvm.
I have a server running a virtualized instance of pfsense. The pfsense LAN interface is bridged using virtio to interact directly to the physical LAN network. The physical machine has a (static) IP address on the network of 10.0.0.100, and the pfsense box has 10.0.0.1, bridged over that connection. The server's gateway is indeed a VM running on the server itself, that's the setup.
I also have a few LXC containers running on that machine, using the same bridge with IP addresses in the range 10.0.0.101-110.
Everything works as expected, except for this one little thing I don't understand 
If I configure port forwarding on the pfsense box for SSH, I can forward to and reach other machines in the 10.0.0.x network (example my laptop at 10.0.0.22) or the pfsense instance (10.0.0.1) from outside.
What I can't reach using port forwarding, is the physical server itself (10.0.0.100) or any of the LXC containers (10.0.0.101-110).
But: if I make the port forwarding land on the pfsense box (10.0.0.1), I have no problem connecting to that and then ssh into the physical server or the LXC containers using exactly those IP addresses port forwarding won't allow/serve.
It seems port forwarding can't work in this scenario where the traffic goes out to the LAN over a specific NIC, and the host it needs to reach is also communicating via that NIC. But when I ssh from pfsense to those containers, I'm doing essentially the same thing, and that works fine.
So while I have a workable solution, I fail to understand why this is the case. Any thoughts?
Is pfsense the only router in this configuration? It sounds like you're tyring to portforward from a different router to stuff behind the pfsense router?
No, it is not. It is behind a cable modem with integrated router that does NAT as well. So the pfsense sees a 192.168 class IP on the WAN side. Unfortunately I can't bridge or bypass that.
The pfsense router is configured in the DMZ of the cable modem/router to get around double NAT. It is has Pure NAT at the moment, and the WAN interface is set to not block non-routable IP addresses.
I successfully configured IPSEC on the pfsense router for my mobile devices, something I could not do with the provider router. I can see the firewall rule giving a "PASS" to the incoming SSH connection on 10.0.0.100 on the physical port of the server as well. And I can connect to other machines on the 10.0.0.x LAN. That's why I thought that was set right. But I'm not 100% sure it does not come into play.
Do you think this is still giving me issues? How could I test that?
Update: I can make the question simpler in nature. Pfsense has a WAN side of 192.168 and a LAN side of 10.0. If I am at home and I hit a ssh session on the WAN address, it will get forwarded to the pfsense router itself, or physical hosts on the network. The server running the VM or the LXC containers time out. I'm starting to think my switch can't handle this.
And probably my last update as I don't think I will be able to figure this out: I can reach the server if I give it a second 10.x address on the LAN. It seems the consistent conclusion is that a virtualized pfsense instance can't port forward to IP addresses on the same bridged interface.