Looking for some networking help :p (proxmox + Pfsense)

Operating Systems & Open Source Linux
, ,
1

Hello again im back with some more noob questions about vms and proxmox :stuck_out_tongue:

I’ve set up my network in this way but I’m wondering if there its a way to eliminate the connection to the server and route everything through the Pfsense vm and how that would be done.

The other option I’m thinking about would be to close all ports looking to internet in the hypervisor and route all the trafic through the pfsense VM (what im doing rn). But I still don’t understand how to route new vms to the safe lan I created in Pfsense
My plan with this server would be to use it as a Pfsense router + NextCloud + Docker container and some other misc stuff

image
image1392×778 108 KB

so the question would be
How to route vms through my pfsense lan?

My plan its almost ready but I still need to solve this issues, thanks in advance and have a great week

1 Like
2

Firstly, replace arrows with lines :sunglasses: … better.

Can we get some clarifying information:

  1. Next, how many network ports do you have on your server, the port on the motherboard + some nic?
  2. What kind of nic - How many ports.
  3. What kind of connection is your ISP using to supply you with internet and what kind of ISP modem/router do you have? Is it docsis cable/vdsl/gpon fiber/… ?
  4. Who is your ISP and what exact modem did they give you?
  5. How much total bandwidth are you looking to route.
  6. Is the switch you have just a “dumb switch” or can it do VLANs?
  7. Is the switch physically near the server?
3 Likes
3

Arrow supremacy :stuck_out_tongue:

  1. 4nic + 1 motherboard
  2. Dell Intel Pro/1000 Vt Quad Port
  3. my isp has provided me a zxhn-f680 with a fiber cable (dynamic IP)
  4. https://www.iplan.com.ar/
  5. 300mb
  6. usw-flex-mini i understand it can do vlans
  7. the whole stup its right by my side
1 Like
4

IMO, the easiest option would be to passthrough your PCI-E quad port NIC to your pfSense VM. Make one port the WAN, then use the other 3 for separate VLANs, or even all of them in LACP or balance-alb if your switch doesn’t support LACP.

If you switch is managed and supports port bonding, aka link aggregation (LAgg), then you’re gold.

Wow, a 5 port switch. Ok, not sure if VLANs are even worth doing on such a device.

Nah, simplest way to configure this would be to make 1 LAN on port 2 and connect that to the switch, then on the Proxmox side, create a virtual NIC in the pfSense VM. In pfSense, set that virtual NIC as OPT1 (probably name it proxmox) and set another subnet in there. Then you get away with 3 ports free, 2 from the NIC, 1 from the motherboard.

On the virtual NIC in proxmox, don’t give any IP to the port, assign one in pfSense instead. Then in proxmox, bridge that virtual NIC with its internal network.

The alternative to this would be to just ignore the virtual NIC and set another LAN on port 3 of the PCI-E NIC and connect that port to the motherboard port, then bridge the motherboard port to your VMs like you normally would (all except pfSense). If you aren’t using VLANs, there is no disadvantage to this, but if you do, you will be limited to gigabit speeds between VMs in different VLANs, as opposed to using a virtual NIC with potentially higher throughput.

If you want to set VLANs on your switch. as you have 5 ports, I would suggest you just do 2 LANs, one on port 2, one on port 3 or 4 and set the switch ports to mode-access on different VLANs. Then send the switch port to your router / AP for one WiFi VLAN / LAN and the other to your wired devices. However, this complicates the setup a lot, and if you don’t have any wireless devices that you don’t trust, you are better off just doing 1 LAN and not using VLANs at all on the switch, only because it gives you the advantage of local switching, instead of having to go to the router and then come back to the devices you want to reach, like say if you are transferring photos from your phone to your PC via WiFi.

Anyway, let me know if my ramble wasn’t clear enough, tonight I don’t feel like I can words well.

:man_shrugging:

1 Like
5

I’ll scream Leeeeeeeroy Jenkins, and try to achieve what you have kindly and neatly proposed.
I don’t think I’ll make it through, bugs will likely attack along the journey. But I’ll do my best.
Btw the ramble was very clear!

1 Like
6

Let us know at what point you stumble. You may use my easy to follow small secure network tutorial and adapt it to your needs. I need to work on ipv6 though, it’s only made for ipv4. I never had an issue with it though.

1 Like
7

First off. There should be exactly one cable going from your server to the CPE. And that cable should be connected to pfsense’s WAN port. Anything else and you might as well not be running pfsense.

If you want access to the CPE from LAN side, there’s a recipe in Netgate’s official documentation that takes care of that.

As for managing your internal network, in my experience it’s not a great idea routing internal traffic through pfsense. The less traffic it has to deal with, the better and snappier it will perform. Sure, you could throw resources at it and mitigate the problem, but it’s still suboptimal and probably unnecessary.

It all depends on what services you’re running of course and how much you trust your LAN clients. Ideally, you’d only be passing 2 NIC ports to the pfsense VM. 1 for WAN and 1 for LAN. And assuming your WAN bandwidth is less than your LAN bandwidth, you won’t be introducing any bottleneck to the network.

For a quad nic + mobo I’d propose this setup.

Mobo > Switch. Proxmox management so you can still access the server when pfsense is down.
Port 1 : Pfsense > CPE. WAN interface
Port 2 : Pfsense > Switch. LAN interface
Port 3 : NAS > Switch. Assuming you’re running some sort of file server VM in there. It should have its own cable to the switch.
Port 4 : Rest of your VMs that you want exposed to the LAN, like a media server or whatever.

And on top of that, make an extra virtual switch in proxmox and assign whatever VMs you want protected from all sides. A password manager for example. This switch should have no physical ports attached. Strictly internal. Pass it to pfsense as well so anyone wanting to get in or out of there has to pass through the firewall. Hopefully these would be low traffic VMs. This is your DMZ.

That’s the gist of it basically. Keep it (relatively) simple. The more stuff you can trust to keep in your main LAN subnet, the less headaches you’ll have down the road.

Other than that, unless you live in the middle of nowhere and nobody ever visits, consider your WiFi hostile and treat it as such. Get some dirt chip routers that can be flashed to openwrt. Give them their own subnets, set them as gateways in pfsense, and ideally never let them touch your actual LAN.

1 Like
8

I’ll have to get my mind to digest all the info that has been given to me rn. Thanks a lot for making the forest a bit brighter! :evergreen_tree: :evergreen_tree: :evergreen_tree:

Once I’ve made a clear path in my mind on how to execute, I’ll come back with some more questions. Its going to be a fun week of bugs and blood in my network hahah

9

this its gold sr, I made it work on first attempt :face_holding_back_tears:

10

Hey Folks, late to the party.

I am having similar issues yet I am having a logical error. I also want to have my pfSense as my end station of my network, intending to run IDS on it.
So…

  1. I have my ISP Modem/Router - 4 LAN Ports/1 WAN
  2. Homeserver, with Proxmox, where I want to deploy my pfSense. 1 Lan on MOBO & 2 on NIC.
  3. Then I have a switch, which I think should be contected to my pfSense interface.
  4. Rest of my devices.

Q1: Where should I connect the first Lan cable which goes out of my ISP Modem? I tried to connect to my Server Mobo, which gives me access to my Proxmox Interface. This way my PC has no Internet, so I have to use my Laptop for the following steps.
Q2: When I boot up pfSense VM, WAN Finds no IP adress - do I have to connect my ISP Modem Wan port to the NIC on my Server?
Q3: I am guessing I have to connect the other port on my NIC to the switch, so I can sniff everything comming from the switch ports?

Thanks for the help!
*Also what is a CPE?