How to prevent DHCP conflicts between two subnets

Hardware Hub Networking
1

Hi.
I have a modem on the 192.168.1.0/24 subnet with DHCP. This modem is directly connected to my Supermicro server that runs ESXi 7.0. ESXi gets its IP from the modem. ESXi is running OPNsense that acts as a router for wired and WIFI connections, also with a DHCP server. OPNsense is on the 192.168.10.0/24 subnet.

What I want to achieve is to be able to connect to the ESXi UI from the 192.168.10.0/24 subnet. I understand that I have to use static routing. But my question is how can I prevent conflicts between the two DHCP servers? The modem should only assign an IP to ESXi, every other device should get its IP from OPNsense. I also have a managed switch that supports VLANs.

Could anyone give me some basic instructions to correctly configure my network.

Thank you.

2

Do you have a spare NIC that you passed through to the router vm? If so it would be easiest to go: Modem → router vm (passthrough nic) → managed switch; then plug your esxi management nic into the switch and have it get its IP address from that router VM.

3

Router VM has a passed through NIC but I don’t want ESXi to get the IP from the router VM. What if the router VM goes down? In this case can’t access ESXi.

4

There shouldn’t be any conflict, if your ISP’s router provides DHCP on 192.168.1.0/24 and OPNsense provides DHCP on 192.168.10.0/24.

There will be double network address translation (NAT), for any computer connected to the 192.168.10.0/24 network, single NAT for any device connected to 192.168.1.0/24.

If your device you connect to 192.168.10.0/24 asks DHCP for an address and leases 192.168.10.100, it really has no concept of what is outside of that 192.168.10.0/24 network. The same goes for anything connected to 192.168.1.0/24.

Outgoing traffic would get routed up through both networks, with the router/gateway navigating for you the routing up and out of each network until it leaves your network if bound for the internet.

Incoming traffic would, unless previously accompanied by outgoing traffic that could get stored and retrieved (NAT table), have to be set up to allow that traffic (port forwarding is one method) to that specific host, and specified in both locations (ISP router and OPNsense).

So if unsolicited traffic came in destined for your ip (let’s say 5.5.5.5) on port 1234, you could have an entry that specifies that port should forward to 192.168.10.1:1234 (opnsense). Once it reaches your OPNsense instance, it also needs information on where to send it, so you have another port forward record to forward it to the specific computer you want it to go to (192.168.10.100:1234). Otherwise the router receiving the incoming request for 5.5.5.5:1234 has no idea where to send it.

So 5.5.5.5:1234 hits isp router,
first NAT of 5.5.5.5:1234 => 192.168.1.100:1234
second NAT of 192.168.1.100:1234 => 192.168.10.100:1234
and it finally makes it to your computer

I’ve solved this at least two different ways, depending on what options your ISP gives you.

The first (best) way was to set up the ISP’s device to be as dumb as possible and only provide the means to connect over the ISP’s change in medium (ethernet cable to coax) and let my router ask the ISP for an IP directly. I’ve done this for Comcast, ATT (before they got the new, unhackable routers with no bypass method for EAP), Spectrum, probably a few more, too.

The second way, mainly with AT&T, because they force you to use their EAP authenticated device on their fiber network, is to set up IP passthrough. Your router behind the ISP’s router (still does translation but…). My router gets a public IP (and a static IP block, which I can assign to vlans or devices directly), which flattens my network a bit.

VLAN tagging can be an option, but it might be overkill for what you’re trying to do (requires some networking knowledge or lots of time banging your head and learning it as you go).

5

Put a static IP on it and that way if the router goes down you only have to be in the same vlan/network and you’ll be able to access it.

6

Commonly this problem is solved using vlans and a managed switch. Vlans let you have multiple layer 2 networks on the same physical device, assigning ports to be in each vlan “untagged”, while your pfsense box gets all the vlans as “tagged”. Because each vlan is independent, they can have their own DHCP scope.

As for routing your ISP modem needs a static route for 192.168.10.0/24, something like:

192.168.10.0/24 gateway 192.168.1.1

7

Regarding the VLAN settings: which settings must I set on the managed switch and which settings in OPNsense?

8

ESXi needs a gateway in its settings. Lets say I set a static IP for ESXi and the router/gateway with the IP 192.168.10.1 goes down can I still access ESXi?

9

Yes, the only thing it will not know how to do is get to the internet when the router VM is down as the gateway is down.

10

if you are a beginner in networking, the easiest way is to have two switches, one for the WAN side and the other for the LAN side, do not connect the switches together and never put WAN and LAN of a firewall on the same switch without VLAN!
Use at least two nics for your ESXi host and passtrough the one for your ISP, don’t use a vnic!

With only one switch you can’t get past VLAN, have a look here, VST is the way to go

https://kb.vmware.com/s/article/1004074

And actually never put your firewall/router on your test system, in case of doubt you want to have internet to solve problems