I’m setting up my first home lab and am looking for advice on best practices and input on my current networking plan. Any inputs are much appreciated!
See the attached diagram for an overview of my proposed network setup:
I am thinking that I’d like to keep my current router/modem from my ISP. It is doing a good job of providing wifi coverage to the house and many family member devices already use it that I don’t want to have to reconfigure. Particularly, I like the idea that if I break something in my home lab there will still be this ISP device in place to provide internet access to the rest of the family.
In that case, I am thinking of connecting my own router to the ISP router and using that to manage my home lab. Currently I am thinking of using a mini PC (will need to buy a network card for it I guess to get a second ethernet port for WAN and LAN) running OPNSense.
I would like to be able to remotely access my homelab devices via a VPN. I would also like to access them from home from my work laptop and phone, and work station PC. The laptop and phone could be connected to the home lab network via a dedicated wifi access point just for those devices (I am thinking of getting a Ruckus R720). I have a wired connection between my work station PC and the ISP router which are in different rooms, and the server gear is in a third room. So I don’t want to run additional ethernet cables between rooms and am hoping to use the current setup. Where possible I would like to use multiple VLANs as an added layer of isolation and security.
My understanding is that I can achieve this by putting this second router in the DMZ of my ISP router then using the second router to manage all firewall setup, VPN access, etc.
Does this approach make sense or will I run into fundamental problems? Is double NAT going to be a problem here, and if so are there good ways to overcome this?
Any suggestions or alternative approaches would be very helpful, thanks for your time!
I am not a networking guy but will provide some comments to the best of my knowledge
The HW firewall is best placed in line ahead of your other network devices routers, wireless access points or switches (As an example… I would recommend getting a dedicated device like netgear which does pfsense )
Your switch is what will help you both extend and segregate your networks (so you could also consider connecting your wireless access points to your switch)
Your wifi access point may not be able to segregate your wireless devices (In my case i have multiple wifi both 2.4Ghz and 5Ghz but regardless of which wifi network I connect to… all devices get same ip address range since my wifi cannot do VLANs )
Not sure you need a second router …
[modem] --wired-- [firewall]
Your firewall has mulltiple LAN ports so extend your network as follows:
Option1:
[firewall] --wired-- [ISP router now acting as wireless access point only]
[firewall] --wired-- [switch] --wired-- [ wired connection devices]
Option2:
[firewall] --wired-- [switch] --wired-- [wired connection devices based on ports available]
[firewall] --wired-- [switch] – wired-- [ISP router now acting as wireless access point only]
I would recommend option2 as your switch can help you setup VLAN segregation and you connect your downstream devices accordingly which may be one or more wired connections and one or more wireless access points
You need to define NAT for devices on your VLANs to communicate externally… Not sure whether a double NAT scenario would arise… but beyond my knowledge or expertise …
The setup is relatively busy for the amount of clients.
Let’s first establish the goals for this setup. I suggest the following, but please correct and add:
basic connectivity (using wifi or cable)
enhanced security by creating and isolation of LAN networks (VLANs; apparently supported by ISP Router)
enhanced security by isolating in-house devices from internet
The ISP router (typically and according to the diagram) acts as firewall, switch and AP.
Key question: What does the ISP router offer (as a firewall) that OPNsense doesn’t offer? (why) Do you need both?
If the Lab VLAN, IOT VLAN deserve the enhanced protection of an additional firewall, why don’t the family devices and the workstation? Also, I assume the IOT wifi exists in the IOT VLAN and the Lab wifi exists in the Lab VLAN (issue in diagram).
The admin VLAN is typically the most sensitive subnet from a security perspective. How come it is split and has access on both sides of the OPNsense firewall (and don’t you have issues with this setup?) You have your two personal devices connected to the admin VLAN - don’t they have to access (and thereby be part of) the other VLANs (it looks like they do by connecting to Home wifi and Lab wifi - diagram issue?)?
Thanks Mailman and Jode for your great advice! I am very new to networking and so I probably have a bunch of misconceptions, please double check if what I am saying sounds wrong because it probably is… I am keen to learn though, but want to do so in a safe way given the significant risks involved and am very open to your advice.
Mailman:
Your proposed alternative does sound appealing.
I am trying to get my head around the hardware setup for your proposed options. This is dictated by existing ethernet cable runs between rooms and where I can physically store things eg. space for the server rack is only possible in the Server Room.
I am now considering a more dedicated setup for the OPNSense router/firewall. If I can locate it in the server rack in my Server Room, I am thinking of an OPNsense DEC2752. If it has to be located before the ISP router in the Common Room then I can go with a non rack mounted version I think.
I have a server rack in my Server Room which contains a Brocade ICX7250-48P managed switch. Are you suggesting buying an additional managed switch and putting it in my Common Room just after the OPNSense device, which itself connects directly to the ISP provided internet connection? This would then have two outputs, one for my ISP router, and one feeding into the Brocade Switch in the Server Room?
Does this plan hinge on the following points?
The ability to setup my ISP router to only be used as a wireless access point and not do any routing or other functions? I will check but I am not sure if it can allow this or not.
Does this mean that if my OPNSense router goes down eg. I break it via tinkering, then my ISP router and all family devices will will also lose network access since they are downstream of it? This seems less appealing to me, though it may well be a necessary evil.
I wonder if my ISP may want their provided router as the first connection to my network, particularly to assist with other family members debugging with the ISP in my absence or as they change the ISP side of the network eg. deploying fiber to the home?
I also am not sure exactly how to create different wifi networks with different VLANs from the same access point (Ruckus R720). This may well be impossible, indeed.
If you can help me understand the above questions, I am very interested to consider your solution further!
Jode:
You raise good points!
I agree with your suggested goals. I would only add the goal of being able to remotely access and manage the home lab devices in a safe way eg. via VPN with multi factor authentication and other security principles applied.
To clarify: my ISP router does not support VLANs, which is one of the reasons for wanting a dedicated OPNsense switch that does allow this feature along with other things that sounds useful to me eg. IPDS, ability to backup settings for easy restore, etc. My diagram may be unclear in this way, or this may be a problem in my understanding. You are correct to note that the IOT and Lab wifi should be in their respective VLANs.
Yes, the ISP router acts as a firewall, switch, and AP.
Your key question:
This is a good question and quite possibly I don’t need both. Maybe it is better to state what I want to achieve, since my limited understanding may be leading me down the wrong path:
My ISP router currently does a good job of providing wifi connections to the house and many family devices already rely on this. So I would like to keep using this rather than set this all up myself and manage it. I also like that regardless of what tinkering/breaking I do on my own home lab setup, these services should still work.
I don’t expect anyone else on this network to need anything complicated from the network eg. they don’t need remote access to it and should not be doing any port forwarding and so on. My hope then is that the ISP firewall and default setup should be enough to protect this part of the network.
Since I do want remote access, I am hoping that by using my own OPNSense router/firewall I can enhance security eg. with things like IPDS, VLANs, easily able to backup router settings and restore them, etc. I do not like how little control I have over the ISP router.
This is my reasoning for the proposed split between ISP router and my own OPNSense router. Does that make more sense now, or am I going down the wrong path with this?
This seems a very good point and was certainly something I was not sure about in my plan. I will again go back to stating my goals in the hopes we can work out a better approach:
I use my personal devices (laptop and phone) for personal and work purposes. For personal internet browsing I expect I am at a higher risk of attack (eg. accidental malware download or whatever) and so was thinking to limit my personal usage to when I am on the wifi network provided by the ISP and not have a direct connection to the more secure home lab network whilst doing this. Whenever I want to manage the home lab network, I would then switch to the link via the Lab Wifi which my intention was to be protected by the OPNSense router and secured.
Since my ISP router does not allow VLAN setup, I figured I would have to use my dedicated WIFI access point via the lab switch to limit the network access to just devices on the lab Wifi network/VLAN to get this safety. I intended to use rules that allow the Admin VLAN to access the Lab VLAN at this stage.
As I said, this part is not clear in my understanding and I think you have already pointed out its flaws eg. such a setup can directly bypass the OPNSense router which I had not realised.
Hopefully that clarifies a bit more my goals. With that in mind, do you have suggestions for a better config that would achieve this?
Your idea is fine, but there are a few considerations I see.
First, double-NAT is a real issue, especially if you plan to open ports from your lap to the public internet. Check if your ISP router has an option to pass its public IP address to OPNSense (AT&T’s gateways have that option), if you don’t have that option, then you will need to configure the forward-all or DMZ address to point to the OPNSense.
Second, OPNSense by default will see devices connected directly to the ISP’s router as if they were external devices, meaning that they will not be able to directly reach anything in the home lab unless you open ports just like you would need to for internet access. To the extent this is an issue, I would suggest connecting those devices to the home lab instead of trying to overly complicate things.
Third, I am not sure why the purple VPN arrow stops at the ISP router. If you plan to connect OPNSense to a public VPN provider, or if you plan to allow inbound VPN connections from your own devices, either way, the arrow should be shown going directly from OPNSense to the internet. Yes, the traffic flows over the ISP router, but the tunnel goes from OPNSense to the Internet, and that is how it should be drawn.
