Hello everyone!
I am trying to setup my home network with a few services and I’m hitting a wall and I would love if some of you could guide me in the right direction.
For context: I’m trying to build my home network along with a trunas server hosting a few services. It’s a long project in the making and I already discussed with a few of you some details back in 2022, mostly regarding the hardware. You can find the old topic here: New home = new network. Need help with checking my network ideas. Thank you!
To quickly sum up:
- My ISP provides me a router/modem box with FTTH and a 2.5Gbps connector (2Gbps down/1 Gbps up internet),
- To fully take advantage of said bandwith, I provided a 2.5G Netgear MS108 switch,
- I want to use my own router/firewall with pfsense to segregate devices and services on my network using VLANs, so I bought a 4x2.5G pfsense box from amazon,
- I have a fujitsu workstation with 2.5G and 1G LAN ports to use as a truenas,
- I added a NetGear WAX610 for WiFi6 and seperate SSIDs for seperate VLAN access.
I bought everything back in february, to play with it a bit, test it out, and when I was sure everything worked as intended, I wanted to fully commit then and transfer all my data and services to a new fresh truenas install.
It worked fine for a bit. Earlier this week, I tried a new config on my pfsense that messed everything up. Since I didn’t have a backup for my config, I thought I’d factory reset it and do it all again, since I hadn’t anything too complicated set up. But I now face several issues that I don’t seem to be able to resolve alone (it’s been 3 days of searching) and I would love some help.
So. Where I’m at so far, that partially works and that worked pretty good for months so far:
On my ISP router, I disabled everything. No WiFi, no DHCP, nothing. There’s only my pfsense router connected, with a fixed IP. I declared the pfsense IP/MAC as DMZ on the box.
This port of my pfsense is declared as WAN. One other connector of my pfsense is connected to my Netgear switch, on an uplink port. This port in pfsense is declared as LAN, with local IP 192.168.0.1/24.
In pfsense I created several VLANs, including VLAN 10 “Local” and VLAN 50 “Outbound”. (later there will be other VLANs for IoT devices and such, but baby steps).
As the name suggests, VLAN 10 is for my local and private devices (desktop, laptop, phone, etc.). It is set on 192.168.1.1/24.
VLAN 50 is for my outfacing services hosted on my truenas (Navidrome, Jellyfin, etc.). Admittedly, only the 2.5G port of my truenas, with several IP aliases of my truenas (192.168.5.0/24) is effectively connected to this one.
I set up a few rules in my pfsense firewall. VLAN10 can NAT to access the internet (I’m typing at my desk right now) and cross to VLAN50 for now, for testing. The plan is to segregate later. Works fine. Same for VLAN50. Connects to WAN and NAT, no problems.
I registered a domain with dynu.com to provide dynamic DNS and let’s encrypt ACME SSL certificate for HTTPS. I have a reverse proxy using nginx proxy manager hosted on truenas. I also dabbled with Haproxy directly on pfsense.
Back last week, when everything worked, my pfsense correctly had my domain directing to my IP using dynamic DNS. Then, I used Haproxy to route HTTP/HTTPS 80/443 from WAN to my nginx proxy manager (on a VLAN), that then routed to the correct ip:ports for my several services using wildcards (i.e. navidrome.domain.org).
Now that I re-set everything, nothing works anymore, and I don’t know why because I think I did everything the same but don’t have the same results.
DynamicDNS : when I set up my dynamic DNS client in pfSense and use “interface to monitor : WAN”, it doesn’t work anymore. I did last week, but not anymore. But, if I use VLAN10 or VLAN50, it does. Does it matter? How could I link back my dynamicDNS to my WAN instead of VLAN?
Haproxy doesn’t seem to work at all. As a backend it cannot connect to nginx, and as a frontend, I don’t seem to be able to route to it at all. Either using my domain name or my public IP.
I know direct routing in pfsense works, since I successfully accessed Navidrome with routing its port from WAN to VLAN50, and I hit it with my public IP and port. So routing from WAN to VLAN50 works. But then, even if I try to bypass haproxy on pfsense to use directly nginx, and route all 80/443 directly to nginx, it still doesn’t work.
Is using Haproxy on pfsense to route incomming 80/443 traffic to a separate nginx (as it worked before then) a sane solution? Is there another one?
If possible, I’d like to keep using nginx proxy manager as a reverse proxy service on my network, if I have to choose between them. The self hosted web GUI is nice for a web frontend noob like me.
I don’t know if everything is clear and feel free to ask for more information if needed, I tried to keep it as concise as possible.
Many many thanks for your help.