Setting up DNS and HTTPS for locally-accessible services - using PFsense

Hardware Hub Networking
, , , ,
1

Background:

  • I have a few services (HomeAssistant, Proxmox, PFSense, Trilium notes, and others) that are only accessible locally and currently lack proper SSL certificates (HTTPS). I’d like to start self-hosting Bitwarden (which requires a proper SSL certificate to work), as well get valid HTTPS certs for the other services that I host.

    Problem:

  • All of these services are accessible only via an HTTPS connection in a web browser. An additional problem is that many of these services are hosted from a single server, but use various ports (ex. Proxmox is port 8006, HomeAssistant is port 8123, Trilium notes is port 9005).

My desired goal:

  • I want to be able to access all of these services via a simple DNS name, and not need to enter a port number when entering the DNS name into my browser. Additionally, I want all of these services to have valid SSL certificates (possibly provided via LetsEncrypt?).
  • For example, proxmox at 192.168.5.5:8006 = pr.example.ca , pfsense at 192.168.5.1:443 = pf.example.ca , homeassistant at 192.168.5.5:8123 = ha.example.ca , Trilium at 192.168.5.5:9005 , etc.
  • Additionally, all of these services will be only accessible when connected to my internal network. I do not want want to use PFsense as the “certificate authority”, as installing the PFsense root certificate onto every device would be annoying.

What I’ve currently done:

  • I’ve purchased a domain from Cloudflare, because, as far as I know, DuckDNS or other free DynamicDNS providers do not allow for sub-sub domains (correct me if I’m wrong?)
  • I’ve installed the ACME + HAProxy packages in PFsense
  • The ACME package in PFsense is linked to my Cloudflare account, so I think I can get proper LetsEncrypt certificates
  • As I understand it, HAProxy is required to convert, for example, homeassistant at 192.168.5.5:8123 , to simply be ha.example.ca (thereby essentially stripping the port number)

Help?

  • I’ve tried following this guide by Lawrence Systems, as well as this guide on flemmingss.com (involves setting up Virtual IP’s for HAProxy) on doing what I want to accomplish, but both methods have been unsuccessful for me, likely because what I’m trying to do is more complex.

Maybe someone can link a more-appropriate guide for what I want to do, or suggest something?

1 Like
2

What issues are you running into, specifically?

I run Caddy as my reverse proxy, but otherwise our setup is quite similar: Cloudflare, xxSense (I use opn, vs pf). DNS challenge setup is fairly straightforward. Something else to consider if HAProxy on PFsense ends up a dead end.

1 Like
3

No need to buy MITM services (Cloudflare) or public domain names, if accessing internal services via HTTPS is what you’re after.

Unless you mean you want to be able to access your internal services, from anywhere in the world?

For local infrastructure, use certbot + nginx (local TLS termination + vhost reverse proxy) in combination with step-ca:

Setup e.g. the whole .internal domain in your DNS server to point to the VM/container/machine hosting the nginx server.

4

Sounds like you are looking for a reverse proxy to sit in front of your existing services as another user said. However your mental model is not quite correct, not stripping the port so much as resolving all requests for ha.example.ca to 192.168.5.5:8123 per your example. The traffic is forwarded directly at the ip:port specified in the proxy configuration. This is sort of important as it results in client ips appearing as coming from the proxy itself unless your applications are reverse proxy-aware.

5

When I followed the Lawrence Systems guide, I simply could not apply the changes to the HAProxy frontend, as it was reporting errors. For the flemmingss guide, I was able to setup HAProxy, but oddly none of the redirects were working correct. For example, if I typed ha.example.ca or ha.example.ca:8123, the webpage would not load whatsoever. I’m doubtful I missed a step for either guide, as I retried following their guides 3x each.

As I said in my (long) post

I do not want want to use PFsense as the “certificate authority”, as installing the PFsense root certificate onto every device would be annoying.

The smallstep.com guide appears to require exactly that; installing my own custom cert authority onto all devices.

Yeah I think you’re right here with all of this, my apoligies. I’ve never really been able to fully comprehend the whole SSL/TLS/HTTPS/Certificate stuff, even after looking at it multiple times over the years.

Maybe you’ll have a suggestion on a guide I can follow for exactly this? Or maybe you can first answer whether I even need to go the Virtual IP route with HAProxy, as my earlier flemmingss.com link did?

6

I created a module for caddy and opnsense to do exactly this:

It automatically adds dns host / host overrides to dnsmasq / unbound if you add a new entry to your caddy file. TLS certs are loaded via DNS-01 challenge. So you don’t expose anything.

I also forked caddy ddns and added a feature to cleanup unused entries, you can use this with the docker module for dynamic service discovery / deletion.

If I find some time I might do a write up of all this.

PFsense has a similar API, the code would be easy to port.

3 Likes
7

IMHO that is an easy task, but AI took you for a ride.

Option 1 assuming you have a domain:

On the application itself, run ACME DNS-01 challenge.
So for example, your Proxmox does get the cert for proxmox.internal.yourdomain.com

Then you got to the unbound server of your pfSense and create an override.
You say that proxmox.internal.yourdomain.com should translate to 192.168.1.10 or whatever the IP of Proxmox is.

If you try to open proxmox.internal.yourdomain.com outside of your network, nothing will happen, since that domain does not exist. If you insert it at home, your DNS will answer https://proxmox.internal.yourdomain.com with 192.168.1.10 and Proxmox will show you a valid Let’s encrypt cert.

Option 2 assuming you don’t a domain:
Get a domain. It is what 15$ a year. Otherwise you have to deal with self signed certs. Works, but is a PITA. Not worth it IMHO.

Option 3 assuming your service does not support ACME:
Get a reverse proxy. The reverse proxy will have a valid cert. The reverse proxy will redirect traffic to your http service. You still do an DNS override but point to the proxy instead of the service. So for example, your proxy runs on 192.168.1.2. Now you set proxmox.internal.yourdomain.com to 192.168.1.2 (your proxy) which will then proxy to http://192.168.1.10:8006. Your client does not know this, it will just see the valid cert of your proxy.

1 Like
8

Set up a reverse proxy to allow https access inside your firewall. You don’t need a cloudflare domain for this (but you can use it).

Don’t configure any of these services to be accessed directly from the interwebs. If you want to access them from the internet setup a VPN into your home. Lawrence has some great videos on how to set this up easily - but this is a totally separate topic and should not be confused.

Specifically for pfsense:

Specifically for proxmox

  • configure your cluster to use a specific ACME service to request and renew certificates
  • create a certificate for your host(s). Proxmox will auto renew
  • Certificate Management - Proxmox VE
9

With your Caddy-DNS-OpenSense project, will it work with essentially all web services which require SSL/TLS encryption?

10

Clarify?

Do you have a guide on doing exactly this? The two guides that I tried following earlier did not appear to work for me.

Isn’t this exactly what I talked about doing with HAProxy?

______________________________________________________

I can see people are providing me lots of suggestions on what exactly to do, but again

______________________________________________________

Maybe someone can even take a look at the guides that I linked earlier and make suggestions on something I can deviat from in the guide so that I can do exactly what I want to accomplish?

11

All that support to be run behind a reverse proxy. I never seen anything that couldn’t be configured to run behind a reverse proxy, so I would guess:

Yes, sometimes with some configuration, e.g. stetting something like allow-origin or trusted-proxy etc.

Edit: @TheAlmightyBaconLord Did a quick and dirty port based on the open-api.spec :smiley:

This is untested!

Build with:

xcaddy build \
    --with github.com/mietzen/caddy-dns-pfsense \
    --with github.com/mholt/caddy-dynamicdns \
    --with <YOUR-DOMAIN-REGISTAR> # https://github.com/caddy-dns

Caddyfile:

{
	acme_dns <YOUR-DOMAIN-REGISTAR> {
		api_key {env.DOMAIN_REGISTAR_API_KEY}
	}
	email {$ACME_EMAIL}
	dynamic_dns {
		provider pfsense {
			host {env.PFSENSE_HOSTNAME}
			api_key {env.PFSENSE_API_KEY}
			insecure true
		}
		domains {
			{$BASE_DOMAIN}
		}
		dynamic_domains
		ip_source static {$CADDY_SERVER_IP}
		check_interval 30m
		ttl 1h
		versions ipv4
	}
}

.env:

BASE_DOMAIN= # example.org
CADDY_SERVER_IP= # 192.168.1.10
ACME_EMAIL= # [email protected]
PFSENSE_API_KEY= # SOMEAPIKEY
PFSENSE_HOSTNAME= # 192.168.1.1 or pfsense.internal
DOMAIN_REGISTAR_API_KEY= # SOMEOTHERAPIKEY
12

Just to be clear, you are only looking for a solution that works ONLY on LAN? You’re not trying to remote in here? If you’re ok with using self-signed certificates, you can just get a caddy container up and running…

Create a domain override in pfsense for your domain and point the A record at the IP address of the caddy container. You should put all your other containers on a separate docker network and stop exposing their ports. Then you can reference them by container_name:port in the caddy config.

Then create caddyfile entries with subdomains per the linked tutorial for each of your apps.

If you need let’s encrypt I believe you can do a DNS record challenge. In that case, use something like cloudflare (it’s free…) to automate the ACME DNS challenge. I would still do the pfsense dns entry for Caddy, however. This should get you valid certs without having to expose anything online.

1 Like
13

What is there to clarify?
It is a very easy task, if you know the what and the why.
I gave you that.
AI apparently did not, that is why you struggle, you don’t understand the basics or try to do all at once. All at once is hard to do and painfuly to troubleshoot.

I told you in the previous post. What is your question?

[quote=“TheAlmightyBaconLord, post:10, topic:249286”]
Isn’t this exactly what I talked about doing with HAProxy?
[/quote] Yes. HAProxy is a reverse proxy.

IMHO way to complicated and sometimes misleading suggestions.

Start slow. Do one thing after another. You have a domain, right?
Start with one service. Use ACME DNS challenge for that service. Then unbound override to point to that service.

Do you next service. If the service does not have ACME, use a proxy for that. But here again, one step after another. Create an DNS overwrite to your proxy. Try to proxy form there to a static Apache webpage first for example. After that worked, get a cert for the proxy.

1 Like
14

Personally, I think it will be more parsimonious to set up a reverse proxy and configure certificates for the proxy rather than each individual service. I also think the DNS challenge is the way to go as it avoids all the nonsense of having to punch a hole in LAN for ACME http challenge.

1 Like
15

what you are trying to do is basically a reverse proxy + internal DNS + Let’s Encrypt via DNS challenge setup. With pfSense + HAProxy + ACME, the clean approach is: use HAProxy to map subdomains (ha.example.ca, pr.example.ca, etc.) to the correct internal IP:port, and use ACME with Cloudflare DNS validation so you can issue valid certs without exposing services publicly. Then, set up DNS overrides in pfSense so those subdomains resolve to your HAProxy IP internally. That way you get clean URLs no ports and proper HTTPS. If HAProxy keeps fighting you, an easier alternative is something like NGINX Proxy Manager or Caddy which handle certs and routing with much less manual config.

2 Likes
16

If i recall, nginx proxy manager (npm) has a really nice user interface that makes the whole process pretty simple

17

Personally I think self-signed certificates for this is the way to go. Yes, you have to copy the root certificate onto every device, but unless you have something like 100+ devices (I did it to 8-10 devices on my network and it took about 15 minutes) that sounds a whole lot less painful than what you’re going through to avoid that single one-time hassle (just set the self signed cert to expire in 70 years). And you don’t have to set up a public domain, or rely on anything external.

Anyway. Self-signed certs. Caddy as reverse proxy. Set up your subdomains in your /etc/hosts, pointing at the IP of the box the service is on, and let caddy direct it to the necessary internal port. Easy peasy.

19

I think the opposite. I don’t see a reason on why my Proxmox should be dependant on a reverse proxy, when it comes with a perfectly fine ACME client.

Punch a hole isn’t the problem with http challenge.
I could even argue that http challenge is more secure, because if my DNS provider does not allow for sophisticated rights management, DNS imposes a greater risk if one host gets compromised. I also use DNS, but simply because it is simpler to setup, because I don’t have to open incoming IPv6 port 80 for each host.

It is the opposite. DNS challange is way simpler. Some browsers (Safari) also already don’t accept certs with a longer lifetime than 90 days. So this is definetly not the way to go for a newcomer.

20

Yeah… that’s what I was saying.

How do clients resolve pve.yourdomain.tld? Presumably you run a local dns server for such a purpose. So what is the difference between dns and a reverse proxy? Over all, setting up one acme challenge for the proxy is clearly going to be less maitenance overhead than having to eg. set up a dozen different cron jobs

21

OPNsense unbound DNS server.

Not a huge difference.
Reverse proxy I have to reboot from time to time. Maybe the reverse proxy also runs on a Hypervisor that needs a reboot. And all traffic has to go trough the reverse proxy. You could add a second DNS server, which is simpler than load balance a proxy, I am too lazy for boht.

IMHO Biggest difference:
Adding a new service is ACME & override vs. Proxy config & cerbot which is IMHO a little bit less work. Also you don’t have unencrypted traffic in your local network, in case something else is compromised.