Intro
This guide is meant to go with The Forbidden Router video series, and this is part 2, building on your XCP-ng config from part 1:
What’s the goal?
To configure SteamCache/Lancache for game caching, PiHole for DNS filtering and to make sure everything is as fast as possible.
It’s always DNS
We have to have a chat about DNS. These things work off of DNS magic, and DNS lookups depend on a hierarchy. We’re adding more lookup steps to DNS which can negatively impact performance, so we need to measure and monitor DNS performance.
This is handy to know by itself, even if you don’t plan to do this, because DNS being wonky can lead to all sorts of problems. If your DNS is sub-par but your network connection is otherwise amazing, it could change how you experience surfing the internet entirely if you move from a slower DNS experience to a faster one.
But I digress…
Setting up the VM under XCP-ng
… we already setup pfSense as or router, now we need a VM to run our containers. I picked AlmaLinux 9, which just came out, to be able to run native Docker. Debian was my second choice, with Ubuntu being my third. CoreOS would have probably been my first pick but some things have been happening around RedHat/CentOS lately.
I connected to xcp-ng via ssh and used wget to save the iso in the iso storage directory for AlmaLinux:
Note: The steam cache will be large. Perhaps terabytes large. I suggest you think about how you do storage.
You can make one huge disk if you don’t want to think about it – 2-3 terabytes? 5 terabytes? Whatever makes sense for your setup. I would recommend setting up a separate disk and setting that disk up in AlmaLinux at /opt or /storitron or something like that, and then we will tell Docker to store the SteamCache volume at that path. Think before doing.
My setup uses about 40tb of cast-off enterprise flash, and I set it up at /storitron so it is separate from the virtual hard disk we’re installing to. Don’t worry if you forgot – it is easy to add a new disk and format it inside this VM later.
Installation is straightforward:
Once that’s done update, and reboot before doing anything
dnf -y update
reboot
dnf install -y yum-utils device-mapper-persistent-data
# This might change from /centos/ to /almalinux/ I've been told?
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
dnf install docker-ce -y
systemctl start docker && systemctl enable docker
At this point, let’s run the hello-world docker container to see if everything is working fine?
Great, that is working well on AlmaLinux 9. One last thing – I recommend you add “regular user” non-root account to the docker group. Why? So you can do docker cli stuff w/o root. It’s easy:
sudo usermod -aG docker username-goes-here
I used w
for my username, so I’d make the last argument of the command w. You might have chosen a different username, which goes there.
If you run ip -4 a
you should see any network interfaces that were automatically configured, complete with DHCP IP address. Because of what we’re going to use this for, let’s reconfigure the network with a static IP address on the lan, outside the DHCP range.
In my case 192.168.1.2 is a free IP address, and easy to remember.
nmtui
is a command line “gui” for configuring the network. Ofc you can edit files in /etc if you prefer, but this is relatively newb friendly and doesn’t require you to remember anything, other than typing nmtui
in at the CLI as root.
After making the change, de-activate and re-activate the connection. (Don’t deactivate the connection via ssh! You’ll be disconnected. You can reboot if in doubt.)
re-run ip -4 a
and you should see the new static IP address you set.
Now let’s install Portainer. What is portainer? It’s a gui for managing containers.
( Hey, TrueNAS team, pay attention here )
# this is a persistent data volume for the container. It survives updates and
# container replacement
docker volume create portainer_data
# this is the container itself
docker run -d -p 8000:8000 -p 9443:9443 --name portainer \
--restart=always \
-v portainer_data:/data \
-v /var/run/docker.sock:/var/run/docker.sock \
portainer/portainer-ce:latest
More info on portainer is in their docs here.
This is the output of running the above command and then docker ps
to confirm it is running.
Portainer should now be accessible at https://(ip you set):9443/ (with a self-signed encryption certificate. It is okay to accept it.)
It’ll ask you to set a secure password, then you should be taken here:
Next we need to setup PiHole and LanCache.
In the hierarchy your machine will query Steam, Steam will query PiHole and PiHole will query the fastest DNS server that we were able to find during the DNS diagnostics way back at the start of the video.
We’ll start with PiHole
The best place for PiHole docs is on github.
They even have a docker run script!
The docker run script is simpler than using the portainer gui to setup the container:
Jeez, look at this UI. All just so we can have an equivalent gui for:
docker run -d \
--name pihole \
-p 192.168.1.2:53:53/tcp -p 192.168.1.2:53:53/udp \
-p 192.168.1.2:80:80 \
-e TZ="America/New_York" \
-v "${PIHOLE_BASE}/etc-pihole:/etc/pihole" \
-v "${PIHOLE_BASE}/etc-dnsmasq.d:/etc/dnsmasq.d" \
--dns=127.0.0.1 --dns=1.1.1.1 \
--restart=unless-stopped \
--hostname pi.hole \
-e VIRTUAL_HOST="pi.hole" \
-e PROXY_LOCATION="pi.hole" \
-e ServerIP="127.0.0.1" \
pihole/pihole:latest
… and yes, unlike some other stuff out there (cough cough TrueNAS) this gui will let you completely configure these CLI options.
… But this Is Stupid. Isn’t there a way to just paste the text and have it parsed? I mean I can just SSH in, paste this command and be done in 12 seconds. If I have to use this GUI, it is going to take 10 minutes.
Some poor soul spent days, weeks maybe, building this UI.
Fortunately, Podman has Stacks, which is very similar to docker compose
and you can paste stuff right in. Sanity wins the day for now! (pay attention here again TrueNAS devs…)
… and with the container running, the pihole is accessible at the IP.
Note that you must change the binding from the default to something like this:
- "192.168.1.2:53:53/tcp"
- "192.168.1.2:53:53/udp"
- "192.168.1.2:67:67/udp" # Only required if you are using Pi-hole as your DHCP server
- "192.168.1.2:80:80/tcp"
See how we’re explicitly adding the IP to bind to? Otherwise docker would bind all IPs on this system on port 53, 80, etc. But we’re going to have other stuff running on other IPs on this system.
Note: If you want to bind it to a different IP, you can, in the docker-compose we pasted. It isn’t necessary for the pi-hole, though. It’ll run off of the IP you setup earlier (192.168.1.2 in my case).
Speaking which, we’re out of IPs on our host machine! We used the one IP we setup earlier, so we should setup another IP for the steam cache since it will also want to use ports 53 and port 80.
use nmtui
to add another IP. I added 192.168.1.3, but you can add anything on your local network that you like.
Configuring Steamcache/Lancache.
The stacks feature of portainer is awesome. I hope other open source projects copy the e-z copy-paste/use git url approach!
We’ll use that to configure Lancache. From their docs:
This docker container provides a caching proxy server for game download content. For any network with more than one PC gamer in connected this will drastically reduce internet bandwidth consumption.
The primary use case is gaming events, such as LAN parties, which need to be able to cope with hundreds or thousands of computers receiving an unannounced patch - without spending a fortune on internet connectivity. Other uses include smaller networks, such as Internet Cafes and home networks, where the new games are regularly installed on multiple computers; or multiple independent operating systems on the same computer.
This container is designed to support any game that uses HTTP and also supports HTTP range requests (used by Origin). This should make it suitable for:
** Steam (Valve)*
** Origin (EA Games)*
** Riot Games (League of Legends)*
** Battle.net (Hearthstone, Starcraft 2, Overwatch)*
** Frontier Launchpad (Elite Dangerous, Planet Coaster)*
** Uplay (Ubisoft)*
** Windows Updates*
This is the best container to use for all game caching and should be used for Steam in preference to the lancachenet/steamcache and lancachenet/generic containers.
Their quickstart guide is handy, but don’t follow it exactly for our setup here.
Head over to stacks in portainer, create a new stack, and then:
version: '2'
services:
dns:
image: lancachenet/lancache-dns:latest
env_file: .env
# restart: unless-stopped
ports:
- ${DNS_BIND_IP}:53:53/udp
- ${DNS_BIND_IP}:53:53/tcp
## HTTPS requests are now handled in monolithic directly
## you could choose to return to sniproxy if desired
#
# sniproxy:
# image: lancachenet/sniproxy:latest
# env_file: .env
# restart: unless-stopped
# ports:
# - 443:443/tcp
monolithic:
image: lancachenet/monolithic:latest
env_file: .env
# restart: unless-stopped
ports:
- 192.168.1.3:80:80/tcp
- 192.168.1.3:443:443/tcp
volumes:
# setup paths that make sense on your host, this one
# is for mine
- ${CACHE_ROOT}/cache:/data/cache
- ${CACHE_ROOT}/logs:/data/logs
But we’re not done, we also need that .env file, that’s where {BLAHBLAH} variables come from:
Here’s the contents. You can hit advanced, then paste this in (or upload the file if you prefer).
NOTE you will have to edit these values to make sense for your setup! How big is the cache? What IPs are you binding it to?
In my case
192.168.1.2 > PiHole IP and host IP
192.168.1.3 > IP for LanCache and LanCache DNS.
The .env file I use:
## See the "Settings" section in README.md for more details
## Set this to true if you're using a load balancer, or set it to false if you're using seperate IPs for each service.
## If you're using monolithic (the default), leave this set to true
USE_GENERIC_CACHE=true
## IP addresses that the lancache monolithic instance is reachable on
## Specify one or more IPs, space separated - these will be used when resolving DNS hostnames through lancachenet-dns. Multiple IPs can improve cache priming performance for some services (e.g. Steam)
## Note: This setting only affects DNS, monolithic and sniproxy will still bind to all IPs by default
LANCACHE_IP=192.168.1.3
## IP address on the host that the DNS server should bind to
DNS_BIND_IP=192.168.1.3
## DNS Resolution for forwarded DNS lookups
UPSTREAM_DNS=192.168.1.2
## Storage path for the cached data
## Note that by default, this will be a folder relative to the docker-compose.yml file
CACHE_ROOT=/storitron
## Change this to customise the size of the disk cache (default 2000000m)
## If you have more storage, you'll likely want to increase this
## The cache server will prune content on a least-recently-used basis if it
## starts approaching this limit.
## Set this to a little bit less than your actual available space
# 20 tb is wendell's setup, 2tb is 'default'
CACHE_DISK_SIZE=20000000m
## Change this to allow sufficient index memory for the nginx cache manager (default 500m)
## We recommend 250m of index memory per 1TB of CACHE_DISK_SIZE
CACHE_INDEX_SIZE=500m
## Change this to limit the maximum age of cached content (default 3650d)
CACHE_MAX_AGE=3650d
## Set the timezone for the docker containers, useful for correct timestamps on logs (default Europe/London)
## Formatted as tz database names. Example: Europe/Oslo or America/Los_Angeles
TZ=America/New_York
Now with this in place we can re-run dnsbench
or the DNS benchmark from Steve Gibson and see how we’re doing.
… looks like I’ll be setting 129.250.35.251 and .250 to be the upstream DNS! This graph is a bit misleading because one of the DNS servers was so slow it poisoned the graphing. Just hover the mouse over or re-run the test. It is also sorted as fastest.
Based on our result, for uncached entries, we add about 0.015 time to the lookup request.
Reconfiguring DHCP
With the DNS servers up and working, the final step is to reconfigure DHCP on the LAN. We can reconfigure DHCP to hand out the IP address of the lancache container for DNS, instead of the default.
On pfSense this is very easy – just under the DHCP server settings.
In my case I will set it to 192.168.1.3
The full hierarchy is 192.168.1.3 > 192.168.1.2 > 129.public.dns.server – 3 levels. Previously it was 192.168.1.1 (pfsense, openwrt, or your router) > ISP dns server. So this is one extra DNS “hop”
However, from the old ISP DNS server plus the increased size of the local cache, it is still a net gain overall.
Don’t forget to set the VM in XCP-ng to auto-start:
It is “critical” this VM be up in order for your network to function.
Similarly, make sure all the containers also auto restart, including portainer:
Making Sure It All Works
For Lancache Monolithic, just connect to the console and run the command:
tail -f /data/logs/access.log
EZ web-gui for troubleshooting. Here we can see me downloading DeathLoop and getting mostly cache misses.
If you don’t see anything here, double check your IP addresses are set. You can try manually setting your DNS server to the IP of your lancache machine and then do some game downloading in steam to see if anything shows up.
From there, you can also check PiHole, as it should have some stats as well.
Congratulations! You’re living in the future.