Hello everyone, I decided to write this guide which is an amalgamation of all the solutions found on this post by Wendell:
This guide is based on my personal experience on 2 TrueNAS systems, follow at your own discretion. I will try to help if I can, and update this guide if I missed something.
I want to thank the following people:
@wendell for the original guide, and for his help with everything.
A user by the name of Jip-Hop on GitHub, who wrote the guide on how to connect the bridge to avoid VMs not having internet access. And created the original script.
A user by the name of tprelog on GitHub for adapting the Script.
@talung for showing me that the basic Kubernetes implementation is flawed and not efficient, and changes to the script. (See comments)
The first issue this aims to solve, is the fact that TrueNAS Scale uses k3s to run its apps, and that’s both not very performant and not usable for most of us with just one NAS.
The second issue is that VMs on TrueNAS do not see the host, and Wendell showed in his video a solution that is working, but interferes with the solution to the first issue.
The reason I put the solution for VMs here, is because when you implement the script for running Docker natively, they lose access to the internet with the solution Wendell showed.
So a little back and forth with the creator of the script, and I was pointed to Jip-Hop’s post with the solution to have both.
The overhead description:
The solution will look like so:
1. Native Docker and Docker Compose
2. Portainer to manage all other Dockers easily
3. Optional settings and recommendations
4. Virtual bridge to allow VMs to see TrueNAS host shares
Let’s get started!
First off, if you have dockers and apps in TrueNAS, be sure to take note of their volumes and docker-compose file if relevant, as you’ll have to recreate them.
Stage 1—Create the Data set:
This is obvious, but it has some pitfalls that some new users to TrueNAS and Linux in general might not be aware of.
We’ll need to create a Dataset in the pool you use for operations, and not just cold storage. So if you have a SSD pool or a more powerful pool that is dedicated to operations, you’ll choose that one.
In order to do that, in TrueNAS go to Storage and find the pool you will be using.
Click on the 3 dots to the right of the first line in that pool and click on Add Dataset.
Now you’ll see the creation screen, give it a name, and nothing else. It’s Critical that you don’t change the share setting to SMB, but instead leave it as Generic. SMB breaks chmod, which is crucial for dockers.
Now that you have the Dataset, we’ll prepare the groundwork for the move to docker.
Stage 2—Making Sure everything is ready for Docker
First thing’s first, we’ll have to make sure the physical interface has an IP address, and that you can reach TrueNAS on that IP. We do not want to be stuck out of TrueNAS and have to physically do these things on the server.
So go to Networks > Locate the active physical connection, and click on it.
Now, make sure DHCP is off, and add an alias in the following format:
where 192.168.0.2 is the IP TrueNAS is responding to and shows the web portal on.
Test and Save the settings, make sure nothing broke, and continue to delete the br interface.
I’ll start off by clearing up things for people who used or set something in the TrueNAS apps, or tried Wendell’s solution with the Bridge. If you are starting from a fresh TrueNAS, skip ahead.
We’ll have to go to TrueNAS > Apps and stop and remove every app that you may have created. Once that’s done, we’ll go to:
and click on Advanced Settings, there we’ll make sure the IP is not set to a bridge, but to the physical Interface.
Once that’s done, go back to Settings from the image and click on Unset Pool. Give it time, and when it’s done we can move to stage 3.
Stage 3—Getting Docker to run Natively
We’ll create a file somewhere that’s accessible to you, if you want you can do it from TrueNAS shell or from a share.
Go here: Use docker-compose on TrueNAS SCALE without Kubernetes · GitHub
and copy the script.
Create the file, let’s call it enable-docker.sh, on your TrueNAS. Edit line 20 to point to the Dataset you created in stage 1, and run the script.
If you run into issues with it saying something about the bash not being recognized, remove " env" from the first line in the script.
If all goes well, you should be able to check systemctl status docker.service to see that it’s working.
When you’re done, go to System Settings > Advanced > Init/Shutdown Scripts. There choose Script instead of Command, point to the script you just created, give it a name, and tell it to run Post Init.
Now you’re set from the Docker side of things, you could use docker and docker compose commands freely. However, if you wish to do this with Portainer and have a few extra things, please continue reading.
Stage 4—Optional Upgrades
I personally am not a fan of being limited to the version of docker and docker compose that comes with TrueNAS, so I updated all of those, and I’ll describe how:
- Enable apt. Apt is present on TrueNAS Scale, but disabled. Run the following
chmod +x /bin/apt*to enable it.
- Go here: Install Docker Engine on Ubuntu | Docker Documentation and only do set up the repository.
apt update && apt list --upgradable | grep dockerfor each line in the result do apt install $Result where you replace $Result with the package name.
Stage 5 - Portainer
Now that we have docker compose (or docker-compose if you did not use Stage 4) we can create Portainer.
Create a docker-compose.yml somewhere in your pool, and paste this inside:
version: '3.9' services: portainer-ce: read_only: true pids_limit: 4096 mem_limit: 2g image: portainer/portainer-ce:latest container_name: portainer restart: unless-stopped volumes: - '/var/run/docker.sock:/var/run/docker.sock' - /mnt/Path/To/Stage1/Dataset/Portainer/data:/data:rw ports: - '8000:8000' - '9443:9443'
Save, and run
docker compose up -d or
docker-compose up -d depending on your choice in Stage 4.
You should have access to Portainer now on port 9443 of the IP you set for TrueNAS.
Stage 5a—Optional things in Portainer
In Portainer, I have a few things that I prefer to set for ease of use and maintenance.
- Go to Environments>local>Environment Details, and put the IP of the server from Stage 1, then save. With this, you can easily launch containers directly from Portainer.
- You can also set Portainer to not ask for a username and password every 8 hours if you are running securely. You can do that by going to Settings>Authentication and setting it to what you feel comfortable with.
- I like to set up Watchtower to make sure the images and containers are always up-to-date, and I do not need to do this manually. In Portainer create a stack and put this inside:
name: watchtower services: watchtower: container_name: watchtower image: containrrr/watchtower networks: default: null volumes: - type: bind source: /var/run/docker.sock target: /var/run/docker.sock bind: create_host_path: true networks: default: name: watchtower_default
Watchtower runs 24 hours after it’s created by default, and every 24 hours after that.
Stage 6—Enabling VM host share access
For the last part, we’ll create a bridge in TrueNAS to enable the VMs to access the host.
Go to Network > Interface > Add and name it br0. Next, make sure DHCP is off, and give it an alias with an IP that’s outside your real network’s segment. i.e., if your network is 192.168.0.1/24 you can give this bridge 192.168.254.1/24.
Test and save your setting, if all goes well we can move to the VMs.
In Virtualization edit your VM and add another NIC to it, by clicking on the VM you want, Devices > Add. In this NIC, it’s important that you choose to attach to br0, then save.
Start your VM, and you’ll be able to set the IP of the second NIC manually. You’ll have to set it as follows:
Subnet Mask: 255.255.255.0
Once you save that, you should see the shares on the IP 192.168.254.1.
If not, go to the hosts file of your OS, and add the following line:
Save, and it should go there directly from now on.
That’s it, I hope this helps you, and saves you the time I had to put into these issues.