TrueNAS Scale Native Docker & VM access to host [Guide]

You have to understand, this is something you create once, set and forget about. (unless I update the script, then you just replace it and forget it)

To have something like this that happens once in a blue moon to run every reboot for everyone is not really something you need. If you change the folder it is something you do manually, then you should also be aware to delete the link and let it create it again.

Hope that makes sense.

โ€ฆ and that takes how much time, even on old hardware ? Like 0.001 sek?

Anyways, it was just a suggestion because I was surprised it didnโ€™t check.
Thanks again for your work & have a nice day.

I could add a lot of these checks, but I want the script to finish ASAP for novice users so they wonโ€™t think itโ€™s not working.

If itโ€™s something you want to add, feel free to add it on your machine, but even for you, itโ€™s not a case that needs to be checked on every reboot.

Thank you so much for the guide! I had a couple of issues that threw me; such as pasting from windows that threw off the script. But the was fixed with dos2unix.

However the only thing I am stuck on now is creating a network in portainer that connects to the rest of my network and gets DHCP addresses from my OpnSense router.

TrueNAS-SCALE-23.10.2 and latest Portainer-ce

2 Likes

Hey, for that you need to create a macvlan in portainer to bridge to a specific stack, and define that stack as network host.

You can look it up there are a lot of guides online, or you could ask one of the LLMs to summarize it for you.

Good luck!

1 Like

Excellent. Thanks for pointing me in that direction; ill give it a shot!

1 Like

Youโ€™re welcome.

I updated the script but Docker is not found, below is the log and the script I used

enable-docker.log
ยงยง Starting script! ยงยง
ยงยง Checking apt and dpkg ยงยง
ยงยง /bin/apt is already executable ยงยง
ยงยง /bin/apt-cache is already executable ยงยง
ยงยง /bin/apt-cdrom is already executable ยงยง
ยงยง /bin/apt-config is already executable ยงยง
ยงยง /bin/apt-extracttemplates is already executable ยงยง
ยงยง /bin/apt-ftparchive is already executable ยงยง
ยงยง /bin/apt-get is already executable ยงยง
ยงยง /bin/apt-key is already executable ยงยง
ยงยง /bin/apt-mark is already executable ยงยง
ยงยง /bin/apt-sortpkgs is already executable ยงยง
ยงยง /bin/dpkg is already executable ยงยง
ยงยง /bin/dpkg-deb is already executable ยงยง
ยงยง /bin/dpkg-divert is already executable ยงยง
ยงยง /bin/dpkg-maintscript-helper is already executable ยงยง
ยงยง /bin/dpkg-query is already executable ยงยง
ยงยง /bin/dpkg-realpath is already executable ยงยง
ยงยง /bin/dpkg-split is already executable ยงยง
ยงยง /bin/dpkg-statoverride is already executable ยงยง
ยงยง /bin/dpkg-trigger is already executable ยงยง
All files in /bin/apt* are executable
ยงยง apt update ยงยง
ยงยง Linking apt sources to your storage for persistance ยงยง
ยงยง Please note that with this you'll have to update the links manually on your storage when there's an update ยงยง
ยงยง Fix the trust.gpg warnings ยงยง
ยงยง Docker Checks ยงยง
ยงยง Keyrings Exist ยงยง
ยงยง Docker List: ยงยง
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian/ bookworm main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/libnvidia/ bookworm main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/helm/ all main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-security/ bookworm-security main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-backports/ bookworm-backports main contrib non-free
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-debug/ bookworm-debug main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/yarn/ stable main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/nvidia/ bookworm main
deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/pcm/ bookworm maindeb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian bookworm stable
Docker executable not found
ยงยง Which Docker:  ยงยง
ยงยง Docker storage-driver ยงยง
ยงยง Docker daemon.json ยงยง
ยงยง Checking file: /etc/docker/daemon.json ยงยง
ยงยง {
  "data-root": "/mnt/OPS/Docker",
  "storage-driver": "overlay2",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
} ยงยง
ยงยง {
  "data-root": "/mnt/OPS/Docker",
  "storage-driver": "overlay2",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
} ยงยง
ยงยง Which Docker:  ยงยง
ยงยง Docker Version:  ยงยง
ยงยง Script Finished! ยงยง
enable-docker.sh
#!/usr/bin/env bash

# Enable docker and docker-compose on TrueNAS SCALE (no Kubernetes)
# Follow the guide in this post before using this script:
# https://forum.level1techs.com/t/truenas-scale-native-docker-vm-access-to-host-guide/190882
# This script is a hack! Use it at your own risk!!
# Edit all the vars under:
# Vars you need to change
# to point to your storage
#
# Schedule this script to run via System Settings -> Advanced -> Init/Shutdown Scripts
# Click Add -> Type: Script and choose this script -> When: choose to run as Post Init
exec >/tmp/enable-docker.log
# Vars you need to change:
# set a path to your docker dataset
docker_dataset="/mnt/OPS/Docker"
# set the docker_daemon path on your storage for it to survive upgrades
new_docker_daemon="/mnt/OPS/Docker/truenas/daemon.json"
# apt sources persist
new_apt_sources="/mnt/OPS/Docker/truenas/aptsources.list"

echo "ยงยง Starting script! ยงยง"

echo "ยงยง Checking apt and dpkg ยงยง"
for file in /bin/apt*; do
  if [[ ! -x "$file" ]]; then
    echo " ยงยง $file not executable, fixing... ยงยง"
    chmod +x "$file"
  else
    echo "ยงยง $file is already executable ยงยง"
  fi
done

for file in /bin/dpkg*; do
  if [[ ! -x "$file" ]]; then
    echo "ยงยง $file not executable, fixing... ยงยง"
    chmod +x "$file"
  else
    echo "ยงยง $file is already executable ยงยง"
  fi
done
echo "All files in /bin/apt* are executable"

echo "ยงยง apt update ยงยง"
sudo apt update &>/dev/null
echo "ยงยง Linking apt sources to your storage for persistance ยงยง"
echo "ยงยง Please note that with this you'll have to update the links manually on your storage when there's an update ยงยง"
aptsources="/etc/apt/sources.list"
if [[ -f "$aptsources" ]] && [[ ! -L "$aptsources" ]]; then
  cp "$aptsources" "$new_apt_sources"
  mv "$aptsources" "$aptsources".old
fi
if [[ ! -f "$new_apt_sources" ]]; then
  touch "$new_apt_sources"
fi
if [[ ! -f "$aptsources" ]]; then
  ln -s "$new_apt_sources" "$aptsources"
fi
echo "ยงยง Fix the trust.gpg warnings ยงยง"
# Create a directory for the new keyrings if it doesn't exist
sudo mkdir -p /etc/apt/trusted.gpg.d

# Find all keys in the old keyring
for key in $(gpg --no-default-keyring --keyring /etc/apt/trusted.gpg --list-keys --with-colons | awk -F: '/^pub:/ { print $5 }'); do
  echo "Processing key: $key"
  # Export each key to a new keyring file in the trusted.gpg.d directory
  gpg --no-default-keyring --keyring /etc/apt/trusted.gpg --export --armor "$key" >/etc/apt/trusted.gpg.d/"$key".asc
done

# Backup the old keyring
mv /etc/apt/trusted.gpg /etc/apt/trusted.gpg.backup

#Docker Checks
echo "ยงยง Docker Checks ยงยง"
sudo apt install -y ca-certificates curl gnupg lsb-release &>/dev/null

if [[ ! -f /etc/apt/keyrings/docker.gpg ]]; then
  echo "ยงยง Missing Keyrings ยงยง"
  sudo mkdir -m 755 -p /etc/apt/keyrings
  curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  sudo chmod 755 /etc/apt/keyrings/docker.gpg
else
  echo "ยงยง Keyrings Exist ยงยง"
fi
if ! grep -q "https://download.docker.com/linux/debian" /etc/apt/sources.list; then
  echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list >/dev/null
  sudo apt update &>/dev/null
else
  echo "ยงยง Docker List: ยงยง"
  cat /etc/apt/sources.list
fi

Docker=$(which docker)
DockerV=$(docker --version)
DCRCHK=$(sudo apt list --installed | grep docker)
if [[ -z "$Docker" ]] || [[ -z "$DCRCHK" ]]; then
  echo "Docker executable not found"
  sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin &>/dev/null
fi
sudo chmod +x /usr/bin/docker*
sudo install -d -m 755 -- /etc/docker
if [[ ! -f /etc/docker.env ]]; then
  touch /etc/docker.env
fi
. ~/.bashrc
echo "ยงยง Which Docker: $Docker ยงยง"

## set the Docker storage-driver
echo "ยงยง Docker storage-driver ยงยง"
version="$(cut -c 1-5 </etc/version | tr -d .)"

if ! [[ "${version}" =~ ^[0-9]+$ ]]; then
  echo "version is not an integer: ${version}"
  exit 1
elif [[ "${version}" -le 2204 ]]; then
  storage_driver="zfs"
elif [[ "${version}" -ge 2212 ]]; then
  storage_driver="overlay2"
fi

## HEREDOC: docker/daemon.json
echo "ยงยง Docker daemon.json ยงยง"
read -r -d '' JSON <<END_JSON
{
  "data-root": "${docker_dataset}",
  "storage-driver": "${storage_driver}",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
END_JSON

## path to docker daemon file
docker_daemon="/etc/docker/daemon.json"

if [[ ${EUID} -ne 0 ]]; then
  echo "ยงยง Please run this script as root or using sudo ยงยง"
elif [[ "$(systemctl is-enabled k3s)" == "enabled" ]] || [[ "$(systemctl is-active k3s)" == "active" ]]; then
  echo "ยงยง You can not use this script while k3s is enabled or active ยงยง"
elif ! zfs list "$docker_dataset" &>/dev/null; then
  echo "ยงยง Dataset not found: $docker_dataset ยงยง"
else
  echo "ยงยง Checking file: ${docker_daemon} ยงยง"
  if [[ -f "$docker_daemon" ]] && [[ ! -L "$docker_daemon" ]]; then
    rm -rf "$docker_daemon"
  fi
  if [[ ! -f "$new_docker_daemon" ]]; then
    touch "$new_docker_daemon"
  fi
  if [[ ! -f "$docker_daemon" ]]; then
    ln -s "$new_docker_daemon" "$docker_daemon"
  fi

  # Read the current JSON from the file
  current_json=$(cat $docker_daemon 2>/dev/null)
  echo "ยงยง $current_json ยงยง"

  # Check if current_json is empty and if so, set it to an empty JSON object
  if [[ -z "$current_json" ]]; then
    current_json="{}"
  fi

  # Merge the current JSON with the new JSON
  merged_json=$(echo "$current_json" | jq --argjson add "$JSON" '. * $add')
  echo "ยงยง $merged_json ยงยง"
  # Check if the merged JSON is different from the current JSON
  if [[ "$merged_json" != "$current_json" ]] || [[ -z "$current_json" ]]; then
    echo "ยงยง Updating file: $docker_daemon ยงยง"
    echo "$merged_json" | tee "$docker_daemon" >/dev/null
    if [[ "$(systemctl is-active docker)" == "active" ]]; then
      echo "ยงยง Restarting Docker ยงยง"
      systemctl restart docker
    elif [[ "$(systemctl is-enabled docker)" != "enabled" ]]; then
      echo "ยงยง Enable and starting Docker ยงยง"
      systemctl enable --now docker
    fi
  fi
fi

echo "ยงยง Which Docker: $Docker ยงยง"
echo "ยงยง Docker Version: $DockerV ยงยง"

echo "ยงยง Script Finished! ยงยง"

This is the output when I try the command directly with ./enable-docker.sh

./enable-docker.sh: line 94: docker: command not found

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

chmod: cannot access '/usr/bin/docker*': No such file or directory

Version: TrueNAS-SCALE-23.10.2

According to your docker.log the problem is in your apt sources. For some reason it goes to nightlies for everything and the docker entry was not added correctly to it. Can you remove the docker part and run it again to see if it adds it correctly?

I will try once I get home.

Meanwhile the script would be up to this portion right?

enable-docker.sh
#!/usr/bin/env bash

# Enable docker and docker-compose on TrueNAS SCALE (no Kubernetes)
# Follow the guide in this post before using this script:
# https://forum.level1techs.com/t/truenas-scale-native-docker-vm-access-to-host-guide/190882
# This script is a hack! Use it at your own risk!!
# Edit all the vars under:
# Vars you need to change
# to point to your storage
#
# Schedule this script to run via System Settings -> Advanced -> Init/Shutdown Scripts
# Click Add -> Type: Script and choose this script -> When: choose to run as Post Init
exec >/tmp/enable-docker.log
# Vars you need to change:
# set a path to your docker dataset
docker_dataset="/mnt/OPS/Docker"
# set the docker_daemon path on your storage for it to survive upgrades
new_docker_daemon="/mnt/OPS/Docker/truenas/daemon.json"
# apt sources persist
new_apt_sources="/mnt/OPS/Docker/truenas/aptsources.list"

echo "ยงยง Starting script! ยงยง"

echo "ยงยง Checking apt and dpkg ยงยง"
for file in /bin/apt*; do
  if [[ ! -x "$file" ]]; then
    echo " ยงยง $file not executable, fixing... ยงยง"
    chmod +x "$file"
  else
    echo "ยงยง $file is already executable ยงยง"
  fi
done

for file in /bin/dpkg*; do
  if [[ ! -x "$file" ]]; then
    echo "ยงยง $file not executable, fixing... ยงยง"
    chmod +x "$file"
  else
    echo "ยงยง $file is already executable ยงยง"
  fi
done
echo "All files in /bin/apt* are executable"

echo "ยงยง apt update ยงยง"
sudo apt update &>/dev/null
echo "ยงยง Linking apt sources to your storage for persistance ยงยง"
echo "ยงยง Please note that with this you'll have to update the links manually on your storage when there's an update ยงยง"
aptsources="/etc/apt/sources.list"
if [[ -f "$aptsources" ]] && [[ ! -L "$aptsources" ]]; then
  cp "$aptsources" "$new_apt_sources"
  mv "$aptsources" "$aptsources".old
fi
if [[ ! -f "$new_apt_sources" ]]; then
  touch "$new_apt_sources"
fi
if [[ ! -f "$aptsources" ]]; then
  ln -s "$new_apt_sources" "$aptsources"
fi
echo "ยงยง Fix the trust.gpg warnings ยงยง"
# Create a directory for the new keyrings if it doesn't exist
sudo mkdir -p /etc/apt/trusted.gpg.d

# Find all keys in the old keyring
for key in $(gpg --no-default-keyring --keyring /etc/apt/trusted.gpg --list-keys --with-colons | awk -F: '/^pub:/ { print $5 }'); do
  echo "Processing key: $key"
  # Export each key to a new keyring file in the trusted.gpg.d directory
  gpg --no-default-keyring --keyring /etc/apt/trusted.gpg --export --armor "$key" >/etc/apt/trusted.gpg.d/"$key".asc
done

# Backup the old keyring
mv /etc/apt/trusted.gpg /etc/apt/trusted.gpg.backup

#Docker Checks
echo "ยงยง Docker Checks ยงยง"
sudo apt install -y ca-certificates curl gnupg lsb-release &>/dev/null

if [[ ! -f /etc/apt/keyrings/docker.gpg ]]; then
  echo "ยงยง Missing Keyrings ยงยง"
  sudo mkdir -m 755 -p /etc/apt/keyrings
  curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  sudo chmod 755 /etc/apt/keyrings/docker.gpg
else
  echo "ยงยง Keyrings Exist ยงยง"
fi
if ! grep -q "https://download.docker.com/linux/debian" /etc/apt/sources.list; then
  echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list >/dev/null
  sudo apt update &>/dev/null
else
  echo "ยงยง Docker List: ยงยง"
  cat /etc/apt/sources.list
fi

EDIT1: This is the output of the of the above script

enable-docker.log
ยงยง Starting script! ยงยง

ยงยง Checking apt and dpkg ยงยง

ยงยง /bin/apt is already executable ยงยง

ยงยง /bin/apt-cache is already executable ยงยง

ยงยง /bin/apt-cdrom is already executable ยงยง

ยงยง /bin/apt-config is already executable ยงยง

ยงยง /bin/apt-extracttemplates is already executable ยงยง

ยงยง /bin/apt-ftparchive is already executable ยงยง

ยงยง /bin/apt-get is already executable ยงยง

ยงยง /bin/apt-key is already executable ยงยง

ยงยง /bin/apt-mark is already executable ยงยง

ยงยง /bin/apt-sortpkgs is already executable ยงยง

ยงยง /bin/dpkg is already executable ยงยง

ยงยง /bin/dpkg-deb is already executable ยงยง

ยงยง /bin/dpkg-divert is already executable ยงยง

ยงยง /bin/dpkg-maintscript-helper is already executable ยงยง

ยงยง /bin/dpkg-query is already executable ยงยง

ยงยง /bin/dpkg-realpath is already executable ยงยง

ยงยง /bin/dpkg-split is already executable ยงยง

ยงยง /bin/dpkg-statoverride is already executable ยงยง

ยงยง /bin/dpkg-trigger is already executable ยงยง

All files in /bin/apt* are executable

ยงยง apt update ยงยง

ยงยง Linking apt sources to your storage for persistance ยงยง

ยงยง Please note that with this you'll have to update the links manually on your storage when there's an update ยงยง

ยงยง Fix the trust.gpg warnings ยงยง

ยงยง Docker Checks ยงยง

ยงยง Keyrings Exist ยงยง

ยงยง Docker List: ยงยง

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian/ bookworm main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/libnvidia/ bookworm main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/helm/ all main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-security/ bookworm-security main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-backports/ bookworm-backports main contrib non-free

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/debian-debug/ bookworm-debug main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/yarn/ stable main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/nvidia/ bookworm main

deb http://apt.tn.ixsystems.com/apt-direct/cobia/nightlies/pcm/ bookworm maindeb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian bookworm stable

EDIT2: Itโ€™s working now

I just deleted both aptsources.list and daemon.json and ran the original script again, Docker is working now.

When you say original script, you mean the one in the OP?

Correct

Cool, glad it solved it for you.

Has anyone already updated to Dragonfish RC?
Is everything working correctly?
Iโ€™m almost doing the update to test :smiley:

I personally am waiting for the final release, should be this month. But nothing should change assuming you use the latest version of the script. All youโ€™ll need to do is change the truenas links to point to dragonfish.

Do you use any Docker container that uses VGA, like jellyfin?
Iโ€™m trying to use a GTX 1070 to do encoder/decoder, but itโ€™s not working anymore.
I had a 970 and it was working, I took some time off and now I got a 1070, but I canโ€™t get jellyfin to transcoder.

I did try to get librephotos working on GPU, but I broke things, because I updated to the latest NVIDIA driver.

You see in TrueNAS the nvidia kernel is controlled by TrueNAS, and in dragonfish it will be 545, where the latest driver is 550.

So you need to make sure the driver matches the kernel, and run SMI to see that cuda detected correctly.

I just upgraded to dragonfish.
In dragonfish, apt is disabled, so the script fails miserably, lol
Can you tell me how I can activate it again?
Package management tools are disabled on TrueNAS appliances.

In DragonFish can enable apt / toggle โ€œdeveloperโ€ mode by running the command โ€œinstall-dev-toolsโ€ or /usr/local/libexec/disable-rootfs-protection.

I opened the โ€˜disable-rootfs-protectionโ€™ script in /usr/local/libexec/ to see what it does.
Basically it shows the protections that iX has made.
Apparently, just put the install-dev-tools command at the beginning of our script and everything will work as before.
Testing here, and everything seems to work as it should.

The script should already enable apt, apt was always disabled by TrueNAS. Did you let the script run after the upgrade? If so, what is the output of the log?

If you look at the thread you posted, it started all the way back at 2022, when SCALE was released. So itโ€™s not a new thing.