TrueNAS Scale Native Docker & VM access to host [Guide]

Thank you again for the continued help. I updated to the newest version of the script in the OP. Things seem to be improving, docker launching on EE boot by default now, but no containers or Portainer “Stacks” are initializing on EE.

Below are the logs from the same (new) script being run post init on Cobia and EE:

Cobia Log:

§§ Starting script! §§
+ PACKAGES=(make open-iscsi python3-babel python3-pip python3-pyfakefs python3-pyotp python3-pytest python3-pytest-asyncio python3-pytest-dependency python3-pytest-rerunfailures python3-pytest-timeout snmp sshpass zstd)
+ PIP_PACKAGES=()
+ chmod +x /usr/bin/apt /usr/bin/apt-cache /usr/bin/apt-cdrom /usr/bin/apt-config /usr/bin/apt-extracttemplates /usr/bin/apt-ftparchive /usr/bin/apt-get /usr/bin/apt-key /usr/bin/apt-mark /usr/bin/apt-sortpkgs
+ chmod +x /usr/bin/dpkg
+ apt update

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Hit:1 https://download.docker.com/linux/debian bookworm InRelease
Reading package lists...
Building dependency tree...
Reading state information...
6 packages can be upgraded. Run 'apt list --upgradable' to see them.
+ apt install -y make open-iscsi python3-babel python3-pip python3-pyfakefs python3-pyotp python3-pytest python3-pytest-asyncio python3-pytest-dependency python3-pytest-rerunfailures python3-pytest-timeout snmp sshpass zstd

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
make is already the newest version (4.3-4.1).
open-iscsi is already the newest version (2.1.8-1).
python3-babel is already the newest version (2.10.3-1).
python3-pip is already the newest version (23.0.1+dfsg-1).
python3-pyfakefs is already the newest version (4.6.3-3).
python3-pyotp is already the newest version (2.6.0-3).
python3-pytest is already the newest version (7.2.1-2).
python3-pytest-asyncio is already the newest version (0.20.3-1).
python3-pytest-dependency is already the newest version (0.5.1-5).
python3-pytest-rerunfailures is already the newest version (10.2-2).
python3-pytest-timeout is already the newest version (2.1.0-3).
snmp is already the newest version (5.9.3+dfsg-2).
sshpass is already the newest version (1.09-1+b1).
zstd is already the newest version (1.5.4+dfsg2-5).
The following package was automatically installed and is no longer required:
  libnvidia-nvvm4
Use 'apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
+ '[' 0 -gt 0 ']'
§§ Checking apt and dpkg §§
§§ /bin/apt is already executable §§
§§ /bin/apt-cache is already executable §§
§§ /bin/apt-cdrom is already executable §§
§§ /bin/apt-config is already executable §§
§§ /bin/apt-extracttemplates is already executable §§
§§ /bin/apt-ftparchive is already executable §§
§§ /bin/apt-get is already executable §§
§§ /bin/apt-key is already executable §§
§§ /bin/apt-mark is already executable §§
§§ /bin/apt-sortpkgs is already executable §§
§§ /bin/dpkg is already executable §§
§§ /bin/dpkg-architecture is already executable §§
§§ /bin/dpkg-buildflags is already executable §§
§§ /bin/dpkg-buildpackage is already executable §§
§§ /bin/dpkg-checkbuilddeps is already executable §§
§§ /bin/dpkg-deb is already executable §§
§§ /bin/dpkg-distaddfile is already executable §§
§§ /bin/dpkg-divert is already executable §§
§§ /bin/dpkg-genbuildinfo is already executable §§
§§ /bin/dpkg-genchanges is already executable §§
§§ /bin/dpkg-gencontrol is already executable §§
§§ /bin/dpkg-gensymbols is already executable §§
§§ /bin/dpkg-maintscript-helper is already executable §§
§§ /bin/dpkg-mergechangelogs is already executable §§
§§ /bin/dpkg-name is already executable §§
§§ /bin/dpkg-parsechangelog is already executable §§
§§ /bin/dpkg-query is already executable §§
§§ /bin/dpkg-realpath is already executable §§
§§ /bin/dpkg-scanpackages is already executable §§
§§ /bin/dpkg-scansources is already executable §§
§§ /bin/dpkg-shlibdeps is already executable §§
§§ /bin/dpkg-source is already executable §§
§§ /bin/dpkg-split is already executable §§
§§ /bin/dpkg-statoverride is already executable §§
§§ /bin/dpkg-trigger is already executable §§
§§ /bin/dpkg-vendor is already executable §§
§§ apt update §§
§§ Linking apt sources to your storage for persistence §§
§§ Fix the trust.gpg warnings §§
gpg: keybox '/etc/apt/trusted.gpg' created
§§ Docker Checks §§
§§ Keyrings Exist §§
§§ Docker List: §§
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian bookworm stable
§§ Which Docker: /usr/bin/docker §§
§§ Docker storage-driver §§
§§ Docker daemon.json §§
§§ Storage Driver: overlay2 §§
§§ Dataset: /mnt/OPS/Docker §§
§§ Attempting to create JSON configuration... §§
§§ Generated JSON content: §§
{
  "data-root": "/mnt/OPS/Docker",
  "storage-driver": "overlay2",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
§§ Checking /etc/docker/daemon.json §§
§§ Checking file: /etc/docker/daemon.json §§
§§ Which Docker: /usr/bin/docker §§
§§ Docker Version: Docker version 27.1.1, build 6312585 §§
§§ Script Finished! §§

EE Log:

§§ Starting script! §§
+ FORCE_ARG=
+ [[ '' == \-\-\f\o\r\c\e ]]
+ [[ ! -S /var/run/middleware/middlewared.sock ]]
+ PACKAGES=(make open-iscsi python3-cryptography python3-pip python3-pyfakefs python3-pyotp python3-pytest python3-pytest-asyncio python3-pytest-dependency python3-pytest-rerunfailures python3-pytest-timeout snmp sshpass zstd)
+ PIP_PACKAGES=()
+ '[' -f /usr/local/libexec/disable-rootfs-protection ']'
+ /usr/local/libexec/disable-rootfs-protection
Flagging root dataset as developer mode
Setting readonly=off on dataset boot-pool/ROOT/24.10.1/opt
Setting readonly=off on dataset boot-pool/ROOT/24.10.1/usr
+ '[' 0 -ne 0 ']'
+ apt update

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Hit:1 https://download.docker.com/linux/debian bookworm InRelease
Reading package lists...
Building dependency tree...
Reading state information...
6 packages can be upgraded. Run 'apt list --upgradable' to see them.
+ apt install -y make open-iscsi python3-cryptography python3-pip python3-pyfakefs python3-pyotp python3-pytest python3-pytest-asyncio python3-pytest-dependency python3-pytest-rerunfailures python3-pytest-timeout snmp sshpass zstd

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
make is already the newest version (4.3-4.1).
open-iscsi is already the newest version (2.1.8-1).
python3-cryptography is already the newest version (38.0.4-3).
python3-pip is already the newest version (23.0.1+dfsg-1).
python3-pyfakefs is already the newest version (4.6.3-3).
python3-pyotp is already the newest version (2.6.0-3).
python3-pytest is already the newest version (7.2.1-2).
python3-pytest-asyncio is already the newest version (0.20.3-1).
python3-pytest-dependency is already the newest version (0.5.1-5).
python3-pytest-rerunfailures is already the newest version (10.2-2).
python3-pytest-timeout is already the newest version (2.1.0-3).
snmp is already the newest version (5.9.3+dfsg-2).
sshpass is already the newest version (1.09-1+b1).
zstd is already the newest version (1.5.4+dfsg2-5).
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
+ '[' 0 -gt 0 ']'
§§ Checking apt and dpkg §§
§§ /bin/apt is already executable §§
§§ /bin/apt-cache is already executable §§
§§ /bin/apt-cdrom is already executable §§
§§ /bin/apt-config is already executable §§
§§ /bin/apt-extracttemplates is already executable §§
§§ /bin/apt-ftparchive is already executable §§
§§ /bin/apt-get is already executable §§
§§ /bin/apt-key is already executable §§
§§ /bin/apt-mark is already executable §§
§§ /bin/apt-sortpkgs is already executable §§
§§ /bin/dpkg is already executable §§
§§ /bin/dpkg-architecture is already executable §§
§§ /bin/dpkg-buildflags is already executable §§
§§ /bin/dpkg-buildpackage is already executable §§
§§ /bin/dpkg-checkbuilddeps is already executable §§
§§ /bin/dpkg-deb is already executable §§
§§ /bin/dpkg-distaddfile is already executable §§
§§ /bin/dpkg-divert is already executable §§
§§ /bin/dpkg-genbuildinfo is already executable §§
§§ /bin/dpkg-genchanges is already executable §§
§§ /bin/dpkg-gencontrol is already executable §§
§§ /bin/dpkg-gensymbols is already executable §§
§§ /bin/dpkg-maintscript-helper is already executable §§
§§ /bin/dpkg-mergechangelogs is already executable §§
§§ /bin/dpkg-name is already executable §§
§§ /bin/dpkg-parsechangelog is already executable §§
§§ /bin/dpkg-query is already executable §§
§§ /bin/dpkg-realpath is already executable §§
§§ /bin/dpkg-scanpackages is already executable §§
§§ /bin/dpkg-scansources is already executable §§
§§ /bin/dpkg-shlibdeps is already executable §§
§§ /bin/dpkg-source is already executable §§
§§ /bin/dpkg-split is already executable §§
§§ /bin/dpkg-statoverride is already executable §§
§§ /bin/dpkg-trigger is already executable §§
§§ /bin/dpkg-vendor is already executable §§
§§ apt update §§
§§ Linking apt sources to your storage for persistence §§
§§ Fix the trust.gpg warnings §§
gpg: keybox '/etc/apt/trusted.gpg' created
§§ Docker Checks §§
§§ Keyrings Exist §§
§§ Docker List: §§
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian bookworm stable
§§ Which Docker: /usr/bin/docker §§
§§ Docker storage-driver §§
§§ Docker daemon.json §§
§§ Storage Driver: overlay2 §§
§§ Dataset: /mnt/OPS/Docker §§
§§ Attempting to create JSON configuration... §§
§§ Generated JSON content: §§
{
  "data-root": "/mnt/OPS/Docker",
  "storage-driver": "overlay2",
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ]
}
§§ Checking /etc/docker/daemon.json §§
§§ Checking file: /etc/docker/daemon.json §§
§§ Which Docker: /usr/bin/docker §§
§§ Docker Version: Docker version 27.1.1, build 6312585 §§
§§ Script Finished! §§

Is your dataset called OPS/Docker?
Also, did you try to bring up portainer with docker compose up -d? if so, what was the output?

Confirming that the dataset is /mnt/OPS/Docker .

when running docker compose up -d in EE it returns the below:

no configuration file provided: not found

I’m at a bit of a loss as to why the same version of docker in Cobia picks up the config from the same script/location, but in EE it fails.

Are you running the compose up in the folder where the portainer yml is?

:man_facepalming: no. With that, i was able to launch Portainer; however, no containers have migrated. I believe this is due to a daemon issue; under EE Portainer shows its root directory as /var/lib/docker (in the Host Overview settings), while under cobia the root directory is, as expected /mnt/OPS/Docker.

Perhaps this would be resolved by running docker compose up -d from /mnt/OPS/Docker/Portainer/data/compose/ (vs /mnt/OPS/Docker)

Note: this did not help

You mean here:
echo “§§ Attempting to create JSON configuration… §§”
read -r -d ‘’ JSON <<END_JSON
{
“data-root”: “$docker_dataset”,
“storage-driver”: “$storage_driver”,
“exec-opts”: [
“native.cgroupdriver=cgroupfs”
]
}
?

Because data root should be what you set in the top of the script. Try deleting that json and running the script again. Something is very weird with your system there.

Also, this is not all the images you should have in apt.

no, sorry, i was speaking specifically to what i’m seeing in portainer as the “root directory”.

I navigated to /mnt/OPS/Docker, and ran docker compose up -d from that location. Despite that, the “root directory” listed in portainer remained /var/lib/docker instead of /mnt/OPS/Docker (which is the case on my old configuration). The location at the top of the script remains docker_dataset="/mnt/OPS/Docker" in the script in both cases.

I can simply try to delete my entire non-working EE partition and update trains again from Cobia to try to clean things up if we think that would work.

I was able to delete my EE boot partition and upgrade lineraly from Cobia → DF → EE and get this working as expected. Once EE booted, i simply had to manually run systemctl start docker and then navigate to the appropriate /path/to/Docker folder and run docker compose up -d.

The only remaining issue is that Docker does not start on EE boot and i must manually run systemctl start docker at each reboot. I’m working to find a resolution

Did you make sure the script is running in post init now? Because that should take care of that.

Yes, docker actually does start at boot due to running the script post init; however, the containers are not present until running a systemctl restart docker command. All containers are set to restart: unless stopped

that is weird. then just add that command at the end of the script and you should be fine.