Blog for historical context of my self-hosting journey

[Tekken_boy]

Making this post to document my own journey and troubles in regards to self hosting. Anything new learned or problems will be posted for educational purposes.

PC Specs: 2970WX, Gigabyte X399 Designare Ex, 32Gb Corsair 3000MHz, 3070Ti 8Gb

[Tekken_boy]

For sensor monitoring
‘sudo apt-get install lm-sensors’

Install CPU stress test: command s-tui to run
‘sudo apt install --install-suggests s-tui’

Installed curl to install tailscale
‘sudo apt install curl’

Install tailscale and add it to the list of machines
‘curl -fsSL https://tailscale.com/install.sh | sh’
‘sudo tailscale up’

Install open ssh client for remoting into server
‘sudo apt install openssh-client’

Open port 22 to relevant Tailscale computers
‘sudo ufw allow ssh’
‘sudo ufw allow from IP to any port 22’
‘sudo ufw enable’
‘sudo ufw status’
Listed port being open to anywhere and Tailscale to main PC
CANNOT CONNECT VIA MAIN PC

Typed command Tailscale up… Might help.

Reset firewall settings. I want to configure the firewall to ONLY allow Tailscale addresses from accessing port 22 for ssh on 15Apr26

Install virtual box to configure VMs for testing.
‘sudo apt-get onstall virtualbox’

Installed Ollama
‘curl -fsSL https://ollama.com/install.sh | sh’
Possible models to pull
Gemma4 e4b model: pulled on 0136 20Apr26
qwen3.5 2b, 4b, 9b

Installed Flatpack and GreenWithEnvy using the following commands after installing NVidia drivers
sudo apt install -y nvidia-driver-580
‘sudo apt install flatpak’
restarted PC then the following
'flatpak --user remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo’
‘flatpak install flathub com.leinardi.gwe’

Not functioning properly. Ability to change fan cruve settings, but 1060 3Gb doesn’t respond. Doesn’t work for 3070Ti either?

Installed ollama service to tailscale IP
‘sudo systemctl edit ollama.service’
Editted upper section of service file to add the following
‘[Service] Environment=“OLLAMA_HOST=IP”’
Restarted Ollama after that with:
‘sudo systemctl restart ollama’
After that, I can now access Ollama on “this website”

I then searched for Docker packages with the first line and installed Docker with the second.
‘sudo apt update && apt-cache search docker’
‘sudo apt install docker.io’
I enabled Docker with:
‘sudo systemctl enable docker’
Installed the Nvidia Container Toolkit using this: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Configured the production repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed ‘s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g’ |
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

(Have no idea what I’m doing at this point in regards to NVIDIA Drivers and packages)

Update and upgraded
Then installed NVIDIA Container Toolkit packages:
Followed this website: https://www.robwillis.info/2025/05/ultimate-local-ai-setup-guide-ubuntu-ollama-open-webui/

Now install NVidia Docker packages
‘sudo apt install nvidia-container-toolkit’
Configured the container runtime
‘sudo nvidia-ctk runtime configure --runtime=docker’
Restarted Docker with
‘sudo systemctl restart docker’

OpenWebUI
‘sudo docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main’

Error message of Docker: invalid reference format
Might not have pulled the Docker image. Did it after the error message
‘sudo docker pull ghcr.io/open-webui/open-webui:main’
Still an error message. Trying Option #2 from the website
‘sudo docker run -d -p 80:8080 --network host --restart always --gpus all -e OLLAMA_BASE_URL=http://127.0.0.1:11434 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main’

Modified the IP to be the Tailscale ID….

‘sudo docker run -d -p 80:8080 --network host --restart always --gpus all -e OLLAMA_BASE_URL=TS-IP -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main’
docker: Error response from daemon: Conflict. The container name “/open-webui” is already in use by container “520e7fd37a3ae9423e494b4f7b68e3c1d21055558514b6cef823213ead90cc50”. You have to remove (or rename) that container to be able to reuse that name.

Ran command ‘sudo docker container prune’ and deleted containers that were lingering from the original pull? command or run? Command. Can also run ‘sudo docker container ps -a’ to show containers that have stopped/hidden?

Re-ran the following command:
‘sudo docker run -d -p 3000:8080 --add host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main’

OpenWebUI now functions. Information saved accordingly. OpenWebUI loads with TS-IP

[Tekken_boy]

Need to add a model. Ollama isn’t connect to the IP I’ve set.
Ran ‘docker logs open-webui’ from https://techjunction.co/unable-to-access-ollama-models-on-open-webui-after-enabling-https-troubleshooting-and-resolution-guide/ for diagnostics?
Logged into webUI from the SERVER pc. Can I add models?

Ollama is not auto connected. ‘sudo ollama serve’. Generated a key?

Removed the existing container to add this command for a new container.
‘docker run -d \\ -p 3000:8080 \\ -e OLLAMA_BASE_URL=??? \\ -v open-webui:/app/backend/data \\ --name open-webui \\ ghcr.io/open-webui/open-webui:main’
check out this link How to Set Up Open WebUI with Ollama (Complete Guide) - ML Journey

Re-ran this command after deleting the previous container
‘sudo docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main’

Installing pingh and iputils-ping, nevermind useless for this case. Installed to home directory.

Deleted post below because it was the same as this one.

[Tekken_boy]

31May26

Attempted to install a NVIDIA V100 16GB SXM2 TESLA GPU for the AI workload over the 3070Ti 8Gb, but there were booting issues when combining the V100 with a 1060 3Gb.

PC would not boot with the V100 installed…… Maybe will have to take it apart and reassemble it and see if that helps.

May also update OpenWebUI. I needed to get the WebUI key and used “sudo docker exec open-webui cat .webui_secret_key”. Worked well!