Local AI Showcases - Chat Edition

I think it would be great to have sort of a showcase of what is possible with local AI, on a chat level, - right now. It would also be great, IMO, to have it in a sort of structured manner for other people to see and make up their own mind if this is something they want to use. For this, I start this thread with the following template in mind:


Machine: Quick description of your system

Problem:
Prompt:
Model:
Harness:
Result:


Repeat

If you like this, share your stories! And it would be great to keep the chatter about technical details or pro/cons … etc. to a minimum, there are plenty other places to exchange thoughts on that.

If you have more experience with more elaborate ways of using these things, agentic workflows … etc. and you see a way to present it in a structured list: Open a thread, I want to know more!

Machine: Strix Halo from Framework - https://frame.work/at/en/desktop 128 GBs of unified RAM

Problem: I run a couple of docker containers behind an NGINX reverse proxy. It works, but I was not satisfied with the organization of things. I knew what needed to be done, but the problem wasn’t painful enough to actually do it on my own.
Prompt: Here you find an attempt to run an NGINX reverse proxy for various docker containers. I am not satisfied with this; the whole copying of the config file and creating a custom NGINX image seems overkill. How can I improve this, how can I split the nginx configuration into multiple parts to have one file per docker service.

  • I added the config, the Dockerfile as well as an Ansible playbook

Model: Qwen3.5-122B-A10B-UD-Q4_K_XL
Harness: Turnstone + Ansible Skill
Result: It basically did what I wanted to do in the first run. I found some quirks that I wanted fixed while testing, it did that too. Nice!


Problem: I have an off-the-shelf router at home for my network setup. It has a couple of VLANs and nothing fancy. Reading stuff about networking and watching Youtube videos gave me knowledge that I want to apply to my home setup, but I am not comfortable enough with the configuration framework of my router to get things done. And not enough time …
Prompt: Please check the attached config file for a Ubiquiti router. Give me a summary of the configuration and identify improvements

  • I added the config backup

Model: Qwen3.5-122B-A10B-UD-Q4_K_XL
Harness: Turnstone
Result: It gave me a nice overview of what was missing and what can be (or must be) improved. I then also gave it a text table of the desired VLAN setup with a firewall matrix and I got a whole script to apply the improvments, fixes and firewall rules that were missing


Problem: I use Navidrome in my homelab and checking for updates I found the AudioMuse-AI plugin that allows a song recommendation mechanism to be used in Navidrome. I wanted to test it, but I was too lazy to set the whole thing up. I use different custom docker networks … etc. and would need to adapt the projects docker files. All do-able, but not straight forward at 10 o’clock in the evening after work and family …
Prompt: Please checkout GitHub - NeptuneHub/AudioMuse-AI: AudioMuse-AI uses sonic analysis to rediscover forgotten songs, uncover hidden connections in your music library, and generate intelligent playlists for Navidrome, Jellyfin, LMS, Lyrion, Emby and Plex: no metadata or external services required. · GitHub, maybe you already have the code under /tmp . I need an ansible playbook that hosts this program. However, one need to adapt the docker compose to run it on a no-internet network. I attach Ansible playbooks to see how I run things until now

  • I attached the navidrome playbook and the docker network playbook

Model: Qwen3.5-122B-A10B-UD-Q4_K_XL
Harness: Turnstone + Ansible Skill
Result: It showed me two ways to solve the problem, I picked one and could host it immediately


Problem: I use forgejo and wanted to use LDAP for user mgmt. Both containers already run on my network but I had trouble connecting the two. At this point turnstone was not able to directly try things out, thus I expected to do some log copying.
Prompt: I have the following playbook to setup forgejo with lldap (already present) However, I am stuck and it does not work. Can you please take a look at it? Example config is here lldap/example_configs/gitea.md at main · lldap/lldap · GitHub

  • I attached the forgejo ansible playbook

Model: Qwen3.5-122B-A10B-UD-Q4_K_XL
Harness: Turnstone + Ansible Skill
Result: We had some back/forth about different solutions and log-results. In the end I was able to verify the theory raised by the model and to validate the solution. This helped me a lot, because I am not too deep into the whole LDAP configuration and now I have something that just works

1 Like

Machine: lol the basement lab is a constellation of machines

Problem: I needed to test out a wiki.
Prompt:

We need a wiki for the lab, i want you to stand up the Outline wiki, nginx TLS-fronted, backed by our Postgres, with Microsoft/Entra sign-in as the only login. Done = I can browse to it and complete a "Continue with Microsoft" login end to end.

we want to use the URL https://wiki.top-s3cret-lab-domain.com to reach the wiki

Use the right tool for each job:
- Proxmox — build the ubuntu VM on the same lab subnet and our usual initial sizing
- RCON — install it on the new VM first so it's remotely manageable; wait for it to load and confirm XConnect can reach it before proceeding. curl -fsSk https://xconnect.top-s3cret-lab-domain.com/bootstrap/install.sh | RCON_JOIN_TOKEN=obviously_not_real_7R2zviGYo sh
- Cloudflare-DNS MCP — create an unproxied DNS-only A record wiki.top-s3cret-lab-domain.com → the VM's IP.
- certbot MCP — find the cert that covers wiki.top-s3cret-lab-domain.com. If one exists deliver it to the box via the pull link and wire nginx; since this box is brand new, that's the initial (bootstrap) delivery — also set up the monthly keyhash refresh for future renewals. If no cert covers the name, STOP and tell me.
- XConnect — on database.top-s3cret-lab-domain.top, create a dedicated outline Postgres role + database.
- Azure CLI MCP — create the Entra app registration in our tenant and wire up SSO, make sure user assignment is required, and assign users Three and Meta as users of the app. Add Three as the apps OWNER.

Outline's requirements — don't skip these:
- Postgres (the DB above) and Redis (required — run Redis alongside Outline on the new VM).
- Run Outline as a container; FILE_STORAGE=local on a persistent volume from docker.getoutline.com/outlinewiki/outline:latest
- Generate SECRET_KEY and UTILS_SECRET (32-byte hex) on the box; set URL=https://wiki.top-s3cret-lab-domain.com, DATABASE_URL, REDIS_URL.
- nginx reverse-proxies 443 → Outline (127.0.0.1:3000) and must forward websocket upgrade headers.
- Entra: web redirect URI = https://wiki.top-s3cret-lab-domain.com/auth/azure.callback, a client secret, delegated openid profile email offline_access + User.Read. Wire AZURE_CLIENT_ID/SECRET/TENANT_ID into Outline. Confirm the exact auth env-var names and callback path against Outline's docs for the version you deploy (they've changed across releases).

Guardrails:
- Verify each layer before moving on (DNS resolves → TLS serves the wiki cert → app is up → login works). Don't declare success until the Microsoft login actually completes.
- Before anything irreversible (deleting the VM, dropping a DB, replacing an existing DNS record), stop and wait for user confirmation.
- Keep secrets (client secret, SECRET_KEY, cert key) out of chat — write them straight into the box's env file.

Model: Deepseek v4 on VLLM
Harness: Turnstone
Result: A wiki, stood up and running with a vm, database, dns, certificate, single sign on in azure entra id…

No but seriously, i posted a whole series of youtube videos in another thread. Infrastructure-as-Code is dead, long live Infrastructure-as-Claude!

2 Likes

Machine: Local AI setup with API access

Problem: Quickly verify basic information about a nonprofit without manually checking multiple records.

Prompt: “Check this organization and summarize the available nonprofit information.”

Model: Local LLM

Harness: A simple workflow using the Nonprofit Check Plus API as one of the data sources.

Result: The AI summarizes the returned information in a structured format, showing how a local model can work with external data to handle a practical research task.