I was curious to know how the nerds here are doing home automation.
Most web guides seem to be ancient or rudimentary in terms of UX, so I was wondering I would ask the nerds here how they’re going about it.
Since I got a GPU for my homeserver, I can finally do at home what I did with LLMs at my lab. I want to replace the functionality of the echo dot for my family, which I was planning through a combination of stuff.
The flow I was thinking of was:
Whisper (audio input) → LLM (probably a non reasoning model, or something super fast like Olmo 7b) → tool call to Home Assistant → action output → TTS model to respond.
While I’m also aware of how things work in the first half, I don’t have much experience with TTS and would love to know more about it.
I’m currently looking into using a mixture of telegram, home assistant, n8n, ollama and gitlab
I’m trying to get a system together where we can message automation ideas to the system, for it to create an issue in gitlab
When a new issue is created for it to run another n8n workflow to document the automation as a comment, create a pull request for approval before implementing it
So far I’ve got it creating issues but I’ve had to resort to Gemini, to get the appropriate home assistant entities and rough idea
Then Gemma for the automation creation
Using n8n with it’s API function and creating tools from subworkflows
My next step is looking at something with the Ryzen AI Max 128gb CPU but that’s a big jump from my teeny Tesla P4 to keep it all internal
I am trying to build out something similar. I build out a basic bot for my parents to use so anytime something comes into a parking lot I scan it with frigate and then pass it through Ollama and send a summary to my family WhatsApp group.