Hi there! Is there anyone who has dabbled enough with home assistant and llm/ollama integration on Linux who could recommend some good plans that are really good to use with home assistant? I am currently using a very basic llama 3.0 model with a 4060ti gpu and it works about 70% of the time, but gets confused with turning some devices on or off, or gets confused with certain commands or general knowledge questions such as " How many jellyfin devices are currently active", “how much bandwidth is currently being used by those devices” etc.
1 Like
Mistral has been working well for me, as has Llama 3.1 which is what HA themselves recommend. I’m running on a 1060 though so limited on the size of models. The best thing to do is look through the models that support tools and see what works best for you.
Do you perhaps know how to be able to judge if models support tools? Like a specific “toolset” or package?
1 Like
Sure, you can filter the list on ollama
I’d love to see how qwq and llama3.3 run but I don’t have enough vram for those. Llama3.2 I found it misunderstood more often than the others but your mileage will vary with a more powerful card
1 Like
I can say I’ve had a similar Home Assistant experience running Lllama 3.1 7b using Ollama on my 7900XTX. Response speed is good, but it just isn’t quite intelligent enough. Simple state switches work fine, but beyond that it’s a 50/50 chance of being right.
I played around with the 7b and 14b distill of Deepseek-R1 (so Llama and qwen respectively) and it does seem to give more accurate responses, but the “thinking” delay kinda ruins the conversational aspect IMO.
I haven’t tried the latest home assistant update with streaming responses yet though, so maybe that will make a difference. My end goal is to try and build out the hardware to run Llama 3.3 70b locally.
1 Like
I just wanted to try them for myself but for both I always get “Unexpected error during intent recognition” but both models work fine with Open WebUI. Did you need to do anything special to get them running? So far I only added the models via the Ollama integration, just like I did for any other model so far. HA 2025.3.4
Regarding the other models I found that even naming colors can be hard to understand. While red and green work, pink or purple very often doesn’t. Being bilingual I tried German and English and that didn’t make a difference.
Try Qwen2.5. I’m using it and it’s pretty good with Home Assistant. I found it a lot better than any Llama version.
The other good one for HA is qwq if I’m not mistaken. Haven’t tried because it’s too heavy for my host.
1 Like
I have been away for awhile so sorry for the late response! When I get the chance I can test these and see how the llama option works and let you know here! That you for the Info!
1 Like
Update, so far, llama 3.3 70b seems to be a really good contender for home assistant, the only real drawback I have encountered is the slow initial response time if you haven’t used voice in awhile, and the higher you all have previously recommended all work pretty well too!