Hey I wanted to share this project I’ve been working on for a few months in my attempt to learn more about current Transformer AI models. The result is a model that can perform function calling with a custom integration for Home Assistant. The model allows you to control devices with plain English and supports text-to-speech and speech-to-text add-ons for vocal interaction. I wanted a completely local solution with no dependency on OpenAI. Another goal of the project was to run on as low spec hardware as possible and ended up with “usable” performance on a Raspberry Pi 4 GB.
Github (Home Assistant Integration + Dataset): GitHub - acon96/home-llm: A Home Assistant integration that allows you to control your house using an LLM running locally
HuggingFace (Model): acon96/Home-3B-v1-GGUF · Hugging Face
Some technical info about it:
- The model is a fine tuning of Phi-2 released by Microsoft. It is a 3B parameter model that swings well above its weight class
- Model is quantized to lower bit values to be used with Llama.cpp which allows you to run the model on a CPU with lower precision (lower RAM usage)
- Uses a “synthetic” dataset which was constructed using permutations of random combinations of different requests and context states
There’s installation instructions in the Github readme, and it should be compatible with fully supervised Home Assistant installations. The project is still in a very “MVP” state but I have been able to consistently turn lights on and off with various prompts. Also, disclaimer: the performance of this on my Raspberry Pi 4 4GB is not great, but if you run the model on a computer with a GPU then it can be very responsive. I have a few ideas already on how to speed up performance on the RPi.
Would love to hear any feedback on how it works. I’m planning on continuing to work on the synthetic dataset and continue with fine tuning Phi-1.5 which is only 1.5B parameters.