Return to Level1Techs.com

Bluetooth Presence Detection for Home Automation -- The Level1 Way

Bluetooth Presence Detection

You might be here because you heard me mention my Bluetooth presence detection setup on the new SelfHosted podcast, or possibly because you want to subject yourself to an internet-of-things setup in your own home while desperately trying to escape the consequential corporate surveillance as a result of that action (soft chuckles for everyone). Let’s talk about what it is, why you might want it, and how you can minimize the downsides

What is this?

Quite simply, the future of the smart home is in your hands and no one else’s. I simply cannot imagine a commercially viable smart home that runs from the cloud unless the inhabitants of that smart home are willing to upgrade and replace the smart home components on a rapid basis. While my personal belief is that novelty, more than utility, drives almost all current “smart home innovation,” I will concede that automating aspects of my domestic life is appealing. I don’t believe it is necessary that anyone give up autonomy and freedom (and Software Freedom) in order to enjoy modern conveniences.

One of the foundational blocks we can talk about for home automations is presence detection. How does the smart home know who is at home at any given time?

With this project I show you how to setup a Raspberry Pi zero for presence detection.

Why would you want presence detection?

If you want to do presence detection, this is the best way, if you carry a cel phone. I’ve spent many hours trying, and failing, to find a better way. Wifi is not fast enough and phones will hop off the wifi to safe power when they’re sleeping. GPS is not accurate enough to distinguish between me going from an evening stroll (and needing outside lights for my return) and being inside my home. Wifi suffers from a bit of the same kinds of problems. There are also false negatives when switching e.g. from 4G/LTE on a phone to WiFi. Bluetooth was the only option.

Even then, Bluetooth was not perfect – Bluetooth low energy tokens are expensive (relatively) and most phones are going to beacon on Bluetooth channels just by virtue of being on.

If you want your garage lights to come on after dark when you arrive at home, automatically, and shut off when you enter your home, then this is for you.

If you’ve experimented with IoT devices, then you know that solutions for this are often costly and sometimes depend on an internet connection (!?) in order to work properly.

It goes without saying that requiring an internet connection for basic “smart home” functionality is one of the dumbest things that there is. This won’t require any type of internet connection.

All right I’m sold – let’s do this!

The secret sauce is a Raspberry Pi Zero W and monitor.sh https://github.com/andrewjfreyer/monitor . TL;DR : Passive Bluetooth presence detection of beacons, cell phones, and other Bluetooth devices. Useful for mqtt-based home automation, especially when the script runs on multiple devices, distributed throughout a property.

Step 1 – Configure your Raspberry Pi

Download Raspbian
Image Raspbian
Mount the Boot Partition
Create wpa_supplicant.conf w/this conf:


country=US

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

update_config=1

network={

ssid="Wireless Network Name"

psk="Wireless Network Password"

key_mgmt=WPA-PSK

}

touch ssh

(on Windows create a file called ssh with no extension in the root of the boot partition. This will enable ssh. On Mac/Linux just touch the file.)

Step 2 — Download & Configure Monitor

ssh [email protected]_the_ip_was_on_your_network (password is raspberry) and set a more appropriate memoerable password.

apt update && apt upgrade
then

apt dist-upgrade
then

apt install pi-bluetooth

Then reboot!

Step 3 – Mosquitto and git

We will need to add the Mosquitto repository to Debian, update and fetch it and the git client.


wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key

sudo apt-key add mosquitto-repo.gpg.key

cd /etc/apt/sources.list.d/

wget http://repo.mosquitto.org/debian/mosquitto-stretch.list

apt-get update

apt-get install -f git libmosquitto1 libmosquitto-dev mosquitto mosquitto-clients

Step 4 – Setup monitor

You may encounter errors during the apt update and upgrade – just reboot and make sure everything has been updated. You can check journalctl -xe for errors after reboot to make sure it came up right.

With all the packages installed and working properly, you can now setup monitor.

Getting the monitor script up and going is not too bad.

git clone https://github.com/andrewjfreyer/monitor.git

[email protected]:~ $ cd monitor/
[email protected]:~/monitor $ chmod +x ./monitor.sh
[email protected]:~/monitor $ ./monitor.sh


then it may complain about mqtt or hcidump being missing. No big deal; just install those. (You may have to sudo these commands so that it is able to install these packages).

The next step is really testing and tying this into something else.

MQTT is a decent “IoT” messaging system so that various devices can communicate. A single raspberry pi monitoring bluetooth devices’ comings and goings around your place is neat, but to really do a good job you need a small army of these. Soon we’ll cover using automation to manage logging, security, and configuration centrally.

For now, you can pull your bluetooth MAC address and add it to known_static_addresses

Troubleshooting

Nothing open source ever goes exactly according to the how to, eh? Yeah, the adventure really clears the mind though.

journalctl -u mosquitto

This is useful for troubleshooting.

In my case, this was fine. However man mosquitto showed it had a -c command for checking the config. Let’s do that.

Segmentation fault 

Oh, look. Something is wrong with the config. After spending some time with gdb I ultimately gave up.

Downgrade the packages:

sudo apt-get install mosquitto=1.6.4-0mosquitto1~stretch1 libmosquitto1=1.6.4-0mosquitto1~stretch1 mosquitto-clients=1.6.4-0mosquitto1~stretch1

… and now it seems to be working.

The last step is to edit mqtt_preferences and specify your mqtt host. Never heard of mqtt? Learn more here:
http://mqtt.org/

What Now?

Well, you’ve got some work to do to add the devices around you.

# ls -al
total 156
drwxr-xr-x 4 pi   pi    4096 Oct 13 05:15 .
drwxr-xr-x 4 pi   pi    4096 Oct 13 03:44 ..
-rw-r--r-- 1 pi   pi      46 Oct 13 03:51 address_blacklist
-rw-r--r-- 1 root root   638 Oct 13 04:36 behavior_preferences
drwxr-xr-x 8 pi   pi    4096 Oct 13 05:13 .git
-rw-r--r-- 1 pi   pi     184 Oct 13 05:13 known_beacon_addresses
-rw-r--r-- 1 pi   pi     158 Oct 13 05:03 known_static_addresses
-rw-r--r-- 1 root root    87 Oct 13 05:14 .manufacturer_cache
-rwxr-xr-x 1 pi   pi   71143 Oct 13 03:45 monitor.sh
-rw-r--r-- 1 pi   pi     498 Oct 13 04:36 mqtt_preferences
-rw-r--r-- 1 root root   204 Oct 13 05:13 .pids
-rw-r--r-- 1 pi   pi       8 Oct 13 03:46 .previous_version
-rw-r--r-- 1 pi   pi     168 Oct 13 05:13 .public_name_cache
-rw-r--r-- 1 pi   pi   29595 Oct 13 03:45 README.md
drwxr-xr-x 2 pi   pi    4096 Oct 13 03:45 support

As you can see a fair number of files have been generated.

From here you could set it up with Home Assistant or another tool that might be useful to automate things. Check out the video for some ideas that I’ve found actually useful for automation over the years.

The Ten Commandments of Automation

TODO

Okay, so there aren’t exactly ten, and they aren’t exactly commandments. But if you’re going to embark on IoT, always give yourself a little introspection:

  • Is this actually going to be easier?
  • What problem(s) am I trying to solve with this device or setup?
  • Will this save me time, or make my life better in some way?
  • Is it easy to recover when it fails?
  • Is it possible to reproduce this setup quickly?
  • Will this cause visitors to struggle when visiting?
  • Is the contemplated future with this working right going to be more relaxing than the current status quo?
  • If something goes wrong, will I have a heart attack from stress?
  • TODO

No no no this is all wrong!

You might be thinking that this guy has lead me down a path and now I’m operating with two-three-four linux-based ssh-enabled surveillance devices that just cannot wait to mine bitcoin for the Kremlin. And there is some truth to that – fortunately you are (will be?) an IT Pro. The danger, and insecurity, comes from the lack of updates. Just like it would be hilarious, and terribly insecure, if your fridge ran Windows 2000, you must not forget that your IoT devices will need support and updates from now until the end of time.

That’s really the lie of all IoT devices – that they will work forever. In reality they have shockingly short product lifetimes and there have been several funny/sad examples in recent news TODO.

The solution for this is to use standard “configuration management” software, such as Puppet, to manage devices like the Raspberry Pis. When you have configuration changes, the RPi are formatted from scratch and reconfigured using modern packages. With the Debian Linux distribution this approach ensures a 5 to 10 year service lifetime while assuring the same level of security as the main line Debian project.

The specifics of this setup will be a project for another day, however.

9 Likes

reserved

1 Like

Reeee

2 Likes

I hope you won’t hate me for this, @wendell but using today’s tech in IoT and Machine learning and stuff, how intelligent can a home automation setup get?

I remember you talking about this in the news about your own exploits, having bluetooth “sensors” in lampposts to automatically light up and open your garage door when your blutooth mac address is picked up.
I guess I’m asking is, how far can we go with current/near future tech? How close to Jarvis (super-computing excluded) can we get with a couple of powerful servers?

How much information does the monitor give about the monitored Bluetooth devices? For instance if it gives signal strength, couldn’t that be used to estimate the approximate location of a device? If you had a simple 3-4 room apartment, it might technically be possible with a PI in each room. Could open up a lot of fine grained positioning within your own home. For instance, find your keys, locate your cat in hiding, or trigger your morning routine when you enter the kitchen.

It is possible to diy this yes but sometimes orientation of the device also affects signal strength so you have to look at signal level across the fleet

Fortunately for now, your system will only be as intelligent as you make it, despite how ‘smart’ the bulbs/switches are.
But there are many open source solutions that make it much easier to add logic and conditionals to the automation.

I love listening to Wendell go down the home automation rabbit hole and can’t wait to hear more.

I currently have Home Assistant running on an old laptop. It serves as a central point for my devices and programs.
I use ESPHome & Tasmota to create custom firmware for my switches, sensors, relays, and lights. I used to crack open my ‘smart’ devices, find the right pins and flash them manually. I’ve recently discovered Tuya Convert. It takes advantage of a flaw in most ESP devices to inject custom firmware with minimal effort on my end.
Once the device has been removed from the cloud and connected to Home Assistant, simple automation can be created inside Home Assistant, I’ve recently started using Node-RED for a much easier and detailed flow control.
Home Assistant has many integrations with useful apps, With some work you can connect it to Google for their virtual assistant (although I don’t know why you would do that). There are open source apps like Almond and Ada, that provide a virtual assistant to help control devices.
If you install Home Assistant on a raspberry pi that has bluetooth, enabling presence detection is as easy as editing a config line.

I can’t wait to hear Wendell’s solution for cameras. I was using contacam with some cheap laptop webcams, but since I’ve switched to linux I haven’t got them set back up yet.

Okay. I have been casually looking for a way to pass “localhost” email from one network to another - the latter has a defined IP address via FreeDNS, but all SMTP ports are blocked. About a month ago I started an idea converting inbox messages - via monitoring /var/mail/someuser using incron - to send a version of that via scp for the receiving end to handle it as needed (possibly returning it to an email).

By doing a ten-minute web search, I cannot tell if MQTT can interact with local mail delivery (main use case is passing potentially important system messages of my laptops to home base when the laptops are outside the LAN), but if there is an example of this, I would be very pleased.

For reference, I included a very draft and WIP version of what I was trying. At worst I can replace this (the scp component) with MQTT (which I could with a smaller set of what I tried - since I would no longer need to define routing rules at this point), however I presume there is still a more clever way to do this already.

gail.txt (2.6 KB)

it would really be handy to have some pics and stuff for what you’ve done maybe on the blog here. I haven’t done enough of that myself and would love to live vicariously.

the chinese ip cams are hard to beat cost/value wise, esp. with custom firmware, but a rasp pi camera is shocklingly good too for what it costs

I still feel like I’m just getting started with this, some of the things I’ve seen on the Home Assistant forum and reddit are insane.
Due to my budget, I’m limited to getting IOT stuff and other electronics from an Amazon returns store nearby.
I’m also very limited because I’m in an apartment.

I had Home Assistant running on a raspberry pi 3 for about a year just to easily run Pi-Hole. I’m trying to get used to Linux now so Home Assistant is running on Ubuntu on an old laptop.

I can definitely point people to different places if they have questions, But I don’t feel like I have enough knowledge to write specifics about it all.
Like usual, I have a very wide and shallow range of knowledge.

I’ve seen this script, but never bother to set it up, mostly because from what i understand it’s not a room level tracking, it’s a global house, with shorter range than wifi.

Have anyone find/done a good doc about how to step-up this with a master node who calculate to witch node you’r closer or something ?

Given that a pi zero can also be a motion camera, deploying with both motion camera and Bluetooth beacon functionality could be an interesting combo for presence detection.

EDIT: Looks like most people use motionEyeOS as a dedicated motion camera, however it is straightforward to install motion, the underlying software for motionEyeOS. This allows for running additional services. For those interested in a recipe,

sudo apt-get install gdebi-core
wget (deb release link here)
sudo gdebi pi_buster_motion_4.2.2-1_armhf.deb
sudo cp /etc/motion/motion.conf /etc/motion/motion.conf.bak
sudo nano /etc/motion/motion.conf

When editing motion.conf, you’ll want to specify

daemon on
mmalcam_name vc.ril.camera
width 960
height 720
framerate 5

which usually uses less than 80% CPU usage when there is motion, or perhaps lower depending on the demand of other processes. and

webcontrol_localhost off
stream_localhost off

in order to allow remote access to the webcontrol and camera stream.

Sounds like something similar to Happy Bubbles.

It’s just a bunch of small bluetooth beacons that report back to your MQTT server if you’re connected and what the strength is. You can then add logic to pick the highest strength and assign your position to that room. They even provide an example of what a simple server would look like on github.

The product they’re offering feels a bit expensive for my tastes, but it’s just a NodeMCU, a wifi chip, and a few other things.
I was able to find a BOM if you want to make one yourself to test it out. I’d probably add a few other sensors to it just to make use of the NodeMCU.

Edit: I just found out Happy Bubbles are no longer in production, but you can find everything on their Github if you want to start from their code.

2 Likes

Seems to be doable with a regular RPi. Its exactly the idea i was thinking about in my previous post. Will definitively take a look at their git and check it out.

Not sure how accurate this can be tho, like the beacons doesn’t seem to take in consideration walls or other sources that could dampen the signal. Might be possible to create a data model that you can give it a rough layout of your house, and it would use an algorithm to more accurately determine your position with walls in consideration.

Happy Bubbles is pretty cool, too bad they’re out of production. Room assistant ( I can’t include links, simply google ‘room assistant pi zero w’) is another option and they have a specific guide for pi zero w.

If you’re planning to use multiple, an RPi might be a bit overkill, Anything with an ESP32 should work.

If you want more accurate tracking through walls, You’d probably need to look into RF Tomography, something like Xandem. This method is a bit more expensive and the software tends to be very locked down.

It seems like the middle ground is to use a combination of bluetooth and motion detectors.

I’ve been skimming over an idea to use some kind of laser tripwire in my doorways to track what room I’m in. The only problem would be when I have guests, and my cat probably.

Defense:
I always imagined the sensors running on a CAN bus system for tracking and detection. What exact suite of sensors to use IDK.

I imagined a system that could “size you up” once presence is detected by mass. If the mass stays cold a flag would fly asking for definition. Maybe a couple of RFID readers waist high in the shrubs hoping for a gym or library card.

Anyway, that’s something different.