[Devember 2021] Reinforcement Learning For Tesla Powerwall

Background
Tesla powerwall has a selection of options for how to use battery power but has several clear data driven inefficiencies in their product selection

  • It doesn’t consider weather forecasting for future considerations beyond storms in the next few hours, meaning tomorrow’s solar generation isn’t factored into any of its modes.
  • It doesn’t seem to model future home usage.
  • It has peak/normal/off-peak time selection but not with any $ associatable to each one.

None of the modes available (backup, self consumption, advanced) have documentation for exactly how they work.

The good news is that the Tesla API has extensive options for both retrieving data and setting modes on the powerwall. So externally it should be possible to make an optimal control system for the powerwall.

Goal and Design:
Creating something generic in this space is difficult because the solar configuration, home usage and grid consumption parameters vary widely from home to home and grid to grid. So while unnecessary use of force it’s quite possible to solve this problem with machine learning / reinforcement learning using an individual’s home’s previous data for ease of adoption / software maintenance.

So the architecture would be something like

  1. Store the various data sources we can gleem for building a model into a sqlite database.
  2. Create an OpenAI Gym that models a home’s power input/outputs and outputs costs based on the collected data.
  3. Train the reinforcement model using that Gym.
  4. Use that trained model to control my power wall.

Because this is machine learning, it seems prudent to be written in Python.

Useful Data
Tesla API (github.com/tdorssers/TeslaPy) produces 15 minute data for

  • Home usage
  • Solar input
  • Battery charge / discharge
  • Grid input / output.

Pysolar (pysolar.org)

  • Sun altitude
  • Sun azimuth

Weather forecast (openweathermap.org)
Day of the week / hour of the day (this the best I can think of for generic home usage predictor)
Power company grid usage cost and grid feedback reward function.

Configuration
Tesla account / password (for both making actions and getting historical data)
Lat/Long of your house (for weather and pysolar)
Openweather API login (free quota is more than enough for individuals)
Power company module name. (A python module but configure which one here)

Limitations / Scoping
Support only the Tesla powerwall, and only a single powerwall, it’s all I have and has an API that I’ve tested and worked with.
Targeting only gym / model building that works for Solar + Powerwall + Grid
Targeting only grid models that can be given a date/hour can produce a cost
Limited precision, one decision per hour and 1% battery precision
Only use forecasting 24 hours into the future

I understand that the people who need this the most such as those in Texas would be excluded by multiple of the limitations here (hour and grid model), but I have to start somewhere.

Challenging Last 1%
Determining the reinforcement model / cheapest training time to achieve “good” results.
Visualizing the impact for others that don’t / can’t run the software, mostly important for Devemeber.
Can data collection, training and acting be cost negative by running on Linode? (Cost savings vs Compute costs).

6 Likes

GitHub project is here

The data collection and model training is all done.

The resulting rewards are exciting and looking a a variety of days the action decisions look optimal.

Tried a lot of RL frameworks but stable baselines3 was the most user friendly, particular for developing your own complex gym.

Biggest problems right now on order:
Need to finish up the cleaned up inference / take action code so that I can publish finished workable solution. (Have a alpha/unintelligible version working).
The power costs are not modularized / not configurable and are currently hard coded set to my AUD Perth power costs.
Would love to test it in windows and create an apt package for easier install instructions.
The Powerwall Gym/Env code is spaghetti, but deadlines don’t create good results they create they best result for that date :stuck_out_tongue:

1 Like

Github has the fully working and inuse solution for me.

The biggest problem for generalism is the powerplan is hard coded to how much I pay for electricity and recieve for feedback.

Beyond that features that I’d like to iterate on

  1. Write some tests for easier iterative development.
  2. Make a package/installer.
  3. Create a GUI.
  4. Check/Make it work for windows.
  5. Cleanup in the gym.

Anyway, exciting devember glad I got something working and out.