To the Home Page

Solar inverter datalogger

The alternative datalogger software for one of the commercially available solar inverters, based on reverse engineering of its Modbus communication protocol

Published on July 29, 2023 · Reading time: 3 minutes

Code on GitHub

Live parameters

Get the current device state on a mobile or desktop device. Pin important entries to the top

Powered by MQTT and WebSocket technologies. Compatible with MQTT clients, such as CircuitPython’s MiniMQTT. No accounts required

Live parameters

Chart view

Choose up to two numeric parameters to be displayed on the same screen for any date range

Historical data is aggregated with ETL queries and stored in multiple tables to achieve better performance with microSD cards

Chart view

Basic automation

Extend the solar inverter’s features with battery output and charge management

Output and charging modes are controlled automatically based on weather conditions. The automation can be disabled on demand

Basic automation

Activity history

Find out what has happened in the past

Battery mode automation, changed settings, inverter errors and communication issues are all stored in the log

Activity history

Technical details

The complete deployment consists of the API backend service (Django REST), the frontend service (React exported to static files), two workers for data collection and processing, the Mosquitto MQTT broker, and the PostgreSQL database. Logs are stored between container restarts using the journald logging driver. All packages and Docker images have exact versions specified, and multi-stage builds were used whenever possible.

The communicator worker is responsible for retrieving data from the inverter, post-processing, saving to the database, and sending to the broker. It can change inverter settings as well, either automatically or based on the user’s request. The common behavior is separated from implementation details.

The transformer worker will periodically aggregate staging data, store it in one of four tables (each tier has a different precision and retention period), and remove old entries. These operations are performed using raw SQL queries, with the help of PostgreSQL features such as date binning and statistical mode.

The web interface uses MQTT over WebSocket for live data and simple JSON APIs for everything else. There are five screens: live parameters, charts, energy production and consumption, log entries, inverter settings with automation options. I’ve designed custom Bootstrap components to improve navigation on mobile.

Charts are the most complex component of the web interface. Pan and zoom are supported, and user can see data for two numeric parameters at once. Native date/time pickers and range presets make it easier to find data.

Architecture diagram
Architecture diagram

Operating system choice

This application is installed on Raspberry Pi 3A with 512 MB of RAM. Back in July 2023, Raspberry Pi OS 12 was not available yet, so I’ve decided to install real Debian with a mainline kernel in UEFI mode, following the instructions from the Pete Batard’s blog (Rufus author). For compatibility reasons, this solution uses a FAT16 EFI partition instead of the more common FAT32, which made the installation process quite annoying. I had to open fdisk multiple times to make sure the partition’s signature is correct.

In the following months, there were no stability issues, even after multiple power outages. Basic features like Wi-Fi communication and power management seemed to be fully supported. I’ve run into some issues I couldn’t resolve, though. Without an I2C or SPI support, you can’t really add battery backup because it’s impossible to read its parameters. The gpu_mem setting was ignored, so about 64 MB of RAM went unused.

I’ve switched to the official operating system during monthly maintenance in early 2024. Raspberry Pi Imager was used to prepare an SD card, customize hostname and login credentials, and enable SSH. The database is stored in a separate directory, so it could be compressed and copied to Pi easily. I’ve installed Docker, enabled firewall, started the Compose-based application for the first time, and that was it.

Check out other blog posts: