To the Home Page

Solar inverter datalogger

The alternative datalogger software for one of the commercially available solar inverters, based on reverse engineering of its Modbus communication protocol

Published on July 29, 2023 · Reading time: 3 minutes

Code on GitHub

Live parameters

Get the current device state on a mobile or desktop device. Pin important entries to the top

Powered by MQTT and WebSocket technologies. Compatible with MQTT clients, such as CircuitPython’s MiniMQTT. No accounts required

Live parameters

Chart view

Choose up to two numeric parameters to be displayed on the same screen for any date range

Historical data is aggregated with ETL queries and stored in multiple tables to achieve better performance with microSD cards

Chart view

Basic automation

Extend the solar inverter’s features with battery output and charge management

Output and charging modes are controlled automatically based on weather conditions. The automation can be disabled on demand

Basic automation

Activity history

Find out what has happened in the past

Battery mode automation, changed settings, inverter errors and communication issues are all stored in the log

Activity history

Technical details

This Django+React application is hosted on the lowest-end ARM64 Raspberry Pi I could buy at the moment (July 2023), the Model 3A+ with 512MB of RAM. It seems to be barely enough for a small dockerized web application.

The SD card is the biggest bottleneck, so an effort has been made to reduce writes and reads as much as possible. I have set up the ZRAM because the CPU is idle most of the time. The swappiness parameter is lowered as well. Logging was not disabled, in fact, I have configured containers to use journald logging driver and preserve logs between restarts.

Software internals

The entire deployment consists of the API service, the frontend service, two workers for data collection and processing, the MQTT broker and the PostgreSQL database.

All packages and base Docker images have exact versions specified. Target images for backend and frontend containers are generated with GitHub Actions, using multi-stage builds whenever possible.

The frontend service uses carefully chosen import statements to greatly reduce tree-shaking time (from about an hour down to just a minute when building on a target device). Generated output files are served by nginx.

There are two worker processes: communicator and transformer. The communicator is responsible for retrieving data from the inverter, post-processing, saving to the database, and sending to the Mosquitto MQTT broker. It can change inverter settings as well.

The transformer will periodically aggregate raw data, store it in one of four tables (each tier has a different precision and retention period), and remove old entries. These operations are performed using raw SQL queries, with the help of PostgreSQL features such as date binning and statistical mode.

User experience

Collected information can be accessed using a MQTT client (live parameters, in read-only mode) or the mobile-friendly web interface. There are five screens: live parameters (backed by MQTT over WebSocket), charts, energy production and consumption, log entries and automation settings.

I’ve designed custom Bootstrap components to improve navigation. Favorite parameters are saved in the browser’s local storage. Log entries can be filtered by their source: inverter errors, automations, and so on.

Charts are the most complex component of the web interface. Users can retrieve data for two numeric parameters at once. I’ve used native date/time pickers and added range presets for the convenience. API service is responsible for choosing a tiered table that offers the best precision, and for reducing the number of data points if necessary.

Operating system choice

Back in July 2023, Raspberry Pi OS based on Debian 12 was not available yet, so I’ve decided to install real Debian with a mainline kernel in UEFI mode, following the instructions from the Pete Batard’s blog (Rufus author). For compatibility reasons, this solution uses a FAT16 EFI partition instead of the more common FAT32, and it made the installation process quite annoying. I had to open fdisk multiple times to change the partition’s signature.

In the following months, there were no stability issues, even after multiple power outages. Basic features like Wi-Fi communication and power management seemed to be fully supported. I’ve run into some issues I couldn’t resolve, though. Without an I2C or SPI support, you can’t really add battery backup because it’s impossible to read its parameters. The gpu_mem setting was ignored, so about 64MB of RAM went unused.

I’ve switched to the official operating system during monthly maintenance in early 2024. Raspberry Pi Imager was used to prepare an SD card, customize hostname and login credentials, and enable SSH – no peripherals were required. The database is stored in the /var/lib/solar directory, so it could be compressed and copied over SSH easily. I’ve installed Docker, enabled firewall (including SSH and HTTPS ports), started the Compose-based application for the first time, and that was it.

Check out other blog posts: