Example script for reading out the smart energy meter with Python and a Raspberry Pi. This simple guide will help you read your home energie smart meter.
Nowadays many houses in the Netherlands have a so-called Smart Energy meter. With this meter, energy companies can read your meter without anyone having to come by. But I want to be able to read that data myself with Python.
Data, it’s gold for some companies. But I can’t stand the fact that, for example, my energy company has data from me and I don’t.
So I started tinkering with a Raspberry and Python code to read the data from a smart meter myself.
Building your own python smart meter setup
This DIY project is actually very simple and you only need a few things to read your meter yourself.
Mind you, you need to be a bit tech-savvy to build this. You don’t need a soldering iron just some linux installation and Python code experience.
The necessities for building this yourself are:
1 Raspberry Pi from 40 bucks to 100 with case and cables (affiliate link)
1 P1 cable 16 bucks (affiliate link)
and some Python code.
I assume that you have a Raspberry and have bought a P1 cable. I also assume that you have installed your raspberry with the Rasp OS.
Raspberry Pi OS (previously called Raspbian) is the Foundation’s official supported operating system. You can install it with NOOBS or download the image below and follow our installation guide.
You can always use any linux OS you want or that you have. You do however need Python 3.8 minimum. Your system comes with 2.7.x and 3.5.x so you better upgrade. Read this if you want to upgrade to Python 3.8.x
Setting up the P1 cable on Raspberry PI
P1, what is a P1 cable? Well it goes into the P1 port 🙂 The P1 port is a serial port on your digital electricity meter, into which you can plug an RJ-11 (Registered Jack) plug (known from the telephone connections) to read the meter readings and consumption. It is not possible to send data to the port!
However, this serial port is not standard, despite the fact that it works at UART TTL level, the logical values are reversed: 1 = 0 and 0 = 1, this can be solved by software, but does not always give the desired result, solution is to do this hardware-wise, for example with a chip to be placed in between. A MAX232 or 7404 is often used.
According to the specification, the port can handle max. 15v, so a real RS232 signal (+ 12v) from a “real” com port must be able to handle it.
The plug that has to be put in the P1 port is the type RJ-11, 4 or 6-pin (known from the telephone connections). The P1 cable has this RJ-11 type of plug on one side and USB on the other side.
So.. simple right, you put one end (RJ-11) of the cable in the smart meter.
The black cable looks like an land line phone plug
And the other end you put in one of the Raspberry USB ports.
Sorry, this is just a photo, usb is plugged in on the side
So as it is an USB port it works as a serial port.
\# Seriele poort confguratie
ser = serial.Serial()
# DSMR 2.2 > 9600 7E1:
ser.baudrate = 9600
ser.bytesize = serial.SEVENBITS
ser.parity = serial.PARITY\_EVEN
ser.stopbits = serial.STOPBITS\_ONE
You also code your python script as a serial port reader. As you can see in above code you call upon the serial packaged (you will have to install) and it reads-out the data with a 9600 baudrate.
It could be that you don’t have a 9600 baudrate smart meter but a 115200 baudrate, I guess that is a newer version (or maybe used on Gas Meters, I’m not sure.)
But don’t worry it’s all in my code.
So you can run my code headless by running it from a cronjob or by hand. It files some log files so you can take a look with tail if it still works.
Installing the right python packages on Raspberry Pi
For this script to work you will need some packages. If you put below text in a requirements.txt file and run this with pip install -r requirements.txt it will install all you need.
click==7.1.2
Flask==1.1.2
future==0.18.2
iso8601==0.1.12
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
plotext==1.0.10
pyserial==3.4
PyYAML==5.3.1
Werkzeug==1.0.1
requirements.txt
The most important package is the pyserial package that actually reads the data coming from your smart energy meter.
This package encapsulates the access for the serial port. It provides backends for Python running on Windows, OSX, Linux, BSD (possibly any POSIX compliant system) and IronPython. The module named “serial” automatically selects the appropriate backend.
So what more do you need?
Place below script in a folder. Make sure the folder contains a folder named:
- logging
- database
And try to run the script!
# Python script om P1 weer te geven
# http://domoticx.com/p1-poort-slimme-meter-telegram-uitlezen-met-python/
import re
import os
import serial # pip install pyserial
import time
import sqlite3
import logging
from datetime import datetime, timedelta
db = '/home/pi/PySmartMeter/database/energie.db'
timestr = time.strftime("%Y%m%d")
folder = "/home/pi/PySmartMeter/logging"
os.makedirs(folder, exist\_ok=True)
f = "/home/pi/PySmartMeter/logging/" + str(timestr) + ".log"
log\_format = "%(asctime)s :: %(levelname)s :: %(name)s :: %(filename)s :: %(message)s"
# logging.basicConfig(level='DEBUG', format=log\_format, datefmt='%d-%b-%y %H:%M:%S')
logging.basicConfig(
filename=f, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG)
# Seriele poort confguratie
ser = serial.Serial()
# DSMR 2.2 > 9600 7E1:
ser.baudrate = 9600
ser.bytesize = serial.SEVENBITS
ser.parity = serial.PARITY\_EVEN
ser.stopbits = serial.STOPBITS\_ONE
# DSMR 4.0/4.2 > 115200 8N1:
#ser.baudrate = 115200
#ser.bytesize = serial.EIGHTBITS
#ser.parity = serial.PARITY\_NONE
#ser.stopbits = serial.STOPBITS\_ONE
ser.xonxoff = 0
ser.rtscts = 0
ser.timeout = 12
ser.port = "/dev/ttyUSB0"
ser.close()
i = 0
watt = \[\]
consumption\_1 = None
consumption\_2 = None
return\_1 = None
return\_2 = None
def expires(minutes: int = 5):
future = datetime.now() + timedelta(seconds=minutes\*60)
return int(future.strftime("%s"))
def average(lst):
return round(int(sum(lst) / len(lst)))
hour\_timestamp = expires(60)
five\_minutes\_timestamp = expires(5)
logging.warning("Start collecting")
while True:
ser.open()
checksum\_found = False
while not checksum\_found:
timestamp = int(time.time())
try:
ser\_data = ser.readline() # Read in serial line.
# Strip spaces and blank lines
ser\_data = ser\_data.decode('ascii').strip()
except Exception as e:
exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
logging.warning(str(e) + " | " + str(exc\_type) +
" | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
pass
try:
if re.match(r'(?=1-0:1.7.0)', ser\_data): # 1-0:1.7.0 = Actual usage in kW
kw = ser\_data\[10:-4\] # Knip het kW gedeelte eruit (0000.54)
# vermengvuldig met 1000 voor conversie naar Watt (540.0) en rond het af
watt.append(int(float(kw) \* 1000))
if re.match(r'(?=1-0:1.8.1)', ser\_data):
consumption\_1 = ser\_data\[10:-5\]
if re.match(r'(?=1-0:1.8.2)', ser\_data):
consumption\_2 = ser\_data\[10:-5\]
if re.match(r'(?=1-0:2.8.1)', ser\_data):
return\_1 = ser\_data\[10:-5\]
if re.match(r'(?=1-0:2.8.2)', ser\_data):
return\_2 = ser\_data\[10:-5\]
try:
conn
except Exception:
conn = sqlite3.connect(db)
try:
if watt:
if timestamp >= five\_minutes\_timestamp:
avg\_watt = average(watt)
conn.execute("""INSERT INTO consumption (consumption, datetime)
VALUES (?,?)""", \[avg\_watt, timestamp\])
conn.commit()
logging.warning("Watt saved to Dbase")
five\_minutes\_timestamp = expires(5)
watt = \[\]
except Exception as e:
exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
logging.warning(str(e) + " | " + str(exc\_type) +
" | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
pass
try:
if None not in (consumption\_1, consumption\_2, return\_1, return\_2):
if timestamp >= hour\_timestamp:
conn.execute("""INSERT INTO totals (consumption\_1, consumption\_2, return\_1, return\_2, datetime)
VALUES (?, ?, ? ,? ,?)""", \[consumption\_1, consumption\_2, return\_1, return\_2, timestamp\])
conn.commit()
logging.warning("Totals saved to Dbase")
hour\_timestamp = expires(60)
consumption\_1 = None
consumption\_2 = None
return\_1 = None
return\_2 = None
except Exception as e:
exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
logging.warning(str(e) + " | " + str(exc\_type) +
" | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
pass
except Exception as e:
exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
logging.warning(str(e) + " | " + str(exc\_type) +
" | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
pass
# Check when the exclamation mark is received (end of data)
if re.match(r'(?=!)', ser\_data, 0):
checksum\_found = True
ser.close()
conn.close()
Questions? Let me know! Coffee? Yes Please!
Originally the initial code came from domoticx.com but I changed it to my needs