Read P1 port smart meter with Raspberry and Python

Read P1 port smart meter with Raspberry and Python

Example script for reading out the smart energy meter with Python and a Raspberry Pi. This simple guide will help you read your home energie smart meter.

Nowadays many houses in the Netherlands have a so-called Smart Energy meter. With this meter, energy companies can read your meter without anyone having to come by. But I want to be able to read that data myself with Python.

Data, it’s gold for some companies. But I can’t stand the fact that, for example, my energy company has data from me and I don’t.

So I started tinkering with a Raspberry and Python code to read the data from a smart meter myself.

Building your own python smart meter setup

This DIY project is actually very simple and you only need a few things to read your meter yourself.

Mind you, you need to be a bit tech-savvy to build this. You don’t need a soldering iron just some linux installation and Python code experience.

The necessities for building this yourself are:

1 Raspberry Pi from 40 bucks to 100 with case and cables (affiliate link)
1 P1 cable 16 bucks (affiliate link)

and some Python code.

I assume that you have a Raspberry and have bought a P1 cable. I also assume that you have installed your raspberry with the Rasp OS.

Raspberry Pi OS (previously called Raspbian) is the Foundation’s official supported operating system. You can install it with NOOBS or download the image below and follow our installation guide.

You can always use any linux OS you want or that you have. You do however need Python 3.8 minimum. Your system comes with 2.7.x and 3.5.x so you better upgrade. Read this if you want to upgrade to Python 3.8.x

Setting up the P1 cable on Raspberry PI

P1, what is a P1 cable? Well it goes into the P1 port 🙂 The P1 port is a serial port on your digital electricity meter, into which you can plug an RJ-11 (Registered Jack) plug (known from the telephone connections) to read the meter readings and consumption. It is not possible to send data to the port!

However, this serial port is not standard, despite the fact that it works at UART TTL level, the logical values ​​are reversed: 1 = 0 and 0 = 1, this can be solved by software, but does not always give the desired result, solution is to do this hardware-wise, for example with a chip to be placed in between. A MAX232 or 7404 is often used.

According to the specification, the port can handle max. 15v, so a real RS232 signal (+ 12v) from a “real” com port must be able to handle it.

The plug that has to be put in the P1 port is the type RJ-11, 4 or 6-pin (known from the telephone connections). The P1 cable has this RJ-11 type of plug on one side and USB on the other side.

So.. simple right, you put one end (RJ-11) of the cable in the smart meter.

Raspberry PI read-out Smart MeterThe black cable looks like an land line phone plug

And the other end you put in one of the Raspberry USB ports.

Read P1 port smart meter with Raspberry and PythonSorry, this is just a photo, usb is plugged in on the side

So as it is an USB port it works as a serial port.

\# Seriele poort confguratie
ser = serial.Serial()

# DSMR 2.2 > 9600 7E1:
ser.baudrate = 9600
ser.bytesize = serial.SEVENBITS
ser.parity = serial.PARITY\_EVEN
ser.stopbits = serial.STOPBITS\_ONE

You also code your python script as a serial port reader. As you can see in above code you call upon the serial packaged (you will have to install) and it reads-out the data with a 9600 baudrate.

It could be that you don’t have a 9600 baudrate smart meter but a 115200 baudrate, I guess that is a newer version (or maybe used on Gas Meters, I’m not sure.)

But don’t worry it’s all in my code.

So you can run my code headless by running it from a cronjob or by hand. It files some log files so you can take a look with tail if it still works.

Installing the right python packages on Raspberry Pi

For this script to work you will need some packages. If you put below text in a requirements.txt file and run this with pip install -r requirements.txt it will install all you need.

click==7.1.2
Flask==1.1.2
future==0.18.2
iso8601==0.1.12
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
plotext==1.0.10
pyserial==3.4
PyYAML==5.3.1
Werkzeug==1.0.1
requirements.txt

The most important package is the pyserial package that actually reads the data coming from your smart energy meter.

This package encapsulates the access for the serial port. It provides backends for Python running on Windows, OSX, Linux, BSD (possibly any POSIX compliant system) and IronPython. The module named “serial” automatically selects the appropriate backend.

So what more do you need?

Place below script in a folder. Make sure the folder contains a folder named:

  • logging
  • database

And try to run the script!

# Python script om P1  weer te geven
# http://domoticx.com/p1-poort-slimme-meter-telegram-uitlezen-met-python/

import re
import os
import serial   # pip install pyserial
import time
import sqlite3
import logging

from datetime import datetime, timedelta

db = '/home/pi/PySmartMeter/database/energie.db'

timestr = time.strftime("%Y%m%d")
folder = "/home/pi/PySmartMeter/logging"
os.makedirs(folder, exist\_ok=True)

f = "/home/pi/PySmartMeter/logging/" + str(timestr) + ".log"

log\_format = "%(asctime)s :: %(levelname)s :: %(name)s :: %(filename)s :: %(message)s"
# logging.basicConfig(level='DEBUG', format=log\_format, datefmt='%d-%b-%y %H:%M:%S')
logging.basicConfig(
    filename=f, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG)

# Seriele poort confguratie
ser = serial.Serial()

# DSMR 2.2 > 9600 7E1:
ser.baudrate = 9600
ser.bytesize = serial.SEVENBITS
ser.parity = serial.PARITY\_EVEN
ser.stopbits = serial.STOPBITS\_ONE

# DSMR 4.0/4.2 > 115200 8N1:
#ser.baudrate = 115200
#ser.bytesize = serial.EIGHTBITS
#ser.parity = serial.PARITY\_NONE
#ser.stopbits = serial.STOPBITS\_ONE

ser.xonxoff = 0
ser.rtscts = 0
ser.timeout = 12
ser.port = "/dev/ttyUSB0"
ser.close()
i = 0
watt = \[\]
consumption\_1 = None
consumption\_2 = None
return\_1 = None
return\_2 = None


def expires(minutes: int = 5):
    future = datetime.now() + timedelta(seconds=minutes\*60)
    return int(future.strftime("%s"))


def average(lst):
    return round(int(sum(lst) / len(lst)))


hour\_timestamp = expires(60)
five\_minutes\_timestamp = expires(5)

logging.warning("Start collecting")

while True:
    ser.open()
    checksum\_found = False

    while not checksum\_found:
        timestamp = int(time.time())

        try:

            ser\_data = ser.readline()  # Read in serial line.
            # Strip spaces and blank lines
            ser\_data = ser\_data.decode('ascii').strip()
        except Exception as e:
            exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
            fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
            logging.warning(str(e) + " | " + str(exc\_type) +
                            " | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
            pass

        try:
            if re.match(r'(?=1-0:1.7.0)', ser\_data):  # 1-0:1.7.0 = Actual usage in kW
                kw = ser\_data\[10:-4\]  # Knip het kW gedeelte eruit (0000.54)
                # vermengvuldig met 1000 voor conversie naar Watt (540.0) en rond het af
                watt.append(int(float(kw) \* 1000))

            if re.match(r'(?=1-0:1.8.1)', ser\_data):
                consumption\_1 = ser\_data\[10:-5\]

            if re.match(r'(?=1-0:1.8.2)', ser\_data):
                consumption\_2 = ser\_data\[10:-5\]

            if re.match(r'(?=1-0:2.8.1)', ser\_data):
                return\_1 = ser\_data\[10:-5\]

            if re.match(r'(?=1-0:2.8.2)', ser\_data):
                return\_2 = ser\_data\[10:-5\]

            try:
                conn
            except Exception:
                conn = sqlite3.connect(db)

            try:
                if watt:
                    if timestamp >= five\_minutes\_timestamp:
                        avg\_watt = average(watt)
                        conn.execute("""INSERT INTO consumption (consumption, datetime)
                                    VALUES (?,?)""", \[avg\_watt, timestamp\])
                        conn.commit()
                        logging.warning("Watt saved to Dbase")
                        five\_minutes\_timestamp = expires(5)
                        watt = \[\]

            except Exception as e:
                exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
                fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
                logging.warning(str(e) + " | " + str(exc\_type) +
                                " | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
                pass

            try:
                if None not in (consumption\_1, consumption\_2, return\_1, return\_2):
                    if timestamp >= hour\_timestamp:
                        conn.execute("""INSERT INTO totals (consumption\_1, consumption\_2, return\_1, return\_2, datetime)
                                    VALUES (?, ?, ? ,? ,?)""", \[consumption\_1, consumption\_2, return\_1, return\_2, timestamp\])
                        conn.commit()
                        logging.warning("Totals saved to Dbase")
                        hour\_timestamp = expires(60)
                        consumption\_1 = None
                        consumption\_2 = None
                        return\_1 = None
                        return\_2 = None
            except Exception as e:
                exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
                fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
                logging.warning(str(e) + " | " + str(exc\_type) +
                                " | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
                pass

        except Exception as e:
            exc\_type, exc\_obj, exc\_tb = sys.exc\_info()
            fname = os.path.split(exc\_tb.tb\_frame.f\_code.co\_filename)\[1\]
            logging.warning(str(e) + " | " + str(exc\_type) +
                            " | " + str(fname) + " | " + str(exc\_tb.tb\_lineno))
            pass

        # Check when the exclamation mark is received (end of data)
        if re.match(r'(?=!)', ser\_data, 0):
            checksum\_found = True

    ser.close()
conn.close()

Questions? Let me know! Coffee? Yes Please!

Originally the initial code came from domoticx.com but I changed it to my needs

Did you find this article valuable?

Support Theo van der Sluijs by becoming a sponsor. Any amount is appreciated!