Receive zigbee sensor data on emonPi

As requested on this thread

Here’s how to receive data from Zigbee devices using emonHub on an emonPi

Hardware Required

  • Sonoff 3.0 USB Zigbee receiver CC2652P (CC2652P Chipset)
  • Zigbee Sensor, tested with Sonoff Zigbee temp sensors SNZB-02 & SNZB-02D

1. Setup receiver dongle

Code name: ZBDongle-P
Chipset: CC2652P

Other supported adaptors: Supported Adapters | Zigbee2MQTT

Flash coordinator FW

$ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e FIRMWARE_URL=https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip ckware/ti-cc-tool -ewv -p /dev/ttyUSB0 --bootloader-sonoff-usb

Requires libseccomp 2.4.2., check version with dpkg -l | grep libseccomp
See: ERROR: Download of firmware was not successful · Issue #4 · git-developer/ti-cc-tool · GitHub

Full instructions Flashing the firmware via cc2538-bsl | Zigbee2MQTT

2. Install ZigbeeMQTT on emonPi

Follow guide: Docker | Zigbee2MQTT

If running Debian Buster :

Need to install libseccomp2:

sudo apt-key adv --keyserver hkps://keyserver.ubuntu.com:443 --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
echo "deb http://httpredir.debian.org/debian buster-backports main contrib non-free" | sudo tee -a "/etc/apt/sources.list.d/debian-backports.list"
sudo apt update
sudo apt install libseccomp2 -t buster-backports

See guide: Docker | Zigbee2MQTT

3. Configure zigbee2mqtt

wget https://raw.githubusercontent.com/Koenkk/zigbee2mqtt/master/data/configuration.yaml -P data

Edit config

$ docker run -d \
   --name zigbee2mqtt \
   --restart=unless-stopped \
   --device=/dev/ttyUSB0:/dev/ttyUSB0\
   -p 8080:8080 \
   -v $(pwd)/data:/app/data \
   -v /run/udev:/run/udev:ro \
   -e TZ=Europe/London \
   koenkk/zigbee2mqtt

Example Config

homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: mqtt://192.168.86.24
  user: pi
  password: emonpimqtt2016
serial:
  port: /dev/ttyUSB0
frontend: true
advanced:
  homeassistant_legacy_entity_attributes: false
  legacy_api: false
  legacy_availability_payload: false
device_options:
  legacy: false
devices:
  '0x00124b002919285f':
    friendly_name: temp1

RasPi Config

# Home Assistant integration (MQTT discovery)
homeassistant: false

# allow new devices to join
permit_join: false

# MQTT settings
mqtt:
  # MQTT base topic for zigbee2mqtt MQTT messages
  base_topic: zigbee2mqtt
  # MQTT server URL
  server: 'mqtt://172.17.0.1'
  # MQTT server authentication, uncomment if required:
  user: emonpi
  password: emonpimqtt2016

frontend: true
# Serial settings
serial:
  # Location of CC2531 USB sniffer
  port: /dev/ttyACM0

Run

docker run -d  --name zigbee2mqtt    --restart=unless-stopped    --device=/dev/serial/by-id/usb-Silicon_Labs_Sonoff_Zigbee_3.0_USB_Dongle_Plus_0001-if00-port0:/dev/ttyACM0    -p 8080:8080    -v $(pwd)/data:/app/data    -v /run/udev:/run/udev:ro    -e TZ=Europe/London    koenkk/zigbee2mqtt

Example output

Topic: zigbeemqtt/temp1
Payload: {"battery":100,"humidity":80,"temperature":22,"voltage":3100}

Use zigbeemqtt frontend to pair new zigbee device, frontend should be available on port 8080

e.g http://emonpi.local:8080

Pair device (without frontend)

It’s also possible to pair a device using MQTT commandline

Message: zigbee2mqtt/bridge/request/permit_join
Payload: {"value": true, "device": "LCD_Temp, "time": 20}

e.g

mosquitto_pub -h bob -u pi -P emonpimqtt2016 -t zigbee2mqtt/bridge/request/permit_join -m '{"value": true}'

New device pairing will be announced on this topic

zigbee2mqtt/bridge/event

Then the device can be given a friendly name in /data/configuration.yaml

Restart zigbeemqtt

After editing config, restart by sending an empty message to: zigbee2mqtt/bridge/request/restart

4. Configure emonHub

Switch to emonHP JSON branch: MQTT JSON subscribe by glynhudson · Pull Request #198 · openenergymonitor/emonhub · GitHub

These steps shouldn’t be required once this PR is merged

cd /opt/openenergymonitor/emonhub
git pull origin master
git checkout mqtt_subscribe
git merge master

Example emonHub config:

[[MQTT_sub1]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = pi
        mqtt_passwd = emonpimqtt2016
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        node_JSON_enable = 1
        node_JSON_basetopic = zigbee2mqtt/temp1

Glynn,

thank you so much for putting this together. I plan to gather up some parts and give this a go.

One question, this process brings the Zigbee data to the emonPi2, as the input node_JSON_basetopic = zigbee2mqtt/temp1. Is the nodename on the input screen of emonCMS displayed as “temp1”?

With the individual inputs as {“battery”:100,“humidity”:72.23,“linkquality”:116,“temperature”:21.79,“voltage”:3000}`??

From there, I would create feeds as I would with any other input (name, data frequency, etc) and then the feeds are sent to the emonCMS cloud per the parameters set in emonHub?

Please confirm.

Thank you,

Gordon

This is timely: I am using Homeassistant but I wanted to find a way to separate my MQTT setup (mostly zigbee2mqtt!) from it.

Correct, but that’s just an example. You can call it whatever you want

Screenshot from 2024-06-19 17-44-04

Excellent! This is what I was hoping you would say.
Thx
Gordon

@glyn.hudson Hi Glyn,

I am starting my way through this and have run into my first issue when trying this on my emonPi2 where docker is returning the message “unknown server OS” (see below).

pi@emonpi:~ $ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e FIRMWARE_URL=https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20220219.zip ckware/ti-cc-tool -ewv -p /dev/ttyUSB0 --bootloader-sonoff-usb
docker: unknown server OS: .

I did verify that libseccomp 2.5.1 is installed. (I’m assuming that a higher version than 2.4.2 is also ok).
pi@emonpi:~ $ dpkg -l | grep libseccomp
ii libseccomp2:armhf 2.5.1-1+rpi1+deb11u1 armhf high level interface to Linux seccomp filter

Not sure how to resolve the unknown server OS error.

Thanks,

Gordon

Is docker working ok, can you try and run:

docker run hello-world

It looks like the FW URL has been updated, try:

$ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e
FIRMWARE_URL=https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip
ckware/ti-cc-tool -ewv -p /dev/ttyUSB0 --bootloader-sonoff-usb

It looks like it updated docker and it is installed correctly.
Unable to find image ‘hello-world:latest’ locally
latest: Pulling from library/hello-world
2d3d56bba6ab: Pull complete
Digest: sha256:94323f3e5e09a8b9515d74337010375a456c909543e1ff1538f5116d38ab3989
Status: Downloaded newer image for hello-world:latest

When I enter the updated URL, I get the following
flag needs an argument: ‘e’ in -e
See ‘docker run --help’.
-bash: ckware/ti-cc-tool: No such file or directory

@glyn.hudson

Hi Glyn, I tried this again. Docker is running, I have version 2.5.1 of libseccomp2 installed and I was finally able to get the Flash Coordinator to run. (I had copied the $ over with the command and ended up with $ $, causing my errors).

But now I get the error at the end “bytearrary index out of range” (see below)

pi@emonpi:~ $ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e FIRMWARE_URL=https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip ckware/ti-cc-tool -ewv -p /dev/ttyUSB0 --bootloader-sonoff-usb
Unable to find image 'ckware/ti-cc-tool:latest' locally
latest: Pulling from ckware/ti-cc-tool
fda0ff469afd: Pull complete
bd70e72a4529: Pull complete
2620ee775689: Pull complete
b63782bf4aa7: Pull complete
8ca19e7c644b: Pull complete
e0cae168a165: Pull complete
5ce45cc6e57f: Pull complete
40b8d839b7eb: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:360d92c2978d181cd86829a4fe80bc9f28418f449461aec10dedddc5c9c7b1a1
Status: Downloaded newer image for ckware/ti-cc-tool:latest
Downloading firmware from https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x                .0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip
Firmware source: 'CC1352P2_CC2652P_launchpad_coordinator_20230507.zip'
Firmware file:   'CC1352P2_CC2652P_launchpad_coordinator_20230507.hex'
sonoff
Opening port /dev/ttyUSB0, baud 500000
Reading data from CC1352P2_CC2652P_launchpad_coordinator_20230507.hex
Firmware file: Intel Hex
Connecting to target...
CC1350 PG2.1 (7x7mm): 352KB Flash, 20KB SRAM, CCFG.BL_CONFIG at 0x00057FD8
Primary IEEE Address: 00:12:4B:00:30:C6:EB:39
    Performing mass erase
Erasing all main bank flash sectors
    Erase done
Writing 360448 bytes starting at address 0x00000000
ERROR: bytearray index out of range

When looking at the full instructions for firmware flash here the only difference I can see is the Primary IEEE address in the instructions is all zeros (0) vs the actual address in my emonPi2.

Being a Linux novice, I’m not sure how to resolve this and could use your assistance.

Thanks,

Gordon

Nice work, it looks like everything was going well, until it failed :frowning:

I’m not sure what’s going on, the FW flashed first time when I tried this. But it was a while ago, so things may have changed.

Maybe try opening an issue on Github repo where the upload script comes from?:

GitHub - JelmerT/cc2538-bsl: Python cross-platform script to upload firmware via the serial boot loader onto the CC13xx, CC2538 and CC26xx SoC.

Hi Glyn,

I did post the issue on GitHub, but I think something has gotten mucked up during this process.

I can no longer use Serial Config and my Network Connections are also not working in emon (but they are working).

Serial Config does not respond to commands and just stays in the mode of “waiting for configuration from device”

image

I can access the device over my network, but when I go into Network in emonCMS it shows as everything disconnected, yet I have both Ethernet and Wifi connected and can access the device either way.

Also, when I click on “Connect to WiFi network” and scan, nothing is returned, and it goes back to the screen below.

I think I need to restore the entire unit to factory settings – just not sure how to do this.

Please advise.

Thanks,

Gordon

image001.png

The easiest way to do this is to download and flash a fresh emonSD image: emonSD Download — OpenEnergyMonitor 0.0.1 documentation

Agreed.

Will this preserve all the data on the emon or should I be backing up the data first? (and where would I find the instructions to do the backup? I’m assuming it is in a directory on the SD card that I can copy and paste onto the new SD card).

Thank you,

Gordon

You can restore the data from the old SD card using a card reader once you’re up and running with the new image, see Import / Backup / Restore / Update — OpenEnergyMonitor 0.0.1 documentation

@glyn.hudson Glyn,

I tried loading the Flash coordinator FW on a new emonPi2 and there is definitely a conflict with emon. Once I run the command below, I can no longer use the Serial Config Tool as it looks like there is a USB/Serial Port conflict.

Here I entered the command string but didn’t have the Sonoff dongle plugged in.

pi@emonpi:~ $ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e FIRMWARE_URL                       =https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0                       /bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip ckware/ti-cc-tool -ewv                        -p /dev/ttyUSB0 --bootloader-sonoff-usb
Unable to find image 'ckware/ti-cc-tool:latest' locally
latest: Pulling from ckware/ti-cc-tool
fda0ff469afd: Pull complete
bd70e72a4529: Pull complete
2620ee775689: Pull complete
b63782bf4aa7: Pull complete
8ca19e7c644b: Pull complete
e0cae168a165: Pull complete
5ce45cc6e57f: Pull complete
40b8d839b7eb: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:360d92c2978d181cd86829a4fe80bc9f28418f449461aec10dedddc5c9c7b1a1
Status: Downloaded newer image for ckware/ti-cc-tool:latest
docker: Error response from daemon: error gathering device information while adding custom device "/dev/ttyUSB0": no such file or directory.

I then plugged in the Sonoff dongle and reran the command.

pi@emonpi:~ $ docker run --rm --device /dev/ttyUSB0:/dev/ttyUSB0 -e FIRMWARE_URL=https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip ckware/ti-cc-tool -ewv -p /dev/ttyUSB0 --bootloader-sonoff-usb
Downloading firmware from https://github.com/Koenkk/Z-Stack-firmware/raw/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20230507.zip
Firmware source: 'CC1352P2_CC2652P_launchpad_coordinator_20230507.zip'
Firmware file:   'CC1352P2_CC2652P_launchpad_coordinator_20230507.hex'
sonoff
Opening port /dev/ttyUSB0, baud 500000
Reading data from CC1352P2_CC2652P_launchpad_coordinator_20230507.hex
Firmware file: Intel Hex
Connecting to target...
CC1350 PG2.1 (7x7mm): 352KB Flash, 20KB SRAM, CCFG.BL_CONFIG at 0x00057FD8
Primary IEEE Address: 00:12:4B:00:30:C6:EB:39
    Performing mass erase
Erasing all main bank flash sectors
    Erase done
Writing 360448 bytes starting at address 0x00000000
ERROR: Timeout waiting for ACK/NACK after 'Send data (0x24)'
pi@emonpi:~ $

When I go to the Serial Config Tool, it never responds with the current configuration
image

Then I unplugged the Sonoff Dongle and it looks like the serial port has disappeared.

Plugging the Sonoff dongle back in, the serial monitor starts again but never returns the configuration.

So, this leads me to believe that the emon USB/Serial gets erased or overwritten per the output below.

Primary IEEE Address: 00:12:4B:00:30:C6:EB:39
Performing mass erase
Erasing all main bank flash sectors
Erase done
Writing 360448 bytes starting at address 0x00000000
ERROR: Timeout waiting for ACK/NACK after ‘Send data (0x24)’

Can you investigate/test at your location to see what is happening and if there is a workaround to have the zigbee dongle and emon co-exist on the emonPi2?

Thanks,

Gordon

The emonPi2 uses the internal /dev/ttyAMA0 serial port, there’s no problem using other USB serial devices which connect to /dev/ttyUSB0 or /dev/ttyUSB1.

Be sure to check which port the serial monitor is reading from before opening it. When the dongle is plugged in and /dev/ttyUSB0 is present the serial monitor will read from this by default, you’ll need to manually select /dev/ttyAMA0 if you want to see the serial data from the emonPi board.

https://docs.openenergymonitor.org/emonpi2/configuration.html#configuration

Glyn,

I still cannot see any CT information with the dongle plugged in or with the dongle unplugged. I am convinced that something happens when the coordinator FW is flashed. Have you tried doing this with an emonPi2?

My plan next is to setup a new RaspberryPi (non-emon) to see if I can get it to work as this will tell me if it is a firmware issue or a conflict with emon.

When I go to Serial Config below are my choices (the zigbee dongle is NOT plugged in)

I select ttyAMA0 and start. The configuration does NOT display. Entering l and Send into the Console also does nothing.

Plugging in the Zigbee dongle does not change any of the above.

If I go to Serial Monitor without the Zigbee dongle plugged in, it shows the following.
image

With the Zigbee dongle plugged in, it shows
image

Interesting, I can’t see anything in the flash coordinator script which should affect ttyAMA0.

No, I’ve not tested this on an emonPi. I used my Linux laptop to flash the Zigbee coordinator,

FINALLY FIGURED IT OUT!

As I mentioned, I tried a brand new RaspberryPi and it too would not see the Zigbee dongle. Running the command lsusb did not list the CP210x UART Bridge so this told me something else was missing. Turns out, you have to enable Serial Port and Serial Console to get this to work.

On the emonPi2 I did the following to verify if the CP210x UART was present. It was not.
image
Using this article was key and I could activate Serial Port and Serial Console on the emonPi2.

After plugging in the Zigbee Dongle, and running lsusb, I saw the CP210x
image

I then went into Serial Config, selected ttyAMA0, and everything popped up as it should.

So, you need to add the steps in the linked article to activate Serial Port and Serial Console before doing any of the steps you have outlined at the beginning of this post.

Regards,

Gordon

Glynn,

For some reason, my emonPi2 stopped reporting current data yesterday afternoon, so this morning I downloaded the latest new image (01Feb2024), imported the data from the old SD card, and did a full update. I also verified that Serial Config is working and has the right firmware.

However, my I1 thru I6 and PF1 through PF6 inputs are all reporting NULL as I noticed from yesterday afternoon.

I am not sure what else to do here as this all started with trying to get ZigBee to work and I was hoping going with a new emon image would resolve the issue. Please help.

When I look at the Serial Console, I am not seeing any I Readings, whereas I do see the I1 thru I6 on my other sites. Also, I notice that the console says 0 of 3 Temperature Sensors found, yet I have two installed and they are reporting.

Regards,

Gordon

version = 1.6.2

hardware = emonPi2

voltage = 1phase

vCal = 101.10

assumedV = 240.00

iCal1 = 100.00, iLead1 = 0.00

iCal2 = 100.00, iLead2 = 0.00

iCal3 = 100.00, iLead3 = 0.00

iCal4 = 20.00, iLead4 = 3.20

iCal5 = 20.00, iLead5 = 3.20

iCal6 = 20.00, iLead6 = 3.20

pulse = 3, pulsePeriod = 100ms

RF = off

datalog = 9.80

json = off

Temperature Sensors found = 0 of 3

Temperature measurement is NOT enabled.

MSG:253,Vrms:240.00,P1:718,P2:263,P3:248,P4:2,P5:2,P6:2,E1:259149,E2:243538,E3:478219,E4:4750,E5:4378,E6:4484,pulse:0

MSG:254,Vrms:240.00,P1:717,P2:251,P3:238,P4:2,P5:2,P6:2,E1:259151,E2:243539,E3:478219,E4:4750,E5:4378,E6:4484,pulse:0

MSG:255,Vrms:240.00,P1:717,P2:269,P3:246,P4:2,P5:2,P6:2,E1:259153,E2:243539,E3:478220,E4:4750,E5:4378,E6:4484,pulse:0

MSG:256,Vrms:240.00,P1:717,P2:254,P3:240,P4:2,P5:2,P6:2,E1:259155,E2:243540,E3:478220,E4:4750,E5:4378,E6:4484,pulse:0

MSG:257,Vrms:240.00,P1:718,P2:251,P3:244,P4:2,P5:2,P6:2,E1:259157,E2:243541,E3:478221,E4:4750,E5:4378,E6:4484,pulse:0

MSG:258,Vrms:240.00,P1:716,P2:254,P3:255,P4:2,P5:2,P6:2,E1:259159,E2:243541,E3:478222,E4:4750,E5:4378,E6:4484,pulse:0

MSG:259,Vrms:240.00,P1:718,P2:256,P3:243,P4:2,P5:2,P6:2,E1:259161,E2:243542,E3:478222,E4:4750,E5:4378,E6:4484,pulse:0

MSG:260,Vrms:240.00,P1:718,P2:251,P3:262,P4:2,P5:2,P6:2,E1:259163,E2:243543,E3:478223,E4:4750,E5:4378,E6:4484,pulse:0

MSG:261,Vrms:240.00,P1:717,P2:248,P3:243,P4:2,P5:2,P6:2,E1:259165,E2:243544,E3:478224,E4:4750,E5:4378,E6:4484,pulse:0

MSG:262,Vrms:240.00,P1:719,P2:4705,P3:4452,P4:3,P5:2,P6:2,E1:259167,E2:243556,E3:478236,E4:4750,E5:4378,E6:4484,pulse:0

Here is my INPUT screenshot.