Installation Emonpi and emonTx

pb66 · 9 February 2021 14:19

Not my strongest subject either. I used to play with it quite a bit but not recently.

@Hassanwahab, what do the following commands respond with

timedatectl

sudo i2cdetect -y 1

ls -la /dev/{tty{ACM,AMA,S,USB},serial}[0-9] 2>/dev/null

sudo journalctl -b | grep emonPiLCD

sudo systemctl status emonhub emonPiLCD mosquitto emoncms_mqtt

Could you also check that the GPIO extender that sits between the emonPi and the RPi GPIO is fitted tight to the RPi since the board will have been taken off to solder the RTC. The extender should be fully home on the RPi then the emonpi board slides down only as far as it needs to meet the standoffs.

PS as a sidenote, since the emonpi has provision for a RTC, perhaps emonScripts should have the option to install the RTC code as another install config option.

borpin · 9 February 2021 16:32

Just a thought, but did you set the clock time/date after installing the RTC. Seems like an obvious question but…

Hassanwahab · 10 February 2021 08:16

Yes, I did set clock time/date after installing the RTC.

borpin · 10 February 2021 08:17

Just post the text rather than as an attachment please.

Hassanwahab · 10 February 2021 08:17

@pb66 @borpin

root@emonpi:/# timedatectl
               Local time: Wed 2021-02-10 08:01:39 GMT
           Universal time: Wed 2021-02-10 08:01:39 UTC
                 RTC time: Wed 2021-02-10 08:01:37
                Time zone: Europe/London (GMT, +0000)
System clock synchronized: no
              NTP service: active
          RTC in local TZ: no




root@emonpi:/#sudo i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- 27 -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- UU -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --


root@emonpi:/# ls -la /dev/{tty{ACM,AMA,S,USB,serial}[0-9] 2>/dev/null


root@emonpi:/#sudo journalctl -b | grep emonpiLCD


root@emonpi:/# sudo systemctl status emonhub emonpiLCD mosquitto emoncms_mqtt
Unit emonpiLCD.service could not be found.
● emonhub.service - emonHub data multiplexer
   Loaded: loaded (/opt/openenergymonitor/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-02-08 10:37:45 GMT; 1 day 21h ago
  Process: 478 ExecStartPre=/bin/mkdir -p -m 0775 /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 498 ExecStartPre=/bin/chgrp -R emonhub /var/log/emonhub/ (code=exited, status=0/SUCCESS)
 Main PID: 503 (python3)
    Tasks: 4 (limit: 2065)
   CGroup: /system.slice/emonhub.service
           └─503 python3 /usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log

Feb 08 10:37:45 emonpi systemd[1]: Starting emonHub data multiplexer...
Feb 08 10:37:45 emonpi systemd[1]: Started emonHub data multiplexer.

● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-02-08 10:37:45 GMT; 1 day 21h ago
     Docs: man:mosquitto.conf(5)
           man:mosquitto(8)
 Main PID: 484 (mosquitto)
    Tasks: 1 (limit: 2065)
   CGroup: /system.slice/mosquitto.service
           └─484 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Feb 08 10:37:45 emonpi systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker...
Feb 08 10:37:45 emonpi systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.

● emoncms_mqtt.service - Emoncms emoncms_mqtt script
   Loaded: loaded (/var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-02-08 10:37:48 GMT; 1 day 21h ago
     Docs: https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
  Process: 670 ExecStartPre=/bin/mkdir -p ${LOG_PATH} (code=exited, status=0/SUCCESS)
  Process: 782 ExecStartPre=/bin/chown ${USER} ${LOG_PATH} (code=exited, status=0/SUCCESS)
  Process: 784 ExecStartPre=/bin/touch ${LOG_PATH}/emoncms.log (code=exited, status=0/SUCCESS)
  Process: 786 ExecStartPre=/bin/chmod 666 ${LOG_PATH}/emoncms.log (code=exited, status=0/SUCCESS)
 Main PID: 788 (php)
    Tasks: 1 (limit: 2065)
   CGroup: /system.slice/emoncms_mqtt.service
           └─788 /usr/bin/php /var/www/emoncms/scripts/services/emoncms_mqtt/emoncms_mqtt.php

Feb 08 10:37:48 emonpi systemd[1]: Started Emoncms emoncms_mqtt script.

borpin · 10 February 2021 08:18

For future reference, when posting code or bash output, please put in 3 ‘backticks’ (found at the top left of the keyboard normally) on a line on their own, then the code, then 3 more backticks on a line following the code.

    ```
    code
    ```

If it is something like php you can add a language identifier that after the first 3 backticks so ```php

borpin · 10 February 2021 08:20

So your system clock is not synchroising to the RTC chip.

Not sure how to do that but I am sure Google is your friend.

Hassanwahab · 10 February 2021 08:23

Thanks Brian,
when I configured the RTC at that time both the options were “Yes”.
I will reconfigure it again and see if the result changes or not than i will you know.

Thanks

pb66 · 10 February 2021 08:47

I would recommend using copy and paste to transfer the commands over to the SSH terminal to avoid typo’s

edit - The first closing curly brace is missing after “USB”

No obvious issues in the stuff you’ve posted so far. Just to check, you are looking at the local emoncms when you say it’s not updating, correct? Not emoncms.org?

Can you also post some emonhub.log.
~~The first closing curly brace is missing after “USB”~~

That should be emonPiLCD, note the capital “P”.

borpin · 10 February 2021 09:04

Time interests me…

Not that many folk run without a network so I am not sure I have yet got to the bottom of it. The assumption is that the RTC is a backup and that the clock still gets synced from NTP and the RTC is updated by the NTP sync.

I think your problem is that the ntp service is still active, so the system is still trying to sync the RTC from system time rather than the other way round.

First

sudo timedatectl set-ntp false

from Real-Time Clock / RTC (Linux)

If no Internet connection is provided, timedatectl can be used to set the date or time and the systemd-timedated service will make sure the new system clock is synchronized with the hardware clock (RTC) immediately

However, I am not sure this will deal with drift.

I think you need to use adjtimex or call hwclock --hctosys regularly from CRON.

Hassanwahab · 10 February 2021 09:07

Hi @pb66

Yes, i am looking at the local emoncms.

root@emonpi:/#sudo journalctl -b | grep emonpiLCD
Feb 10 08:08:41 emonpi sudo[12809]:     root : TTY=pts/0 ; PWD=/ ; USER=root ; COMMAND=/bin/systemctl status emonhub emonpiLCD mosquitto emoncms_mqtt

root@emonpi:/# ls -la /dev/{tty{ACM,AMA,S,USB,serial}[0-9] 2>/dev/null

root@emonpi:/home/pi# ls -la /dev/{tty{ACM,AMA,S,USB},serial}[0-9] 2>/dev/null
lrwxrwxrwx 1 root root 5 Feb 14 2019 /dev/serial0 → ttyS0
lrwxrwxrwx 1 root root 7 Feb 14 2019 /dev/serial1 → ttyAMA0
crw-rw---- 1 root dialout 204, 64 Feb 10 09:02 /dev/ttyAMA0
crw-rw---- 1 root dialout 4, 64 Feb 14 2019 /dev/ttyS0

2021-02-10 09:01:52,460 DEBUG    MainThread Signal 15 received.
2021-02-10 09:01:52,605 INFO     MainThread Exiting hub...
2021-02-10 09:01:52,752 INFO     MainThread Exit completed
2021-02-10 09:01:54,260 INFO     MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-02-10 09:01:54,261 INFO     MainThread Opening hub...
2021-02-10 09:01:54,261 INFO     MainThread Logging level set to DEBUG
2021-02-10 09:01:54,262 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-10 09:01:54,263 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-02-10 09:01:56,266 WARNING  MainThread Device communication error - check settings
2021-02-10 09:01:56,267 INFO     MainThread Setting RFM2Pi baseid: 5 (5i)
2021-02-10 09:01:57,269 INFO     MainThread Setting RFM2Pi frequency: 433 (4b)
2021-02-10 09:01:58,272 INFO     MainThread Setting RFM2Pi group: 210 (210g)
2021-02-10 09:01:59,274 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2021-02-10 09:02:00,276 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-10 09:02:01,278 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-10 09:02:01,279 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-10 09:02:01,281 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2021-02-10 09:02:01,284 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
2021-02-10 09:02:01,285 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
2021-02-10 09:02:01,286 INFO     MainThread Setting MQTT node_format_enable: 1
2021-02-10 09:02:01,286 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2021-02-10 09:02:01,287 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2021-02-10 09:02:01,288 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2021-02-10 09:02:01,290 DEBUG    MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
2021-02-10 09:02:01,291 DEBUG    MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
2021-02-10 09:02:01,291 INFO     MainThread Setting emoncmsorg apikey: set
2021-02-10 09:02:01,292 INFO     MainThread Setting emoncmsorg url: https://emoncms.org
2021-02-10 09:02:01,292 INFO     MainThread Setting emoncmsorg senddata: 1
2021-02-10 09:02:01,293 INFO     MainThread Setting emoncmsorg sendstatus: 1

LOG LEVEL: DEBUG

pb66 · 10 February 2021 09:09

While I won’t dispute your intent or the possibility that there’s an issue there, the time appears to be reasonably correct and if posting to local emoncms, both are using the same clock and therefore unaware of relationship to external time, plus the data is passed via MQTT without a timestamp so time becomes almost irrelevant. I wouldn’t pursue this avenue until it’s confirmed the issue is due to a time difference, ie see data with the wrong time being passed to emoncms and then ignored.

Hassanwahab · 10 February 2021 09:11

root@emonpi:/home/pi# ls -la /dev/{tty{ACM,AMA,S,USB},serial}[0-9] 2>/dev/null
lrwxrwxrwx 1 root root          5 Feb 14  2019  /dev/serial0 -> ttyS0
lrwxrwxrwx 1 root root          7 Feb 14  2019  /dev/serial1 -> ttyAMA0
crw-rw---- 1 root dialout 204, 64 Feb 10 09:02  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Feb 14  2019  /dev/ttyS0

pb66 · 10 February 2021 09:11

Sorry, the order of my comments in my last post to you got mixed up.

pb66 · 10 February 2021 09:15

emonhub is trying to listen to the bluetooth for data!

Change emonhub.conf to use /dev/serial0 not /dev/ttyAMA0 and that should give you data. But I wonder why your RPi image is different to the usual OEM way?

There may also be a baud stability issue with the way it’s configured unless the core_freq is fixed, can you post the content of /boot/config.txt please

borpin · 10 February 2021 09:16

But it will drift and is an issue with the setup of the RTC. Installing an RTC is part of the thread/conversation.

pb66 · 10 February 2021 09:20

But right now he’s trying to get data and trying to do 2 things at once just causes confusion, I suggest the current issue of no data is sorted first, then perhaps look to improve the RTC install. The fact this is NOT going to be internet connected hence the need for a RTC is also a key part of this thread, so even when perfected, the RTC still cannot sync, so it’s a tad pointless perfecting something that won’t work. Especially when it confuses the issue at hand.

I think the RTC thing is worthy of it’s own discussion as an OEM project concideration, as I said earlier

It would be useful if installing an RTC was consistent across all users despite the number being fairly low, but with improved docs and automated install, maybe there would be more takers? The shop could even offer a RTC option and solder the headers on for a fee incl the rtc.

After the “no data” issue is resolved, if the emonPi’s LCD doesn’t start working as expected I would recommend debugging that too before looking any closer at the rtc, I only asked for the rtc info to confirm 2 things, the time was correct(ish), and the rtc was functional whilst not causing an issue on the I2C bus.

Hassanwahab · 10 February 2021 10:20

Changed emonhub.conf to use /dev/serial0. These logs are below

2021-02-10 10:12:12,948 WARNING  MainThread RFM2Pi thread is dead.
2021-02-10 10:12:12,949 WARNING  MainThread Attempting to restart thread RFM2Pi (thread has been restarted 17 times...)
2021-02-10 10:12:12,950 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-10 10:12:12,952 DEBUG    MainThread Opening serial port: /dev/serial0 @ 38400 bits/s
2021-02-10 10:12:14,959 INFO     MainThread RFM2Pi device firmware version & configuration: not available
2021-02-10 10:12:14,960 INFO     MainThread Setting RFM2Pi baseid: 5 (5i)
2021-02-10 10:12:15,962 INFO     MainThread Setting RFM2Pi frequency: 433 (4b)
2021-02-10 10:12:16,964 INFO     MainThread Setting RFM2Pi group: 210 (210g)
2021-02-10 10:12:17,966 INFO     MainThread Setting RFM2Pi quiet: 1 (1q)
2021-02-10 10:12:18,969 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-10 10:12:19,971 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-10 10:12:19,972 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-10 10:12:19,977 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1 USA 0
2021-02-10 10:12:20,083 DEBUG    RFM2Pi     19 NEW FRAME : OK 5 0 0 0 0 0 0 244 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2021-02-10 10:12:20,085 WARNING  RFM2Pi     Exception caught in RFM2Pi thread. Traceback (most recent call last):
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 32, in wrapper
    return func(*args)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 101, in run
    rxc = self._process_rx(rxc)
  File "/opt/openenergymonitor/emonhub/src/emonhub_interfacer.py", line 347, in _process_rx
    elif len(rxc.realdata) % ehc.check_datacode(datacode) != 0:
  File "/opt/openenergymonitor/emonhub/src/emonhub_coder.py", line 10, in check_datacode
    return struct.calcsize(datacode)
TypeError: Struct() argument 1 must be a str or bytes object, not list

/boot/config.txt

#dtoverlay=gpio-ir,gpio_pin=17
#dtoverlay=gpio-ir-tx,gpio_pin=18

# Additional overlays and parameters are documented /boot/overlays/README

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
dtoverlay=vc4-fkms-v3d
max_framebuffers=2

[all]
#dtoverlay=vc4-fkms-v3d
dtoverlay=i2c-disable-bt
dtoverlay=i2c-rtc,ds3231

pb66 · 10 February 2021 10:34

I don’t think that’s right!

Could it supposed be dtoverlay=pi3-disable-bt (or maybe even dtoverlay=disable-bt, IIRC the former is officially superseded by the latter) ? And that is a question, I don’t know offhand what it’s intended to be exactly on the OEM sd image nor what if any changes you’ve intended.

I assume it is a Pi3, correct?

Personally i would use

enable_uart=1
core_freq=250

instead of any other serial port overlays such as those previously mentioned.

The core_freq=250 isn’t officially needed as the enable_uart=1 should fix the core_freq, but it seems there may be a current bug in the OS. (see enable_uart=1 NOT fixing core_freq to 250MHz · Issue #4123 · raspberrypi/linux · GitHub)

Can you also confirm the content of cmdline.txt

sudo cat /boot/cmdline.txt

Hassanwahab · 10 February 2021 10:52

I didn`t made any changes to it but changed it to pi3 now as advised above.

below are the results for cmdline.txt

console=tty1 root=PARTUUID=43ed7bb4-02 rootfstype=ext4 elevator=deadline fsck.re                                                                             pair=yes rootwait