Emonsd Feb24 - ttyAMA0 used by kernel causing interference with RFM / emonhub

Hi All

Sorry I’ve been so quiet for such a long time, health problems,

I’ve been debugging an issue after a fresh upgrade to emonSD Feb2024 on emonpi1.
Symtom: emonhub would stop receiving data from serial / RF seemingly randomly, daily. I would also see random inputs appear and in some cases nodeid 5 data would appear as another nodeid altogether.

Many days of debugging emonhub and atmega firmware, finally culminating in logging every character sent and received on serial and I discovered this in /boot/firmware/cmdline.txt:

cat /boot/firmware/cmdline.txt 
console=serial0,115200 console=tty1 root=PARTUUID=567d5380-02 rootfstype=ext4 fsck.repair=yes rootwait

As you can see, serial0 has the kernel using it as a console, causing spurious data to be sent to the RFM module (in this case, randomly changing config including nodeid to spurious values and occasionally datalogging interval to a very low value).
You can imagine the debugging this took to find the problem.

For reference cmdline.txt on older emonsd (2021, debian_version 10.13) looks like this:

console=tty1 root=PARTUUID=41d38177-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

This will be more prevalent on firmware that uses CM/emonlib due to the fact the first serial character is interpreted as a command (as opposed to reverse polish in jeelib based DS format firmwares).

[EDIT]
The easiest way to fix it is to run sudo raspi-config

Choose:
Option 3 Interface Options,
then Option 5 Serial Port
Answer No to ‘Login Shell Available over serial’
Answer Yes to ‘Serial Port Hardware enabled’

Then, as raspi-config suggests, reboot.

Or, edit /boot/firmware/cmdline.txt with your favourite editor and remove the console=serial0,115200 part. Reboot.

4 Likes

@TrystanLea
It would appear that this requires urgent attention.

Thank you for spotting this @alandpearson!

I will make the amendment to the image to disable this and try to work out why this got past the build process on this image. There was a big /boot related path change from /boot to /boot/firmware so it must have been related to that.

1 Like

Here’s my offending commit to the emonHub installer update install to check for boot/firmware/config location · openenergymonitor/emonhub@a6bb078 · GitHub I thought I was fixing the path for /boot/cmdline.txt but instead applied the command to /boot/firmware/config.txt

2 Likes

No problem @TrystanLea
If you are fixing the image, there are three other issues you may want to look at:

  1. I haven’t got to the bottom of this, but seems related to log2ram. It appears emonhub log files get moved regularly from the ram filesystem to /var/log.old but then logging stops altogether. I first thought it was logrotated causing the issue, but it is not. Set logging to debug on a reasonably busy system to trigger it.

  2. emonpi LCD is not correctly updated when using CM firmware, pulse values are allocated to temperature and other issues. This is caused by adding a new datacode (MSG) to the emonhub node5 config in emonhub.conf, The existing emonPiLCD1.py script pulls the old node format CSV string via MQTT, splits it and puts in array then updates LCD based on this array. It is now off-by-one. The good news is this way of doing it hasn’t been necessary in emonPiLCD1.py for years as now the individual MQTT topics for each LCD item are configured in the cfg file. Unfortunately this doesn’t work for all items… you’ll have a pull request this afternoon fixing it.
    [EDIT] PR:
    emonPiLCD1.py fix for CM firmware by alandpearson · Pull Request #149 · openenergymonitor/emonpi · GitHub

  3. Not strictly emonsd related, but RF transmission doesn’t work as it should in emonCM frontend (both LPL and RFM69 JEELIB_NATIVE packet format), breaking emonglcd and also not working with EmonHubOEMInterfacer.py.
    RF reception is fully broken (or rather the presentation of the data on serial is incorrect) when using JEELIB_NATIVE for emonpiCM firmware.
    Github pull request fixing both is here:
    emonPi_CM firmware fixes - Fix receiving in JEELIB_NATIVE mode, Fix transmit in both modes by alandpearson · Pull Request #148 · openenergymonitor/emonpi · GitHub

Enjoy :wink:

2 Likes

Thanks @alandpearson

I’ve fixed the emonhub install & update script now, including another related bug that actually skipped everything in in the emonSD_pi_env section. I’m not sure on the implication of that yet, I need to check a couple of things on a fresh image.

So at least the inital fix for this issue is now available and that is to run a full update via the Admin > Update page:

1 Like

Great.

I’ve added a link above for the PR to fix emonPiLCD1.py

If you’re in a merging mood, can you also look at this long overdue PR (you asked me to resubmit this 1 year ago… I only did it last week) for emonlibCM to add DS2438 temperature sensing devices ?

Thanks again,
Alan

1 Like