EmonPi - Local emonCMS not getting data

Hardware emonPi
1

Hi,

I have had an emonPi running for a couple of years, primarily configured to track our solar and house consumption, but it has recently stopped working and I wondered if anyone has any ideas. Sorry for the lengthy post!

We recently had some work done on our consumer unit which meant that the emonPi got turned off. Upon reconnecting it no feed data appeared to be being received. I power cycled (shutting down first!) the unit several times but it did not appear to make much difference. I had noticed in the past that occasionally the LCD screen would get some ‘strange’ / garbled characters on it, but I had ignored it.

I was running a 2018 image, and so I took the opportunity to upgrade the base image to the 2021 version. When I tried to upgrade the FW programmer avrdude kept reporting errors and timed out.I followed the manual upgrade instructions and it got half way and then hung. Still later I tried again manually and it appeared to work.

I then set-up the unit and got data - for about 5 minutes. Looking in emonhub it appeared as though the pi had stopped receiving any data as the messages stopped scrolling up the screen, not sure whether that is the emonPi not sending it or emonHub stopped working, though I did try to restart emonHub and it made no difference.

As it appeared to be an occasional issue I tried to think what may cause the issue.

  1. I have tried a different power supply.
  2. Perhaps it may be an intermittent serial connection between the emonPi and the Pi itself? I disassembled the two boards and reseated the 0.1" header extension before reassembling. I thought I had cracked it, but it only ran for about 15 minutes before it stopped again.

If anybody has any other ideas they would be very much welcomed. If you have the time and need me to post any debug information, please just let me know.

Thank you very much in advance!

I would like to purchase an emonTx to add some additional channels, but at the moment there is no point as it does not run for very long.

Thank you in advance!

2

Welcome, Neil, to the forum.

Sorry to read you’re having problems. Did you manage to do a controlled shutdown of the emonPi when the work started, or did the electrician just turn everything off? Possibly more to the point, did the power come on and off a few times while they were working on the C.U?

From what you’ve written, and the emonPi experts (which I’m not) might have a different view, but I suspect the best course of action will be to get a new SD card and flash the latest image to that, and check how it performs. If all is OK, then you can look at using the backup and restore procedures to lift the old data from the 2018 SD card. Even if it isn’t OK, we know the starting point exactly, which will make diagnosing the problem very much simpler. It might well be that the SD card has suffered some damage that’s preventing it from being written to in parts, in which case you might not be able to recover all of the data even then.

3

Hi Robert,

Thank you for the welcome!

I should have mentioned … I used a brand new SD card when I installed the latest image (2021), I still have 2018 installed on the original card.

The power was just turned off … it should not have had its power turned off at all … with hindsight I should have shut it down just in case.

When the feeds stop responding the LCD screen just shows ‘-’ next to the parameters that it displays - when it is working it shows the correct values or 0 if the CT’s etc are not plugged in.

4

I’m still not certain of the train of events.

You flashed a new SD card with the 2021 image, then you manually tried to upgrade it?
Between putting the new SD card in, did you power it and leave it for a very long time to allow it to automatically update itself, or did you immediately (or very quickly) jump in and try to do things?

If the latter, I think you might have halted the automatic update and left the system in an unknown state. In that case, flash the SD card again, put it in, power it up and leave it for a long time (how long, I’m not sure, but it’s much more than a few minutes, or even a few tens of minutes.)
If you are re-flashing the SD card, put a plain text file (it can be empty) called “ssh” in the /root directory before you install the card in the Pi - this will allow SSH access while it is updating and afterwards, which is always useful.
(Some details here: EmonPi and SSH and I find a pair of key files extremely useful - without a password set, they authorise the machine you’re using to access the Pi, not the user.)

If you’ve still got a problem, do you have an FTDI - USB programmer ( Development Boards - Programmers - Shop | OpenEnergyMonitor from The Shop, or its predecessor)? That would allow you to run the “Emon” front end separately from the Pi and establish beyond doubt whether the data was being generated by the front end.

5

Hi Robert,

I think I have confused things.

The emonPi was running the 2018 image. The power was turned off, (though I am not completely convinced that this is the cause of the issue), and when the power was re-applied the unit powered up OK, but the LCD was not reporting the CT’s power, and the feeds were not being updated in the web page. I restarted the unit a couple of times (shutting it down properly).

I had been meaning to update to the latest version of the SW (2021) so I took the opportunity to image a brand new SD card with 2021.

I let the unit update itself (the LCD display was displaying that it was updating), however when I checked the firmware in the Arduino had not updated only the PI, so I manually tried to update the Arduino FW only. In the upgrade log the avrdude output indicated that it did not update - it failed with some sync errors. I tried this twice more and on the final attempt it appeared to work.

The whole unit was then restarted. The feeds worked for about 5 minutes ( I thought I had it working!) and then stopped updating. I have restarted it a number of times, sometimes the feeds work to start with and then stop, or sometimes they never start.

I have got a TTL to USB serial board somewhere around - that is the direction I was thinking of going, or possibly 'scoping the serial lines to determine whether the Arduino stops sending data or not to try and isolate the issue. I wondered whether it was an issue that had been seen before …

6

I have looked again in the emonHub log and on this occasion I have noticed that I get this error …

2021-10-23 14:19:46,057 INFO MainThread Creating EmonHubJeeInterfacer ‘RFM2Pi’
2021-10-23 14:19:46,058 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-10-23 14:19:48,061 WARNING MainThread Device communication error - check settings

But it has worked for a short period of time. I was starting to wonder whether it is something like a dry joint, but perhaps I should re-install it again.

7

Even when I thought that the ATMega had reprogrammed successfully there are still some error messages at the bottom, are they expected?

avrdude-original: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude-original: Device signature = 0x1e950f (probably m328p)
avrdude-original: safemode: lfuse reads as 0
avrdude-original: safemode: hfuse reads as 0
avrdude-original: safemode: efuse reads as 0
avrdude-original: NOTE: “flash” memory has been specified, an erase cycle will be erformed

  •              To disable this feature, specify the -D option.*

avrdude-original: erasing chip
avrdude-original: reading input file “latest.hex”
avrdude-original: input file latest.hex auto detected as Intel Hex
avrdude-original: writing flash (18526 bytes):

Writing | ################################################## | 100% 2.66s

avrdude-original: 18526 bytes of flash written
avrdude-original: verifying flash memory against latest.hex:
avrdude-original: load data flash data from input file latest.hex:
avrdude-original: input file latest.hex auto detected as Intel Hex
avrdude-original: input file latest.hex contains 18526 bytes
avrdude-original: reading on-chip flash data:

Reading | ################################################## | 100% 2.01s

avrdude-original: verifying …
avrdude-original: 18526 bytes of flash verified

avrdude-original: safemode: lfuse reads as 0
avrdude-original: safemode: hfuse reads as 0
avrdude-original: safemode: efuse reads as 0
avrdude-original: safemode: Fuses OK (E:00, H:00, L:00)
strace: |autoreset: Broken pipe
strace: |autoreset: Broken pipe
strace: |autoreset: Broken pipe
strace: |autoreset: Broken pipe
strace: |autoreset: Broken pipe

avrdude-original done. Thank you.

strace: |autoreset: Broken pipe

However on previous occasions I have got errors such as this

avrdude-original: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude-original: Device signature = 0x1e950f (probably m328p)
avrdude-original: safemode: lfuse reads as 0
avrdude-original: safemode: hfuse reads as 0
avrdude-original: safemode: efuse reads as 0
avrdude-original: NOTE: “flash” memory has been specified, an erase cycle will be

  •              To disable this feature, specify the -D option.*

avrdude-original: erasing chip
avrdude-original: reading input file “latest.hex”
avrdude-original: input file latest.hex auto detected as Intel Hex
avrdude-original: writing flash (18526 bytes):

Writing | #### | 8% 0.22s
avrdude-original: stk500_paged_write(): (a) protocol error, expect=0x14, resp=0x38
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_recv(): programmer is not responding
avrdude-original: stk500_recv(): programmer is not responding
avrdude-original: stk500_recv(): programmer is not responding
^Cstrace: |autoreset: Broken pipe

I think I am correct in saying that the re-programming and data feeds are sent over the same interface, and so if there was an intermittency here it could be causing both issues?

8

I’m not totally familiar with the use of Avrdude, but ending with “verified” and then “broken pipe” is normal, so it looks as if it did reprogram the “emon” part properly.

Everything does go via the same serial connection, so it could be a bad connection, it could also be something in the Pi attempting to use the serial port at the same time. But that shouldn’t happen unless you were using the Serial Monitor in Admin, or you’d gone into the Pi via SSH and then worked the serial port with miniterm.

I’ve not responded to anything like this before, and I can’t remember seeing anything like it before.

I think you need to find the programmer, connect it to the “emon” board only (remove it from the Pi) and you’ll need the 5 V d.c. to power it - it won’t take its power from the programmer. If you don’t have the shop programmer, beware there’s a big “Gotcha” with the pin labelling on the emon pcb: the Tx and Rx pins are crossed between the connector and the '328P, so the pin labelled “Rx” actually transmits the data because it’s looking for the Rx pin on the programmer, likewise the pin labelled “Tx” is looking for the Tx pin from which it expects to receive data.

Then check for data coming out, there is nothing required from the programmer/Pi to start it off. You will need the Arduino IDE ideally (I can’t help you if you choose Platformio - it screwed my system up totally, got deleted pronto), but any “Serial Terminal” program on your computer should show you serial data (38400 baud, 8, none) and you should get “OK 5” and a stream of 36 bytes as 1-3 decimal digits each, finally “(-0)”, the signal strength, every 5 or 10 s - the same as appeared in the emonHub log.

If that fails after so many minutes, we know it’s a fault in the front end. If not, and it’s not silly (like the extender doesn’t reach - easy enough to feel if it’s right as you mate the two boards together) I doubt I can help you any more, it’s going to be a Pi expert who’s needed.

9

Thanks Robert,

I have got it set up with just the emonPI on its own … I will wait and see what happens. I guess that I can also monitor the serial output when connected to the Pi. Anyway I need to wait the 10-15 minutes!

emonPi V2.92
OpenEnergyMonitor.org
startup…
LCD found i2c 0x27
CT 1 Cal: 90.90
CT 2 Cal: 90.90
VRMS AC ~247.58V
AC Wave Detected - Real Power calc enabled
Vcal: 256.80
Vrms: 230V
Phase Shift: 1.70
no CT detected
Detect 1 DS18B20
RFM69CW Init:
Node 5 Freq 433Mhz Network 210

Available commands:

  • i - set node IDs (standard node ids are 1…30)*
  • b - set MHz band (4 = 433, 8 = 868, 9 = 915)*
  • g - set network group (RFM12 only allows 212, 0 = any)*
  • c - set collect mode (advanced, normally 0)*
  • …, a - send data packet to node , request ack*
  • …, s - send data packet to node , no ack*
  • …, p - Set AC Adapter Vcal 1p = UK, 2p = USA*
  • v - Show firmware version*
  • q - set quiet mode (1 = don’t report bad packets)*
    OK 5 0 0 0 0 0 0 1 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 22 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 26 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 24 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 7 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 90 94 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
    OK 5 0 0 0 0 0 0 172 93 206 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)

Thanks again for your thoughts and ideas. Will let you know how it goes …

10

You must not connect two serial outputs together!

Is it still running 3½ hours later?

11

Hello @kempni @Robert.Wall

It looks like something is interfering with that firmware upload process causing it to fail part way through the upload. This can happen if there is another service still connected to that serial port that has not exited properly, the main culprits would either be emonhub or the serial monitor feature available from the emoncms admin page.

Hopefully your latest test returns a more stable result, it looks like you did eventually get a full firmware upload? It should then be possible to put it all back together…

I wonder if the stopping after 5 mins is the updater process running the firmware upload? If it keeps happening could you check if the emoncms updater is being run at the same time, you should see the update in process on the emoncms log page…

12

Thank you for your comments.

An update! I re-imaged the SD card and did not restore my old data, and just configured the unit again. I tried running it for 20 minutes, and I thought voila it is all running!

I boxed everything back up and put it back in its home. After about 20 minutes everything stopped responding! Co-incidentally the unit did do another update cycle … so perhaps it could have been that.

I now have it fully operational running open-frame in its home whilst monitoring the serial output from the emonPi (Tx from the emonPi only) and it has been running fine for an hour. if it dies again I should be able to narrow the issue down a bit.

So it could be …

  1. An errant update (?)
  2. Something that only happens when it is boxed up, perhaps mechanical stress on a joint exacerbated by heat, or an unintentional connection etc.
  3. None of the above!
13

Small update.

I left everything open-frame and it ran all night and most of today. I put it all back together in the case and it did not send any sensor data!

‘Fiddling’ with the boards in the case caused it to work after a reboot.

Again the unit worked for a while and then stopped responding. Investigating in the failed state there was no sensor data being sent over the serial bus from the emonPi to the Raspberry Pi.

Rebooting the unit once in this state led to the LCD not initialising (single line of blocks) and some strange characters over the serial interface.

▒▒▒▒▒▒%▒^▒$▒▒▒)▒)▒׹▒{K_'▒▒▒c▒^/▒9▒A▒▒ !)▒▒▒▒ (0▒▒o▒=!▒▒▒▒nPi V2.92
OpenEnergyMonitor.org
startup…
▒@l▒@▒▒չ▒5▒▒em▒nPi V2.92
OpenEnergyMonitor.org
startup…
▒▒r▒с2▒չ▒5R▒▒▒}▒A▒▒▒▒rʒj▒▒
  W▒▒▒▒▒▒V▒S▒+▒▒K▒▒K▒VHhW,.]]▒…
emonPi V2.92
OpenEnergyMonitor.org
startup…
LCD not found
CT 1 Cal: 90.90
CT 2 Cal: 90.90
VRMS AC ~0.87V
AC NOT detected - Apparent Power calc enabled
Assuming VRMS: 230V
Detected 1 CT’s0 DS18B20 detected
RFM69CW Init:
Node 5 Freq 433Mhz Network 210

Available commands:
  <nn> i - set node IDs (standard node ids are 1…30)

Note that when it did start working again the LCD was not detected. In addition I have noticed that the emonPiLCD service appears to stop. Is it possible that the I2C bus suffers from an issue which stalls the micro on the emonPi board and causes the service on the Pi to halt?

14

Now that’s really weird.

Just to make sure I’ve got it right, it worked fine out of the case. As soon as you put it back in, it fails.

Did you completely reassemble the case, or did you leave the metal end off?
That might seem like a very funny thing to write, the reason is the anodising on the aluminium is an effective insulator, so without the metal ends and the screws biting through, it’s unlikely that there would be an electrical connection between the two sides via the metal end.

The case is not necessarily a good screen.
Checking around my fully assembled emonPi with a multimeter, I see continuity between the top and bottom extrusions, but no connection to the front plate, nor the bottom plate.
I have continuity between the USB 0V, the USB shell and the antenna, but no connection to any case parts.

15

Hi Robert
I am not sure that I have proven a causal relationship between being in the case and the unit stopping working, however it certainly seems to be that way - but sometimes straightaway, sometimes after a while.

I should keep better records! I had started to believe that putting everything in the case was forcing the cabling between the LCD and the emonPi board to suffer a different pressure / arrangement and that some intermittency here was causing the issue.

However last night I assembled the unit without the LCD plate in the housing, screwed both ends on the case … it ran for a couple of hours and then stopped.

The emonHub log does not look correct … an excerpt below …

195 183 41 8 208 43 162 140 41 114 161 41 241 206 134 188 225 44 76 166 83 170 129 19 193 199 43 163 82 100 138 197 177 88 216 25 2 80 65 174 16 22 39 218 8 0 79 221 138 18 9 76 45 76 211 92 7 126 137 67 65 43 137 30 33 122 23 11 64 128 52 8 137 40 105 11 7 160 60 162 2 201 255 11 76 6 25 146 12 41 156 155 112 68 216 8 24 58 80 28 66 36 241 25 7 3 239 65 96 40 243 1 31 184 220 24 209 214 173 68 9 18 198 181 82 192 70 145 219 42 188 228 7 32 3 20 64 97 215 70 225 72 143 107 242 146 165 177 176 86 17 49 38 146 230 31 164 0 113 14 112 243 222 67 82 162 132 140 230 154 21 26 3 74 55 72 146 29 199 3 16 88 215 160 76 3 82 144 203 243 147 108 162 187 126 9 84 65 170 172 34 75 72 129 118 193 151 3 44 114 20 55 35 122 10 204 98 38 80 129 11 237 75 88 107 24 7 139 134 25 227 131 102 98 149 30 112 199 95 231 145 174 6 25 8 34 15 155 160 11 30 2 42 240 239 37 97 42 9 3 192 25 130 42 52 91 238 53 142 50 194 26 16 58 108 23 45 36 1 10 50 120 199 3 132 122 87 0 134 154 142 21 132 65 156 4 64 127 121 203 147 65 110 236 8 69 71 31 64 52 2 189 128 90 219 113 137 78 104 28 203 140 127 14 128 55 15 108 204 99 96 75 214 84 1 248 10 191 139 67 22 126 3 88 160 121 71 159 129 112 7 10 71 250 15 186 88 104 97 144 (-0)
2021-10-29 09:25:13,034 WARNING RFM2Pi 4538 RX data length: 13997 is not valid for datacodes [‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘h’, ‘L’]

This is really puzzling and frustrating me!

image
image887×190 11.6 KB

In the image it can be seen where it ran for quite a long time yesterday and then stopped updating after 19:00. You can then see where I turned it off, removed the LCD, reassembled and set it running, and then sometime between 1:00 and 3:00 the data that was being recorded ‘froze’ again.

16

That’s the under-statement of the day :open_mouth: It’s very definitely not right. I can’t see any pattern to it, it’s not text, it looks fairly random to me. That has definitely come out of the “emon” front end (it’s the front end that puts the “(-0)” on the end).

and now

plus the gibberish output, makes me think a bad joint or a broken track on the pcb.

I think @TrystanLea needs to get involved again.

17

Hello @kempni certainly seems like strange behaviour and sounds like a fault with the hardware somewhere. Would you be happy to send the unit back to us so that we can investigate and send you a replacement in the meantime?

1 Like
18

Hi Try this.

https://community.openenergymonitor.org/t/inputs-are-not-being-updated-after-a-reboot/17907/2

Looks like in updating the Software/OS Serial port has been turned off as in my case. Hope it work for you.

19

Hi David,

Thank you for the ideas. Some of the time it was working, which leads me to believe that the serial port is not permanently switched off.