Yesterday evening, the EmonPi quit posting measurements from both wireless EmonTH and optical pulse counter. There is a tell-tale line in emonhub.log when restarting:
2021-04-06 07:18:15,366 WARNING MainThread Device communication error - check settings
Sure enough, also any attempts to reflash firmware to either RFM69 or emonPi fail:
Start ATmega328 serial upload using avrdude with latest.hex
Attempt: 1
avrdude -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/opt/openenergymonitor/emonpi/firmware/compiled/latest.hex
Note: progress written to logfile /var/log/emoncms/firmware.log
avrdude-original: Using autoreset DTR on GPIO Pin 7
strace: |autoreset: Broken pipe
ERROR: Not in sync
Needless to say, numerous power cycles have been made. Server software has been updated. Mosquitto and emoncms both are working OK. There is a “waiting to connect” (or something like that) on the LCD status display after reboot. Network connection is working, and the emoncms admin web interface (and ssh) are responding just fine. I did not notice any mains power glitch at the time when the unit stopped working.
Is there anything more I can try in order to get the board back to life?
You should stop emonhub first, because that will compete for the serial port:
sudo service emonhub stop
and then restart it afterwards sudo service emonhub start
The only further thought I have is if you have a programmer, try programming the “emon” part separately, connected via the programmer directly to your computer. To do that, you must disassemble your emonPi, separate the “emon” part from the Pi, then connect the programmer and the 5 V USB power. And you must have installed the Arduino IDE and libraries onto your computer, and downloaded the sketch from Github.
Note: the baud rate to program the ATMega is different to the baud rate it uses to communicate the data. If you can program it and then it refuses to communicate, you need to change the baud rate in emonhub.conf. Several different values have been used over time: 38400 and 9600 are the likely ones.
I think the board is really dead or very well pretending to be.
I also sort of found a cause, too. I remembered that did manually turn off a load (water boiler) from the switchboard at the same time emonPi stopped responding. I have an override to switch the loads to night/day mode. Usually, the boiler is only used by night, but when we have unusually high hot water consumption, I can force the “night mode” on. Now I turned it back off, and seemingly killed my emonPi.
How that would cause a spike to kill it, is a mystery, however. The only physical connection to emonPi was the power adapter in an outlet close to the switchboard. Other connections would be ethernet cable and the optical pulse reader.
I already ordered a new board, but it would still be nice to know how to further diagnose the now silent one.
Very true. I cannot ever recall having heard/read of the ATMega being damaged - in any way. Where did the spike come from? The water boiler is surely a resistive load, and as such it will not generate a spike on its own, but have you got anything like a harmonic filter that contains inductors in your supply, I’d be very surprised if one of those generated a spike but it is not inconceivable.
It is not responding when you connect the programmer directly - not via the RPi? Unless you have the full Atmel diagnostic software and the ISP programmer interface (as distinct from the FTDI programmer), I doubt that you can do any more.
Is the serial interface of your Pi still good? Can you loop back Tx-Rx and use something like Minicom/Miniterm to confirm?
This is a good point. I have yet to unmount the box from its position, since it is still handling some data handling scripts and other measurement data pushed to it. Fortunately, I do have a spare RPi board, in case the serial interface indeed proves to be the reason for the failure. I might have time tomorrow to examine further.
I switched the RPi board to a known good one, still no success. So I must assume that the emonPi board is faulty, in one way or another. Have to wait for the new board, then.
I am mildly pissed. Got the new board, but the same old error message. So I bit a bullet and reinstalled everything from scratch, since I still could not find anything useful in the log files. Now everything works again.
I don’t really appreciate this kind of magic, I would much rather know what caused the problem and how to fix it without complete reinstall. Needless to say, I updated the OS and software before this, with no success.
Well, at least now I have a working spare system with both RPi and EmonPi.
Hello @luru argh that’s frustrating! but good to hear that you now have it back up and running. I’m as confused as you are as to why the (I assume emonSD image) reinstall fixed the issue? If you want to send the emonpi board back for a refund your more than welcome, not sure if that’s worthwhile with postage costs? sorry that a more rational explanation is not more apparent!
