Reading your posts again, Im a little confused by these points, did the data come through again after the reboot, or was it the hardware reboot (power cycle or reset button?) that bought the data back online again?
I don’t think I can be of much help - I’ve never studied the sketch in the RFM69Pi. I assume it’s much the same as the emonPi, but even while working on the emonPiCM, I didn’t go into it in great detail. I’ve studied the transmit side of the “classic” JeeLib, but never the receive side.
If I had to make a guess, I’d say that by some mechanism, a character in the message got lost and it’s looking for data that’s never going to come and hasn’t timed out - or it has timed but doesn’t tell anybody.
1 Like
No, I’m fairly happy with the way I have the system set up. Actually I don’t need an alert because I already get one. I have a script that runs hourly and sends email if it spots any problems. So I had an email every hour from 22:00 last night - unfortunately I hadn’t looked at them 
No it’s a new problem for me, so no last time.
Yes the data came through after a sudo reboot entered via ssh. No power interruption. Last power interruption was a few days ago after a power cut, which the pi survived because of its battery, but unfortunately it lost its LAN connection stack, so needed a hard reboot since I couldn’t access it over the network.
1 Like
One thing I did notice were three new unknown nodes on the input page, but I’ve no idea when they were generated since I don’t normally look at that page. Incidentally, it’s very tedious deleting each input one at a time until eventually the node winks out of existence. Is there any way to delete the entire spurious node more simply?
Put a smart plug on the supply side and just do a hard reset 
.
Phantom (and duplicate) inputs can be dealt with the same way, using the “clean” input api
the api can be called eg by daily cron and any inputs without a processlist will be deleted, so put a redundant process in any legitimate but unused inputs to keep things tidy eg “reset to original” will effective have no effect other than to save an input from deletion.
Ok, so that suggests an old image and explains the lack of logs following a reboot. I still wouldn’t rule out the rfm browning out just yet just because of a UPS, the rfm is far more sensitive to low voltage than a Pi, if the UPS (or any other power source) was operating at the lower end of an acceptable voltage range and dipped just momentarily, the rfm wouldn’t be able to recover. Even a good PSU/UPS can deteriorate with time and/or even temporarily with cold temps. I never rule out PSU issues until I can prove otherwise.
If it happens again please try and save any emonhub.logs, check the service with sudo systemctl status emonhub, try changing a setting in emonhub.conf and try the reset_rfm2pi script in that order and record any info, before resorting to a reboot or power cycle. Perhaps automate these steps with your alert until we get to the bottom of it. If you are near the device when it stops, can you see if the rfm2pi LED is on, off or flashes intermittently (not the pi led) the rfm2pi led is tiny so maybe familiarise yourself with it whilst things are working first.
If you pre-install the script in the thread I linked, then resetting the rfm2pi is just one command reset_rfm2pi, there’s nothing bad in there, the script’s content is in that thread so you can check it out before installing.
Can you also make sure the emonhub logging level is set to debug in emonhub.conf and also set quiet = false so we can see if there is other traffic that maybe interfering with the rfm2pi too.
I would also be keen to know what FW is installed on the rfm2pi, this is also logged to emonhub.log at startup eg
Jan 15 15:44:56 docker4 emonHub[15767]: INFO MainThread EmonHub emonHub (emon-pi variant) v2.1.5
Jan 15 15:44:56 docker4 emonHub[15767]: INFO MainThread Opening hub...
Jan 15 15:44:56 docker4 emonHub[15767]: INFO MainThread Logging level set to DEBUG
Jan 15 15:44:56 docker4 emonHub[15767]: INFO MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
Jan 15 15:44:56 docker4 emonHub[15767]: DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
Jan 15 15:44:58 docker4 emonHub[15767]: INFO MainThread RFM2Pi device firmware version: [RF12demo.13]
Jan 15 15:44:58 docker4 emonHub[15767]: INFO MainThread RFM2Pi device current settings: E i5 g210 @ 433 MHz q1
[borrowed from Hanging hardware, requiring a power cycle - #3 by Guff666]
Also Dave, there should/could be another rotated emonhub log /var/log/emonhub/emonhub.log.1 as emonhub rotates the logs at 5mb.
A smart plug won’t achieve anything since the pi has battery backup.
1 Like
It’s a relatively new unit that has the pi permanently powered from the battery and charges the battery in the background. (It’s an Adafruit PowerBoost 1000 Charger The green light (full charge) is normally on and was when I looked about two hours before the RF failure. The battery would last longer than that (has in previous tests). There was no power failure/glitch noticed by any other system or by the MK1 eyeball
I know which the rfm2pi LED is. I used to watch it but now I can’t because I finally got around to cutting a slot in the case lid so the antenna can poke up so I can fit the lid to keep the dust out. Whose bright idea was it to supply a black case so you can’t see what’s inside? I’m looking for a transparent replacement, ideally just a lid but I’ve no idea where to get that and Adafruit seem to be discontinuing their case but I haven’t got to the bottom of that yet. I don’t know what else would fit.
That’s what kills emon-systems - mine just records errors and warnings because otherwise it fills disks. So I’m extremely reluctant to do that. That’s why the firmware version isn’t logged.
Mmmmm, ok so to recap. We have no LED accessible, no emonhub logs available and you’re not willing to even consider the most common issue of low voltage to rfm. All I can say is I hope it doesn’t happen again and it was just a glitch, because hope is just about all we got in the toolbox.
I have no idea what would give you that impression, the emonhub.logs have been, by far, without doubt, the most useful debug tool this project has had for many years. It is the only piece of software on the emonsd images that rotates it’s own logs when needed and guarantees it cannot possibly fill the 40mb log partition with an absolute max of 2 files under 5mb each. (unless you mess with the logging to use syslog)
I understand running with warning level logs when all is well, but when trying to debug, you need debugging level info for a fighting chance, so even if only temporarily, even just to confirm there is nothing of use there, you really do need to set loglevel to debug. You can wait so see if it occur again or occurs more frequently first if you choose, but with the IT equivalent of shutting your eyes and sticking your fingers in your ears you’re unlikely to find many clues.
1 Like
I have some data to add to the thread. I’ve now had this happen to my Pi three times, but the last time I was able to intervene.
This is not a standard emonHub. I have the RFM2Pi board installed on a Pi3B that is running some Docker containers. emonHub is installed as a systemd service. The Pi has a LiFePO4wered Pi UPS connected. The Pi was running happily for nearly a year before I added the emonHub to it.
The symptoms are that no mqtt data flowed after 04:59 this morning. When I realised and checked:
- The emonhub service was running
- The log file shows nothing after 04:59
I restarted the service:
- The log showed the service configuring the RFM2Pi board
- but no telemetry was received
I then changed a setting in emonhub.conf (I changed quiet = true in the RFM2Pi config). Once I restarted the service, telemetry started to flow again.
Here’s the log from just before it stops to now:
2021-02-01 04:59:12,323 DEBUG RFM2Pi 55621 NEW FRAME : OK 24 174 0 23 0 199 1 28 0 1 0 0 0 (-59)
2021-02-01 04:59:12,325 DEBUG RFM2Pi 55621 Timestamp : 1612155552.323061
2021-02-01 04:59:12,326 DEBUG RFM2Pi 55621 From Node : 24
2021-02-01 04:59:12,328 DEBUG RFM2Pi 55621 Values : [17.400000000000002, 2.3000000000000003, 45.5, 2.8000000000000003, 1]
2021-02-01 04:59:12,329 DEBUG RFM2Pi 55621 RSSI : -59
2021-02-01 04:59:12,330 DEBUG RFM2Pi 55621 Sent to channel(start)' : ToEmonCMS
2021-02-01 04:59:12,331 DEBUG RFM2Pi 55621 Sent to channel(end)' : ToEmonCMS
2021-02-01 04:59:12,428 DEBUG MQTT Publishing: emon/emonth6/temperature 17.400000000000002
2021-02-01 04:59:12,430 DEBUG MQTT Publishing: emon/emonth6/external temperature 2.3000000000000003
2021-02-01 04:59:12,434 DEBUG MQTT Publishing: emon/emonth6/humidity 45.5
2021-02-01 04:59:12,436 DEBUG MQTT Publishing: emon/emonth6/battery 2.8000000000000003
2021-02-01 04:59:12,438 DEBUG MQTT Publishing: emon/emonth6/pulsecount 1
2021-02-01 04:59:12,440 DEBUG MQTT Publishing: emon/emonth6/rssi -59
2021-02-01 04:59:12,441 INFO MQTT Publishing 'node' formatted msg
2021-02-01 04:59:12,442 DEBUG MQTT Publishing: emonhub/rx/24/values 17.400000000000002,2.3000000000000003,45.5,2.8000000000000003,1,-59
2021-02-01 04:59:20,945 DEBUG emoncmsorg Buffer size: 3
... first service restart
2021-02-01 14:52:40,444 DEBUG MainThread Signal 15 received.
2021-02-01 14:52:40,522 INFO MainThread Exiting hub...
2021-02-01 14:52:40,748 INFO MainThread Exit completed
2021-02-01 14:52:43,221 INFO MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-02-01 14:52:43,222 INFO MainThread Opening hub...
2021-02-01 14:52:43,223 INFO MainThread Logging level set to DEBUG
2021-02-01 14:52:43,223 INFO MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-01 14:52:43,225 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-02-01 14:52:45,232 INFO MainThread RFM2Pi device firmware version: [RF12demo.13]
2021-02-01 14:52:45,234 INFO MainThread RFM2Pi device current settings: E i5 g210 @ 433 MHz q1
2021-02-01 14:52:45,235 INFO MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:52:46,238 DEBUG MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-01 14:52:46,239 DEBUG MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-01 14:52:46,241 INFO MainThread Creating EmonHubMqttInterfacer 'MQTT'
2021-02-01 14:52:46,243 DEBUG RFM2Pi acknowledged command: > 1p
2021-02-01 14:52:46,245 DEBUG MainThread Setting MQTT pubchannels: ['ToRFM12']
2021-02-01 14:52:46,247 DEBUG MainThread Setting MQTT subchannels: ['ToEmonCMS']
2021-02-01 14:52:46,248 INFO MainThread Setting MQTT node_format_enable: 1
2021-02-01 14:52:46,248 INFO MainThread Setting MQTT nodevar_format_enable: 1
2021-02-01 14:52:46,249 INFO MainThread Setting MQTT nodevar_format_basetopic: emon/
2021-02-01 14:52:46,249 INFO MainThread Setting MQTT node_JSON_enable: 0
2021-02-01 14:52:46,251 INFO MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2021-02-01 14:52:46,252 DEBUG MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
2021-02-01 14:52:46,253 DEBUG MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
2021-02-01 14:52:46,255 WARNING MainThread Setting emoncmsorg apikey: obscured
2021-02-01 14:52:46,255 INFO MainThread Setting emoncmsorg url: https://emoncms.org
2021-02-01 14:52:46,256 INFO MainThread Setting emoncmsorg senddata: 1
2021-02-01 14:52:46,257 INFO MainThread Setting emoncmsorg sendstatus: 1
2021-02-01 14:52:46,551 DEBUG RFM2Pi acknowledged command: <nn> i - set node ID (standard node ids are 1..30)
2021-02-01 14:52:46,655 DEBUG RFM2Pi acknowledged command: <n> b - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:52:46,759 DEBUG RFM2Pi acknowledged command: <nnnn> o - change frequency offset within the band (default 1600)
2021-02-01 14:52:46,967 DEBUG RFM2Pi acknowledged command: <nnn> g - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:52:47,072 DEBUG RFM2Pi acknowledged command: <n> c - set collect mode (advanced, normally 0)
2021-02-01 14:52:47,299 DEBUG RFM2Pi acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:52:47,405 DEBUG RFM2Pi acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:52:47,510 DEBUG RFM2Pi acknowledged command: <n> q - set quiet mode (1 = don't report bad packets)
2021-02-01 14:52:47,615 DEBUG RFM2Pi acknowledged command: <n> x - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:52:47,925 DEBUG RFM2Pi acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f - FS20 command (868 MHz)
2021-02-01 14:52:48,030 DEBUG RFM2Pi acknowledged command: <addr>,<dev>,<on> k - KAKU command (433 MHz)
2021-02-01 14:52:48,235 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz q1
... 2nd restart after chaning config
2021-02-01 14:59:34,400 INFO MainThread Setting RFM2Pi quiet: 0 (0q)
2021-02-01 14:59:34,429 DEBUG RFM2Pi acknowledged command: > 0q
2021-02-01 14:59:34,531 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:34,937 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 0 249 179 159 154 188 13 94 70 96 193 85 227 147 251 132 226 67 84 196 103 (-89)
2021-02-01 14:59:35,402 INFO MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:59:35,440 DEBUG RFM2Pi acknowledged command: > 1p
2021-02-01 14:59:35,745 DEBUG RFM2Pi acknowledged command: <nn> i - set node ID (standard node ids are 1..30)
2021-02-01 14:59:35,849 DEBUG RFM2Pi acknowledged command: <n> b - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:59:35,953 DEBUG RFM2Pi acknowledged command: <nnnn> o - change frequency offset within the band (default 1600)
2021-02-01 14:59:36,161 DEBUG RFM2Pi acknowledged command: <nnn> g - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:59:36,264 DEBUG RFM2Pi acknowledged command: <n> c - set collect mode (advanced, normally 0)
2021-02-01 14:59:36,471 DEBUG RFM2Pi acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:59:36,577 DEBUG RFM2Pi acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:59:36,682 DEBUG RFM2Pi acknowledged command: <n> q - set quiet mode (1 = don't report bad packets)
2021-02-01 14:59:36,787 DEBUG RFM2Pi acknowledged command: <n> x - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:59:37,097 DEBUG RFM2Pi acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f - FS20 command (868 MHz)
2021-02-01 14:59:37,201 DEBUG RFM2Pi acknowledged command: <addr>,<dev>,<on> k - KAKU command (433 MHz)
2021-02-01 14:59:37,406 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:37,512 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 2 125 130 47 102 182 207 99 42 216 26 96 133 20 42 44 89 108 216 86 154 (-95)
2021-02-01 14:59:42,243 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 10 10 239 175 248 136 4 253 148 54 137 145 247 113 238 172 185 139 139 8 249 (-92)
2021-02-01 14:59:44,075 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 21 3 59 175 189 130 80 238 222 195 56 177 65 116 201 136 158 49 92 28 240 (-89)
2021-02-01 14:59:45,789 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 9 95 190 23 118 11 175 23 9 12 39 212 129 244 39 97 97 117 28 47 17 (-86)
2021-02-01 14:59:46,500 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 11 171 154 242 75 88 126 71 97 229 206 160 178 25 86 219 101 236 97 215 54 (-87)
2021-02-01 14:59:46,709 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 14 255 221 120 41 110 4 216 169 119 42 121 160 102 68 202 62 190 29 219 127 (-90)
2021-02-01 14:59:46,817 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 25 203 245 213 159 255 201 209 24 111 239 56 125 81 105 255 57 73 66 203 254 (-89)
2021-02-01 14:59:49,073 DEBUG MainThread Signal 15 received.
2021-02-01 14:59:49,191 INFO MainThread Exiting hub...
2021-02-01 14:59:49,353 INFO MainThread Exit completed
2021-02-01 14:59:50,750 INFO MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-02-01 14:59:50,751 INFO MainThread Opening hub...
2021-02-01 14:59:50,752 INFO MainThread Logging level set to DEBUG
2021-02-01 14:59:50,753 INFO MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-01 14:59:50,755 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-02-01 14:59:52,761 INFO MainThread RFM2Pi device firmware version: [RF12demo.13]
2021-02-01 14:59:52,762 INFO MainThread RFM2Pi device current settings: E i5 g210 @ 433 MHz
2021-02-01 14:59:52,764 INFO MainThread Setting RFM2Pi quiet: 0 (0q)
2021-02-01 14:59:53,767 INFO MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:59:54,770 DEBUG MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-01 14:59:54,772 DEBUG MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-01 14:59:54,781 INFO MainThread Creating EmonHubMqttInterfacer 'MQTT'
2021-02-01 14:59:54,784 DEBUG RFM2Pi acknowledged command: > 0q
2021-02-01 14:59:54,787 DEBUG MainThread Setting MQTT pubchannels: ['ToRFM12']
2021-02-01 14:59:54,789 DEBUG MainThread Setting MQTT subchannels: ['ToEmonCMS']
2021-02-01 14:59:54,790 INFO MainThread Setting MQTT node_format_enable: 1
2021-02-01 14:59:54,791 INFO MainThread Setting MQTT nodevar_format_enable: 1
2021-02-01 14:59:54,792 INFO MainThread Setting MQTT nodevar_format_basetopic: emon/
2021-02-01 14:59:54,795 INFO MainThread Setting MQTT node_JSON_enable: 0
2021-02-01 14:59:54,797 INFO MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2021-02-01 14:59:54,800 DEBUG MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
2021-02-01 14:59:54,802 DEBUG MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
2021-02-01 14:59:54,803 WARNING MainThread Setting emoncmsorg apikey: obscured
2021-02-01 14:59:54,804 INFO MainThread Setting emoncmsorg url: https://emoncms.org
2021-02-01 14:59:54,806 INFO MainThread Setting emoncmsorg senddata: 1
2021-02-01 14:59:54,807 INFO MainThread Setting emoncmsorg sendstatus: 1
2021-02-01 14:59:54,887 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:54,991 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 16 240 40 214 3 188 11 186 1 186 32 241 1 171 207 89 30 103 20 47 145 (-95)
2021-02-01 14:59:55,093 DEBUG RFM2Pi acknowledged command: > 1p
2021-02-01 14:59:55,398 DEBUG RFM2Pi acknowledged command: <nn> i - set node ID (standard node ids are 1..30)
2021-02-01 14:59:55,502 DEBUG RFM2Pi acknowledged command: <n> b - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:59:55,609 DEBUG RFM2Pi acknowledged command: <nnnn> o - change frequency offset within the band (default 1600)
2021-02-01 14:59:55,821 DEBUG RFM2Pi acknowledged command: <nnn> g - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:59:55,927 DEBUG RFM2Pi acknowledged command: <n> c - set collect mode (advanced, normally 0)
2021-02-01 14:59:56,138 DEBUG RFM2Pi acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:59:56,244 DEBUG RFM2Pi acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:59:56,350 DEBUG RFM2Pi acknowledged command: <n> q - set quiet mode (1 = don't report bad packets)
2021-02-01 14:59:56,458 DEBUG RFM2Pi acknowledged command: <n> x - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:59:56,773 DEBUG RFM2Pi acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f - FS20 command (868 MHz)
2021-02-01 14:59:56,907 DEBUG RFM2Pi acknowledged command: <addr>,<dev>,<on> k - KAKU command (433 MHz)
2021-02-01 14:59:57,112 DEBUG RFM2Pi device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:57,220 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 13 167 33 45 23 167 38 211 249 13 111 252 66 154 64 136 184 198 171 161 89 (-92)
2021-02-01 14:59:57,329 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 9 89 111 212 92 85 223 103 188 54 122 202 245 88 143 188 213 154 138 230 97 (-92)
2021-02-01 14:59:59,653 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 14 49 164 186 181 23 123 110 239 57 131 46 221 166 233 117 34 156 21 7 27 (-95)
2021-02-01 14:59:59,996 DEBUG RFM2Pi 7 NEW FRAME : OK 24 198 0 45 0 215 1 28 0 1 0 0 0 (-55)
2021-02-01 14:59:59,999 DEBUG RFM2Pi 7 Timestamp : 1612191599.995925
2021-02-01 15:00:00,000 DEBUG RFM2Pi 7 From Node : 24
2021-02-01 15:00:00,000 DEBUG RFM2Pi 7 Values : [19.8, 4.5, 47.1, 2.8000000000000003, 1]
2021-02-01 15:00:00,001 DEBUG RFM2Pi 7 RSSI : -55
2021-02-01 15:00:00,002 DEBUG RFM2Pi 7 Sent to channel(start)' : ToEmonCMS
2021-02-01 15:00:00,003 DEBUG RFM2Pi 7 Sent to channel(end)' : ToEmonCMS
2021-02-01 15:00:00,225 INFO MQTT Connecting to MQTT Server
2021-02-01 15:00:00,310 DEBUG RFM2Pi Discarding RX frame 'unreliable content': ? 19 23 84 78 128 109 156 75 122 79 199 124 239 175 159 103 200 27 19 193 29 (-96)
2021-02-01 15:00:00,328 DEBUG emoncmsorg Buffer size: 1
The RFM board does not sometimes recover until a complete power down (remove power completely) and restart.
What version of emonhub is this? There have been some changes to it.
Sorry Brian, what do you mean version? h/w, s/w??
Post can’t be empty
Just to help us get a better idea of what you have, emonhub is just the software, so when you say you have just added emonhub, do you mean you have just installed emonhub or do you mean you have added the rfm2pi board and installed emonhub? ie was the rfm2pi previously there, possbly with another SW not emonhub?
I do not think that is the case here as there are some “discarded packets” coming through and at least one good packet from node 24 so the rfm2pi is working.
All the discarded packets are very low signal, so is it possible the signal of the devices has changed, devices moved, aerials dislodged (tx or re ends), obstacles added (car, door etc) or interference introduced eg machinery or high current loads etc.
Or have you changed groups either at the rx or tx ends, the default is group 210.
The other common issue is the rfm2pi antenna shorting on the RPi’s gpio pins if it’s not seated correctly or the soldering/tail is too big on the underside of the rfm2pi.
Basically it looks like the rfm2pi is getting few decent data from the other devices for one reason or another. Can you try moving the devices closer together for a test?
Also, how many devices do you have? Plus did the node 24 continue after the end of that log? The emonTH only transmits every 60secs so I would only expect to see the one payload following the restart.
Are we expecting more nodes or are you looking to debug why node 24 previously stopped ? Because 24 rssi is -55, which is fine.
More information on my setup.
I originally purchased a prebuilt emonHub last year. Due to the local RF environment, this was co-sited with another Pi that was running Docker and Home Assistant with the ZWave and Zigbee USB controllers. In an attempt to reduce the clutter, I moved the RFM2Pi to the Docker Pi and installed the emonHub software from github (GitHub - openenergymonitor/emonhub: Python service linking and decoding input to MQTT & Emoncms). i created a systemd service file and it’s been running since 6/1.
emonHub is up to date with origin/master, so it’s the latest version.
There are five emonTH nodes and under normal circumstances all are received with signals in excess of -55db. I just truncated the log at the point I did because there were no errors after that.
Judging by the RSSI of the unreliable packets, I suspect that somebody else locally is running something on the same frequency.
Since I restarted yesterday afternoon, it hasn’t missed a packet as far as I can see from the logs in Home Assistant.
I wasn’t saying it was, but if something has stopped working and I have not changed anything, I will try a hard power off boot. Remarkable how often this cures things.
I wasn’t saying it was, but if something has stopped working and I have not changed anything, I will try a hard power off boot. Remarkable how often this cures things.
Agreed, and that’s how I cured the problem the first time, but this is not sustainable. I’m attempting to build a system that requires zero physical interaction - hence the UPS etc. I can easily build a recovery script that can be executed automatically if sensor data dries up. I can’t do this if a power-cycle is required.
This time, changing the script and restarting the service cured the problem. Maybe the emonHub code can be modified to simulate this behaviour every time the service is started. That way, all I need to do is restart the service to recover.
The RSSI levels you’re seeing is indicative of a relatively quiet RF environment.
i.e. the likelihood of any interference being present is quite low.
The numbers you’re getting indicate a signal level near the edge of reliable operation.
(the more negative the number is, the weaker the signal. e.g. -60 is weaker than -50)
Absolutely, the reason I commented that power cycling wasn’t the case here is purely because so many user do just that and it makes debugging pretty difficult. You have given us enough information for mr to be pretty damn sure that is it the rfm module on the rfm2pi board that is locking out, however not quite enough info to be absolutely sure.
By changing the setting in emonhub.conf (quiet = false) I strongly suspect you reinitialised the rfm module on the rfm2pi and that would probably had done the trick (to dubug, not permanently fix), But because you restarted emonhub you have introduced more than one “fix” at once and we cannot be sure of what the results indicate exactly.
When this occurs again just change a rf setting in emonhub, I would recommend changing the node id (baseid = 5 or 15 depending on your setup, 5 is an emonpi and 15 used to be the rfm2pi’s but the line has blurred in recent years). By changing node iid and nothing else. If the system comes back to life, you will have proved that the rfm module can be kick started with code, without reseting the rfm2pi, or restarting the emonhub service, or indeed power cycling or rebooting.
If changing the node id doesn’t result in one or more emonth’s poping up in the log within (say) 90secs or so then that one single test has struck out and the next test should be to use the reset_rfm2pi script in the linked thread, perhaps install it now so that you can just issue the command should you need to if the first test bears no fruit.
Whilst all this is being done you could be watching the logs by tailing the emonhub.log in another ssh window. When posting logs it would be useful if you can post logs that stretch beyond what you think we might need to see as often the lack of data is just as informative as a positive fault indicator, for example the log you last posted could have shown (say) 3mins of operation and that would allow us to see the working emonTH’s and the activity inbetween and not leave us wondering what fault we’re looking for. eg
2021-02-01 04:59:12,442 DEBUG MQTT Publishing: emonhub/rx/24/values 17.400000000000002,2.3000000000000003,45.5,2.8000000000000003,1,-59
2021-02-01 04:59:20,945 DEBUG emoncmsorg Buffer size: 3
... first service restart
2021-02-01 14:52:40,444 DEBUG MainThread Signal 15 received.
2021-02-01 14:52:40,522 INFO MainThread Exiting hub...
raises lots of questions, I do not believe the log would end on that “emoncmsorg Buffer size: 3” if we’re looking at an rfm2pi issue, but it is feasible that ending might be seen if the emonhub service had quietly locked up (service still running) or if the log partition had filled up, so if you could post longer, un edited logs please it would speed things up.
What you mean here is you bought a pre-built emonBase, I presume, there is no such thing as “an emonhub” it is a piece of software only!
You can if you choose, try restarting the emonhub service, perhaps before any other tests, I’m fairly confident it will have no effect, when you change settings there is no need to restart emonhub, it will pickup any change in settings and apply them, runtime settings get applied on the fly, init_settings cause the relevant interfacer to be deleted and reinitialised without restarting the service, emonhub is not designed to be stopped and started.
I would strongly recommend keeping quiet = false and loglevel = debug until we have some concrete info and can tune into the relevant symptoms of the issue.
Please do not do this, or at least not yet. This type of solution is exactly the reason you are suffering from this issue and it hasn’t been fixed despite us being very aware of the issue for several years. Users tend to “auto reset” or add a direct serial connection, learn to live with it, move to alternative equipment or the issue “just goes away” because they happen to change something unwittingly and not report back.
There are many things we can try, but it has to be in a logical order and step by step, trying loads of stuff at once will not pinpoint the issue.
I could give you at least 3 different ways to automate kick starting the data flow, but none would be a fix, there is even a FW version somewhere that regularly reinitialises the rfm every 60s, this is crude and again a major reason why this issue has not been fixed yet.
so to recap, install the reset_rfm2pi script
sudo wget -O /usr/bin/reset_rfm2pi http://openenergymonitor.org/forum-archive/sites/default/files/reset_rfm2pi.txt
sudo chmod +x /usr/bin/reset_rfm2pi
and then when it happens next
- Open a second ssh window and tail the log so you can see changes as they happen
assuming there is no rf traffic, including discarded packetssudo tail -f /var/log/emonhub/emonhub.log - Check the status and try restarting the emonhub service only, I expect no change
- Edit the baseid in emonhub.conf and save, I think this will kick start the data so wait upto 2mins to be sure and IF there is no rf traffic, then . .
- Use the
reset_rfm2picommand, if there is still no data . . - change the baseid back to see the change being applied in the logs and post all your results.
If No. 3 does the job, I would recommend trying another PSU, take the UPS out of the equation just for now. I can give you some more tests to do or FW to try should it fail again then.