emonTx data disappeared

Guff666 · 1 February 2021 15:10

I have some data to add to the thread. I’ve now had this happen to my Pi three times, but the last time I was able to intervene.

This is not a standard emonHub. I have the RFM2Pi board installed on a Pi3B that is running some Docker containers. emonHub is installed as a systemd service. The Pi has a LiFePO4wered Pi UPS connected. The Pi was running happily for nearly a year before I added the emonHub to it.

The symptoms are that no mqtt data flowed after 04:59 this morning. When I realised and checked:

The emonhub service was running
The log file shows nothing after 04:59

I restarted the service:

The log showed the service configuring the RFM2Pi board
but no telemetry was received

I then changed a setting in emonhub.conf (I changed quiet = true in the RFM2Pi config). Once I restarted the service, telemetry started to flow again.

Here’s the log from just before it stops to now:

2021-02-01 04:59:12,323 DEBUG    RFM2Pi     55621 NEW FRAME : OK 24 174 0 23 0 199 1 28 0 1 0 0 0 (-59)
2021-02-01 04:59:12,325 DEBUG    RFM2Pi     55621 Timestamp : 1612155552.323061
2021-02-01 04:59:12,326 DEBUG    RFM2Pi     55621 From Node : 24
2021-02-01 04:59:12,328 DEBUG    RFM2Pi     55621    Values : [17.400000000000002, 2.3000000000000003, 45.5, 2.8000000000000003, 1]
2021-02-01 04:59:12,329 DEBUG    RFM2Pi     55621      RSSI : -59
2021-02-01 04:59:12,330 DEBUG    RFM2Pi     55621 Sent to channel(start)' : ToEmonCMS
2021-02-01 04:59:12,331 DEBUG    RFM2Pi     55621 Sent to channel(end)' : ToEmonCMS
2021-02-01 04:59:12,428 DEBUG    MQTT       Publishing: emon/emonth6/temperature 17.400000000000002
2021-02-01 04:59:12,430 DEBUG    MQTT       Publishing: emon/emonth6/external temperature 2.3000000000000003
2021-02-01 04:59:12,434 DEBUG    MQTT       Publishing: emon/emonth6/humidity 45.5
2021-02-01 04:59:12,436 DEBUG    MQTT       Publishing: emon/emonth6/battery 2.8000000000000003
2021-02-01 04:59:12,438 DEBUG    MQTT       Publishing: emon/emonth6/pulsecount 1
2021-02-01 04:59:12,440 DEBUG    MQTT       Publishing: emon/emonth6/rssi -59
2021-02-01 04:59:12,441 INFO     MQTT       Publishing 'node' formatted msg
2021-02-01 04:59:12,442 DEBUG    MQTT       Publishing: emonhub/rx/24/values 17.400000000000002,2.3000000000000003,45.5,2.8000000000000003,1,-59
2021-02-01 04:59:20,945 DEBUG    emoncmsorg Buffer size: 3

... first service restart

2021-02-01 14:52:40,444 DEBUG    MainThread Signal 15 received.
2021-02-01 14:52:40,522 INFO     MainThread Exiting hub...
2021-02-01 14:52:40,748 INFO     MainThread Exit completed
2021-02-01 14:52:43,221 INFO     MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-02-01 14:52:43,222 INFO     MainThread Opening hub...
2021-02-01 14:52:43,223 INFO     MainThread Logging level set to DEBUG
2021-02-01 14:52:43,223 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-01 14:52:43,225 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-02-01 14:52:45,232 INFO     MainThread RFM2Pi device firmware version: [RF12demo.13]
2021-02-01 14:52:45,234 INFO     MainThread RFM2Pi device current settings:  E i5 g210 @ 433 MHz q1
2021-02-01 14:52:45,235 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:52:46,238 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-01 14:52:46,239 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-01 14:52:46,241 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2021-02-01 14:52:46,243 DEBUG    RFM2Pi     acknowledged command: > 1p
2021-02-01 14:52:46,245 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
2021-02-01 14:52:46,247 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
2021-02-01 14:52:46,248 INFO     MainThread Setting MQTT node_format_enable: 1
2021-02-01 14:52:46,248 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2021-02-01 14:52:46,249 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2021-02-01 14:52:46,249 INFO     MainThread Setting MQTT node_JSON_enable: 0
2021-02-01 14:52:46,251 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2021-02-01 14:52:46,252 DEBUG    MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
2021-02-01 14:52:46,253 DEBUG    MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
2021-02-01 14:52:46,255 WARNING  MainThread Setting emoncmsorg apikey: obscured
2021-02-01 14:52:46,255 INFO     MainThread Setting emoncmsorg url: https://emoncms.org
2021-02-01 14:52:46,256 INFO     MainThread Setting emoncmsorg senddata: 1
2021-02-01 14:52:46,257 INFO     MainThread Setting emoncmsorg sendstatus: 1
2021-02-01 14:52:46,551 DEBUG    RFM2Pi     acknowledged command: <nn> i     - set node ID (standard node ids are 1..30)
2021-02-01 14:52:46,655 DEBUG    RFM2Pi     acknowledged command: <n> b      - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:52:46,759 DEBUG    RFM2Pi     acknowledged command: <nnnn> o   - change frequency offset within the band (default 1600)
2021-02-01 14:52:46,967 DEBUG    RFM2Pi     acknowledged command: <nnn> g    - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:52:47,072 DEBUG    RFM2Pi     acknowledged command: <n> c      - set collect mode (advanced, normally 0)
2021-02-01 14:52:47,299 DEBUG    RFM2Pi     acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:52:47,405 DEBUG    RFM2Pi     acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:52:47,510 DEBUG    RFM2Pi     acknowledged command: <n> q      - set quiet mode (1 = don't report bad packets)
2021-02-01 14:52:47,615 DEBUG    RFM2Pi     acknowledged command: <n> x      - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:52:47,925 DEBUG    RFM2Pi     acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f     - FS20 command (868 MHz)
2021-02-01 14:52:48,030 DEBUG    RFM2Pi     acknowledged command: <addr>,<dev>,<on> k              - KAKU command (433 MHz)
2021-02-01 14:52:48,235 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1

... 2nd restart after chaning config


2021-02-01 14:59:34,400 INFO     MainThread Setting RFM2Pi quiet: 0 (0q)
2021-02-01 14:59:34,429 DEBUG    RFM2Pi     acknowledged command: > 0q
2021-02-01 14:59:34,531 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:34,937 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 0 249 179 159 154 188 13 94 70 96 193 85 227 147 251 132 226 67 84 196 103 (-89)
2021-02-01 14:59:35,402 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:59:35,440 DEBUG    RFM2Pi     acknowledged command: > 1p
2021-02-01 14:59:35,745 DEBUG    RFM2Pi     acknowledged command: <nn> i     - set node ID (standard node ids are 1..30)
2021-02-01 14:59:35,849 DEBUG    RFM2Pi     acknowledged command: <n> b      - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:59:35,953 DEBUG    RFM2Pi     acknowledged command: <nnnn> o   - change frequency offset within the band (default 1600)
2021-02-01 14:59:36,161 DEBUG    RFM2Pi     acknowledged command: <nnn> g    - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:59:36,264 DEBUG    RFM2Pi     acknowledged command: <n> c      - set collect mode (advanced, normally 0)
2021-02-01 14:59:36,471 DEBUG    RFM2Pi     acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:59:36,577 DEBUG    RFM2Pi     acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:59:36,682 DEBUG    RFM2Pi     acknowledged command: <n> q      - set quiet mode (1 = don't report bad packets)
2021-02-01 14:59:36,787 DEBUG    RFM2Pi     acknowledged command: <n> x      - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:59:37,097 DEBUG    RFM2Pi     acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f     - FS20 command (868 MHz)
2021-02-01 14:59:37,201 DEBUG    RFM2Pi     acknowledged command: <addr>,<dev>,<on> k              - KAKU command (433 MHz)
2021-02-01 14:59:37,406 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:37,512 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 2 125 130 47 102 182 207 99 42 216 26 96 133 20 42 44 89 108 216 86 154 (-95)
2021-02-01 14:59:42,243 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 10 10 239 175 248 136 4 253 148 54 137 145 247 113 238 172 185 139 139 8 249 (-92)
2021-02-01 14:59:44,075 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 21 3 59 175 189 130 80 238 222 195 56 177 65 116 201 136 158 49 92 28 240 (-89)
2021-02-01 14:59:45,789 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 9 95 190 23 118 11 175 23 9 12 39 212 129 244 39 97 97 117 28 47 17 (-86)
2021-02-01 14:59:46,500 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 11 171 154 242 75 88 126 71 97 229 206 160 178 25 86 219 101 236 97 215 54 (-87)
2021-02-01 14:59:46,709 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 14 255 221 120 41 110 4 216 169 119 42 121 160 102 68 202 62 190 29 219 127 (-90)
2021-02-01 14:59:46,817 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 25 203 245 213 159 255 201 209 24 111 239 56 125 81 105 255 57 73 66 203 254 (-89)
2021-02-01 14:59:49,073 DEBUG    MainThread Signal 15 received.
2021-02-01 14:59:49,191 INFO     MainThread Exiting hub...
2021-02-01 14:59:49,353 INFO     MainThread Exit completed
2021-02-01 14:59:50,750 INFO     MainThread EmonHub emonHub (emon-pi variant) v2.1.5
2021-02-01 14:59:50,751 INFO     MainThread Opening hub...
2021-02-01 14:59:50,752 INFO     MainThread Logging level set to DEBUG
2021-02-01 14:59:50,753 INFO     MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2021-02-01 14:59:50,755 DEBUG    MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2021-02-01 14:59:52,761 INFO     MainThread RFM2Pi device firmware version: [RF12demo.13]
2021-02-01 14:59:52,762 INFO     MainThread RFM2Pi device current settings:  E i5 g210 @ 433 MHz
2021-02-01 14:59:52,764 INFO     MainThread Setting RFM2Pi quiet: 0 (0q)
2021-02-01 14:59:53,767 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2021-02-01 14:59:54,770 DEBUG    MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2021-02-01 14:59:54,772 DEBUG    MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2021-02-01 14:59:54,781 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT'
2021-02-01 14:59:54,784 DEBUG    RFM2Pi     acknowledged command: > 0q
2021-02-01 14:59:54,787 DEBUG    MainThread Setting MQTT pubchannels: ['ToRFM12']
2021-02-01 14:59:54,789 DEBUG    MainThread Setting MQTT subchannels: ['ToEmonCMS']
2021-02-01 14:59:54,790 INFO     MainThread Setting MQTT node_format_enable: 1
2021-02-01 14:59:54,791 INFO     MainThread Setting MQTT nodevar_format_enable: 1
2021-02-01 14:59:54,792 INFO     MainThread Setting MQTT nodevar_format_basetopic: emon/
2021-02-01 14:59:54,795 INFO     MainThread Setting MQTT node_JSON_enable: 0
2021-02-01 14:59:54,797 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2021-02-01 14:59:54,800 DEBUG    MainThread Setting emoncmsorg pubchannels: ['ToRFM12']
2021-02-01 14:59:54,802 DEBUG    MainThread Setting emoncmsorg subchannels: ['ToEmonCMS']
2021-02-01 14:59:54,803 WARNING  MainThread Setting emoncmsorg apikey: obscured
2021-02-01 14:59:54,804 INFO     MainThread Setting emoncmsorg url: https://emoncms.org
2021-02-01 14:59:54,806 INFO     MainThread Setting emoncmsorg senddata: 1
2021-02-01 14:59:54,807 INFO     MainThread Setting emoncmsorg sendstatus: 1
2021-02-01 14:59:54,887 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:54,991 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 16 240 40 214 3 188 11 186 1 186 32 241 1 171 207 89 30 103 20 47 145 (-95)
2021-02-01 14:59:55,093 DEBUG    RFM2Pi     acknowledged command: > 1p
2021-02-01 14:59:55,398 DEBUG    RFM2Pi     acknowledged command: <nn> i     - set node ID (standard node ids are 1..30)
2021-02-01 14:59:55,502 DEBUG    RFM2Pi     acknowledged command: <n> b      - set MHz band (4 = 433, 8 = 868, 9 = 915)
2021-02-01 14:59:55,609 DEBUG    RFM2Pi     acknowledged command: <nnnn> o   - change frequency offset within the band (default 1600)
2021-02-01 14:59:55,821 DEBUG    RFM2Pi     acknowledged command: <nnn> g    - set network group (RFM12 only allows 212, 0 = any)
2021-02-01 14:59:55,927 DEBUG    RFM2Pi     acknowledged command: <n> c      - set collect mode (advanced, normally 0)
2021-02-01 14:59:56,138 DEBUG    RFM2Pi     acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2021-02-01 14:59:56,244 DEBUG    RFM2Pi     acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2021-02-01 14:59:56,350 DEBUG    RFM2Pi     acknowledged command: <n> q      - set quiet mode (1 = don't report bad packets)
2021-02-01 14:59:56,458 DEBUG    RFM2Pi     acknowledged command: <n> x      - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2021-02-01 14:59:56,773 DEBUG    RFM2Pi     acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f     - FS20 command (868 MHz)
2021-02-01 14:59:56,907 DEBUG    RFM2Pi     acknowledged command: <addr>,<dev>,<on> k              - KAKU command (433 MHz)
2021-02-01 14:59:57,112 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz
2021-02-01 14:59:57,220 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 13 167 33 45 23 167 38 211 249 13 111 252 66 154 64 136 184 198 171 161 89 (-92)
2021-02-01 14:59:57,329 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 9 89 111 212 92 85 223 103 188 54 122 202 245 88 143 188 213 154 138 230 97 (-92)
2021-02-01 14:59:59,653 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 14 49 164 186 181 23 123 110 239 57 131 46 221 166 233 117 34 156 21 7 27 (-95)
2021-02-01 14:59:59,996 DEBUG    RFM2Pi     7 NEW FRAME : OK 24 198 0 45 0 215 1 28 0 1 0 0 0 (-55)
2021-02-01 14:59:59,999 DEBUG    RFM2Pi     7 Timestamp : 1612191599.995925
2021-02-01 15:00:00,000 DEBUG    RFM2Pi     7 From Node : 24
2021-02-01 15:00:00,000 DEBUG    RFM2Pi     7    Values : [19.8, 4.5, 47.1, 2.8000000000000003, 1]
2021-02-01 15:00:00,001 DEBUG    RFM2Pi     7      RSSI : -55
2021-02-01 15:00:00,002 DEBUG    RFM2Pi     7 Sent to channel(start)' : ToEmonCMS
2021-02-01 15:00:00,003 DEBUG    RFM2Pi     7 Sent to channel(end)' : ToEmonCMS
2021-02-01 15:00:00,225 INFO     MQTT       Connecting to MQTT Server
2021-02-01 15:00:00,310 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 19 23 84 78 128 109 156 75 122 79 199 124 239 175 159 103 200 27 19 193 29 (-96)
2021-02-01 15:00:00,328 DEBUG    emoncmsorg Buffer size: 1

borpin · 1 February 2021 19:07

The RFM board does not sometimes recover until a complete power down (remove power completely) and restart.

What version of emonhub is this? There have been some changes to it.

Guff666 · 2 February 2021 10:08

Sorry Brian, what do you mean version? h/w, s/w??

djh · 2 February 2021 11:00

Post can’t be empty

pb66 · 2 February 2021 11:12

Just to help us get a better idea of what you have, emonhub is just the software, so when you say you have just added emonhub, do you mean you have just installed emonhub or do you mean you have added the rfm2pi board and installed emonhub? ie was the rfm2pi previously there, possbly with another SW not emonhub?

I do not think that is the case here as there are some “discarded packets” coming through and at least one good packet from node 24 so the rfm2pi is working.

All the discarded packets are very low signal, so is it possible the signal of the devices has changed, devices moved, aerials dislodged (tx or re ends), obstacles added (car, door etc) or interference introduced eg machinery or high current loads etc.

Or have you changed groups either at the rx or tx ends, the default is group 210.

The other common issue is the rfm2pi antenna shorting on the RPi’s gpio pins if it’s not seated correctly or the soldering/tail is too big on the underside of the rfm2pi.

Basically it looks like the rfm2pi is getting few decent data from the other devices for one reason or another. Can you try moving the devices closer together for a test?

Also, how many devices do you have? Plus did the node 24 continue after the end of that log? The emonTH only transmits every 60secs so I would only expect to see the one payload following the restart.

Are we expecting more nodes or are you looking to debug why node 24 previously stopped ? Because 24 rssi is -55, which is fine.

Guff666 · 2 February 2021 15:43

More information on my setup.

I originally purchased a prebuilt emonHub last year. Due to the local RF environment, this was co-sited with another Pi that was running Docker and Home Assistant with the ZWave and Zigbee USB controllers. In an attempt to reduce the clutter, I moved the RFM2Pi to the Docker Pi and installed the emonHub software from github (GitHub - openenergymonitor/emonhub: Python service linking and decoding input to MQTT & Emoncms). i created a systemd service file and it’s been running since 6/1.

emonHub is up to date with origin/master, so it’s the latest version.

There are five emonTH nodes and under normal circumstances all are received with signals in excess of -55db. I just truncated the log at the point I did because there were no errors after that.

Judging by the RSSI of the unreliable packets, I suspect that somebody else locally is running something on the same frequency.

Since I restarted yesterday afternoon, it hasn’t missed a packet as far as I can see from the logs in Home Assistant.

borpin · 2 February 2021 16:32

I wasn’t saying it was, but if something has stopped working and I have not changed anything, I will try a hard power off boot. Remarkable how often this cures things.

Guff666 · 2 February 2021 19:55

I wasn’t saying it was, but if something has stopped working and I have not changed anything, I will try a hard power off boot. Remarkable how often this cures things.

Agreed, and that’s how I cured the problem the first time, but this is not sustainable. I’m attempting to build a system that requires zero physical interaction - hence the UPS etc. I can easily build a recovery script that can be executed automatically if sensor data dries up. I can’t do this if a power-cycle is required.

This time, changing the script and restarting the service cured the problem. Maybe the emonHub code can be modified to simulate this behaviour every time the service is started. That way, all I need to do is restart the service to recover.

Bill.Thomson · 2 February 2021 20:27

The RSSI levels you’re seeing is indicative of a relatively quiet RF environment.
i.e. the likelihood of any interference being present is quite low.

The numbers you’re getting indicate a signal level near the edge of reliable operation.
(the more negative the number is, the weaker the signal. e.g. -60 is weaker than -50)

pb66 · 2 February 2021 21:36

Absolutely, the reason I commented that power cycling wasn’t the case here is purely because so many user do just that and it makes debugging pretty difficult. You have given us enough information for mr to be pretty damn sure that is it the rfm module on the rfm2pi board that is locking out, however not quite enough info to be absolutely sure.

By changing the setting in emonhub.conf (quiet = false) I strongly suspect you reinitialised the rfm module on the rfm2pi and that would probably had done the trick (to dubug, not permanently fix), But because you restarted emonhub you have introduced more than one “fix” at once and we cannot be sure of what the results indicate exactly.

When this occurs again just change a rf setting in emonhub, I would recommend changing the node id (baseid = 5 or 15 depending on your setup, 5 is an emonpi and 15 used to be the rfm2pi’s but the line has blurred in recent years). By changing node iid and nothing else. If the system comes back to life, you will have proved that the rfm module can be kick started with code, without reseting the rfm2pi, or restarting the emonhub service, or indeed power cycling or rebooting.

If changing the node id doesn’t result in one or more emonth’s poping up in the log within (say) 90secs or so then that one single test has struck out and the next test should be to use the reset_rfm2pi script in the linked thread, perhaps install it now so that you can just issue the command should you need to if the first test bears no fruit.

Whilst all this is being done you could be watching the logs by tailing the emonhub.log in another ssh window. When posting logs it would be useful if you can post logs that stretch beyond what you think we might need to see as often the lack of data is just as informative as a positive fault indicator, for example the log you last posted could have shown (say) 3mins of operation and that would allow us to see the working emonTH’s and the activity inbetween and not leave us wondering what fault we’re looking for. eg

2021-02-01 04:59:12,442 DEBUG    MQTT       Publishing: emonhub/rx/24/values 17.400000000000002,2.3000000000000003,45.5,2.8000000000000003,1,-59
2021-02-01 04:59:20,945 DEBUG    emoncmsorg Buffer size: 3

... first service restart

2021-02-01 14:52:40,444 DEBUG    MainThread Signal 15 received.
2021-02-01 14:52:40,522 INFO     MainThread Exiting hub...

raises lots of questions, I do not believe the log would end on that “emoncmsorg Buffer size: 3” if we’re looking at an rfm2pi issue, but it is feasible that ending might be seen if the emonhub service had quietly locked up (service still running) or if the log partition had filled up, so if you could post longer, un edited logs please it would speed things up.

What you mean here is you bought a pre-built emonBase, I presume, there is no such thing as “an emonhub” it is a piece of software only!

You can if you choose, try restarting the emonhub service, perhaps before any other tests, I’m fairly confident it will have no effect, when you change settings there is no need to restart emonhub, it will pickup any change in settings and apply them, runtime settings get applied on the fly, init_settings cause the relevant interfacer to be deleted and reinitialised without restarting the service, emonhub is not designed to be stopped and started.

I would strongly recommend keeping quiet = false and loglevel = debug until we have some concrete info and can tune into the relevant symptoms of the issue.

Please do not do this, or at least not yet. This type of solution is exactly the reason you are suffering from this issue and it hasn’t been fixed despite us being very aware of the issue for several years. Users tend to “auto reset” or add a direct serial connection, learn to live with it, move to alternative equipment or the issue “just goes away” because they happen to change something unwittingly and not report back.

There are many things we can try, but it has to be in a logical order and step by step, trying loads of stuff at once will not pinpoint the issue.

I could give you at least 3 different ways to automate kick starting the data flow, but none would be a fix, there is even a FW version somewhere that regularly reinitialises the rfm every 60s, this is crude and again a major reason why this issue has not been fixed yet.

so to recap, install the reset_rfm2pi script

sudo wget -O /usr/bin/reset_rfm2pi http://openenergymonitor.org/forum-archive/sites/default/files/reset_rfm2pi.txt
sudo chmod +x /usr/bin/reset_rfm2pi

and then when it happens next

Open a second ssh window and tail the log so you can see changes as they happen
```
sudo tail -f /var/log/emonhub/emonhub.log
```
assuming there is no rf traffic, including discarded packets
Check the status and try restarting the emonhub service only, I expect no change
Edit the baseid in emonhub.conf and save, I think this will kick start the data so wait upto 2mins to be sure and IF there is no rf traffic, then . .
Use the reset_rfm2pi command, if there is still no data . .
change the baseid back to see the change being applied in the logs and post all your results.

If No. 3 does the job, I would recommend trying another PSU, take the UPS out of the equation just for now. I can give you some more tests to do or FW to try should it fail again then.

Guff666 · 3 February 2021 08:53

All copied and understood, Paul.

incidentally, regarding the log I sent. There was literally nothing in there between 04:59 and when I restarted the service.

I have journal retention enabled on the pi, so if you want more log info, let me know and I’ll dig it out.

Guff666 · 27 December 2021 10:58

Hi all, Happy Christmas.
So, after many months of flawless operation, the problem recurred last night. Of course it did so when we are away

Luckily, I got an alert from Home Assistant that all was not well and I have SSH access back into the house, but I only have termios on my iPad to play with.

My steps were to:

Confirm that the emonhub logs stop at 02:16 this morning.
Restart emonhub service and wait - no change
edit emonhub.conf (increase logging to DEBUG) - saw some log entries from RFM12 but still no telemetry.
restart emonhub service again - no change
ran reset_rfm12 script - BINGO!

I can’t easily copy/paste log files from here but I’ll be home tomorrow so I can (hopefully) update this post with additional info.