EmonTX v3 taking a walk

I have an EmonTx that I’ve had for over a year, along with some EmonTH sensors and a ‘homebrew’ EmonHUB that is streaming the data from the TX & TH units to my EmonCMS server (I saw that Wifi project for the TX and that’s really neat - would be great if that would work for all modules and eliminate the need for the EmonHUB, but not sure how it would work on battery power). After I gave up on the tinkering with it to re-arrange the DS18B20 sensors and I just ‘went with the flow’, it was running solid for quite some time - probably a year or so - but then a few months ago, it started going on walkabouts. It would just stop sending in data, requiring me to go power cycle it. No other nodes stopped sending data, including an EmonTH that is powered off the same four port USB power supply.

Any thoughts as to why it worked solid for so long and then just started taking walks?

Hi Dave, by “emonHUB” I assume you mean “emonbase” by the context. An emonbase is a physical device (eg a Pi with a RFM2Pi attached or a emonPi etc) and emonHub is a software that can run on an emonbase to funnel data from multiple sources to multiple targets.

So can you expand on what a “homebrew” emonbase is? Do you mean it’s not a Pi + RFM2Pi? or do you mean it is running a “homebrew” image?

The emonTx or the emonBase?

When the emonTx goes offline does the onboard led still flash or does it stop (assuming it normally does flash)?

There are many reasons it could go offline and there isn’t enough info here to even speculate what it probably could be. but it could possibly be interference, a deterioration in signal or clashing of transmission packets among other things, Can you provide some emonhub.logs from when the emonTx is working and when it isn’t?

And tell us more about how many devices there are, how frequently they are updating, the usual RSSI’s (if using rfm69cw based tranceiver).

Can you also tell us where you can see or confirm the emontx is offline, eg are the inputs updating on the inputs page of emoncms or is it just the feeds page that stops updating, or are you just seeing gaps in your visualizations?

Paul, yes, you are correct, I mean ‘emonbase’ (by ‘homebrew’, I mean I didn’t get the whole unit from OEM, just the RFM2Pi).

The EmonTX is what needs to be power cycled. When it stops updating in EmonCMS (I first notice it in the charts), the red light does not flash and on the ‘Feeds’ page of EmonCMS it shows no contact in ‘x’ seconds/minutes. As soon as I power cycle the EmonTX, I start getting data again.

I currently have four OEM devices - the TX and three EmonTH units - one 8" away mounted to the same board as the TX, being powered off the same USB ‘power strip’.

The EmonTX I think is updating every 10 or 20 seconds (I don’t recall what is set into it and it’s somewhat hard to tell the update interval from the feeds page because it doesn’t update ‘every 10 seconds or 20 seconds’. The TH that is next to it is probably also at a minute or less - maybe 30 sec, and the battery powered TH are I believe 5 minutes.

I can grab more info later if needed, but as the TX isn’t transmitting based on the lack of the LED flashing, I don’t think there will be anything in the logs to help.

No, I think you are right there. How long have you had the emonTx? Do you recall which sketch/firmware is loaded?

Do you have an AC:AC adapter attached? and if so do you recall if you removed the internal link so that the emonTx is not being powered by both the 9vac and the 5vdc?

Do you have a usb programmer to connect to the emonTx to see what version FW it’s running and possibly to install an updated version?

Do you have an alternative 5vdc power supply to try?

The symptoms you describe (led stops flashing and unit resets with a power cycle) are not a common issue, mostly when emonTx’s drop out it’s an rf issue or at the receiving end. They are usually pretty robust and reliable. What else do you have attached to it? or do you just have the 4 ct’s?

If the emonTx itself isn’t flashing its LED and power cycling restarts it, then are you physically disturbing the emonTx at all when you do that? If there’s a possibility that you are, then I’d suspect (in order) a failed power supply connection - either the 5 V USB or the ac adapter, a dry joint on the RFM radio module or a dry joint elsewhere.
I’ve cannot ever remember seeing a software fault causing an emonTx to stop transmitting - except when the RFM module fails to answer because of a soldering fault.
Is the 5 V common to all those units, or is each one over-load protected and independent, meaning that has the USB supply shut down, and removing the load resets it?

This is my setup. The only thing that would be disturbed would be the power plug/cable. Next time it takes a walk, I could try power cycling the switch on the power supply so as to not move any part of the EmonTX or it’s power cable. But after a year or so of solid reporting with no movement to the setup, I’m not sure that it is something being disturbed… We’ll see

At this point, I have only the four CTs connected to the TX.

I’m not sure what the internal architecture of the USB power supply is, but if I had to guess, I would say it’s probably a common bus across all ports.

I haven’t hooked up to it to see what firmware version it’s reporting, but I’m 95% sure I’m using ‘emonTxV3_4_DiscreteSampling’ v1.8.

Hi Dave,
You are not alone! (I just reset ‘em when I need to)
I have 4 EmonTx’s (all in a row), sharing an AC-AC power supply. One of them does this intermittently (LED & signal stop) - at one point it was happening roughly once a week, but then not in the last few months (where’s the wood to knock). And another one did it (I think) twice in the last year or two. Sometimes I’ll just push the reset button on the hung Tx - which could indeed disturb the physical connections, other times I’ll flip the switch on the socket (which resets all of them) - but there wouldn’t be any physical effect on the Tx here. Both actions have always fixed things - and I haven’t noticed any correlation between how I reset it & how long it lasts.
Mine are connecting to a standard EmonHub (Pi 2 + RFM + OEM SD image) about 30’ away (and a few EmonTH’s connecting too).
I was wondering about temperature - mine are in the garage, which partially follows outdoor conditions. Utah, US summers can get quite warm (~100 F), and the daily range is fairly large (~60 F - ~100 F). But it would need to be that in combination with maybe a slightly dry solder junction (since it’s only 1 Tx in 4 that has a significant issue).
If it starts to get more frequent again (maybe next summer), I was thinking I’d try the problem child on batteries.

Thanks,
Sandy

Hmm. A year or so is moving into dry joint territory. That’s getting towards long enough for a bit of tarnish or corrosion to have formed and spoil what used to be a reliable (but unsatisfactory) connection.

Just a quick update, so far, it hasn’t taken a walk since I posted about it happening… We’ll see when it does it next and see what can be found there. Is there any chance connecting up via serial before trying to reset it might provide anything of use?

That’s not likely to help. If you have a.c. power, the LED should flash each time the emonTx transmits, and that’s an indication that it’s running through the main loop. If it hangs, it won’t send anything via the serial port anyway. Connecting to serial would only help with no a.c power connected. Then, the LED does not flash in order to economise on battery life, but the serial output will only tell you much the same thing but using the measured values. (I.e., the fact that it’s looping, or stuck. There are no other diagnostics.)

Well, a quick update to this - After running without issue since October, it finally took a walk again about a week ago. About the only thing I can think of is maybe the warmer temps are doing something - It had gotten up to about 86 in the garage that day, but at the time it stopped reporting, it was about 73.

It does indeed sound temperature-related, but at only 23 °C, I would not expect a problem.

Here are a few thoughts:

The next time, can you try measuring the 3.3 V rail, if it’s accessible?

If it fails regularly, or you can reliably provoke a failure, can you run it without the radio module (in software - don’t try to unsolder it, just comment out the one line send_rf_data();) and see if it still fails? (OK, I know you’ll be without output - but does the LED still flash?)
Monitoring the serial output while you do this might actually confuse the test, since it will run on 5 V from the programmer as well as from the USB.

Is it a power supply problem - how are you resetting it? If you’re pressing the reset button, it might not be, but if you’re pulling the USB connector out, that would reset the internal regulator i.c.
Have you removed the jumper link (JP2) to force it to use only the USB power? That power regulator should shut down on over-current or over-temperature, then reset itself after it has cooled. Can you confirm it doesn’t come back after a break?

Does it still fail if you use a different port on your USB hub?

Could it be a brief brown-out on your 5 V USB power supply - sufficiently low to foul up the RFM69 but not low / long enough to cause a full and clean restart?
Although the emonTH and emonTx are similar beasts, that the TH doesn’t fail is not cast iron evidence that the TX won’t given the same treatment.

If nothing above points us towards a fault, then as you’re using mains power, it would be possible to modify the sketch to give it a watchdog timer, and that would cause a software reset of the processor (but not of the RFM69CW).

Unfortunately, there is nothing that I’m aware of that can make it stop working. it just stops whenever it feels like it. And that could be days, weeks or months.

This time I reset it by using its reset button and it came back to life the same as if I had pulled the plug out of the power supply. FWIW, I also have a Raspberry Pi running off the same USB Power supply and it hasn’t had any issues.
Where did you suggest to try to read the 3.3v? from something externally accessible? If so, I could try that next time it goes, or if it’s not accessible outside, I’d probably have to remove the board from the case and fasten it to the wall to be able to access it. And yes, it’s set to only use the USB source for power.

If you want to be prepared to measure the 3.3v rail in the event of another fail then you could attach a couple of wires to the 6way screw terms, yes it still involves taking the lid off, but it means you can put it back in the can once the wires are attached and then hook up the meter to the wires if/when it fails without risking disturbing the emonTx and it’s connections.

Can you provide some emonhub.log, a little snippet now whilst it is working and then some when it next fails (before you try anything else or reset it). Looking at the RFM2Pi sketch it looks as if the LED only flashes when a packet passes crc, so if something is corrupting the packet, it would fail CRC checks, not flash the led and appear like the emonTx isn’t sending when it may well be doing so.

The snippet now might give us some insight into the “usual” payload and maybe tell us a bit about it’s firmware version too.

The 3.3 V should also be accessible on the Temperature/Pulse RJ45 pins 2 (3.3 V) & 5 (GND) if it is a V3.4 (and I think yours is?)

Here is a bit from the emonhub taken today. Next time it takes a walk I’ll post up another.

2017-06-02 09:17:03,931 DEBUG    RFM2Pi     370919 NEW FRAME : OK 10 127 11 4 12 25 4 205 3 26 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-59)
2017-06-02 09:17:03,935 DEBUG    RFM2Pi     370919 Timestamp : 1496413023.93
2017-06-02 09:17:03,936 DEBUG    RFM2Pi     370919 From Node : 10
2017-06-02 09:17:03,937 DEBUG    RFM2Pi     370919    Values : [2943, 3076, 1049, 973, 120.58, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:03,938 DEBUG    RFM2Pi     370919      RSSI : -59
2017-06-02 09:17:03,938 DEBUG    RFM2Pi     370919 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:03,939 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 2943,3076,1049,973,120.58,0,0,0,0,0,0,1
2017-06-02 09:17:03,941 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -59
2017-06-02 09:17:03,944 DEBUG    RFM2Pi     370919 adding frame to buffer => [1496413023.931317, 10, 2943, 3076, 1049, 973, 120.58, 0, 0, 0, 0, 0, 0, 1, -59]
2017-06-02 09:17:03,945 DEBUG    RFM2Pi     370919 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:04,775 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 191 122 207 250 239 61 57 247 219 55 255 193 213 222 169 104 95 131 235 30 255 (-96)
2017-06-02 09:17:06,223 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 21 0 0 28 2 206 0 0 245 0 156 (-62)
2017-06-02 09:17:20,793 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 185 210 191 219 166 247 85 83 170 243 231 217 245 115 251 63 214 172 219 199 96 (-89)
2017-06-02 09:17:22,413 DEBUG    RFM2Pi     370920 NEW FRAME : OK 10 160 7 217 7 22 4 174 3 24 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-55)
2017-06-02 09:17:22,417 DEBUG    RFM2Pi     370920 Timestamp : 1496413042.41
2017-06-02 09:17:22,417 DEBUG    RFM2Pi     370920 From Node : 10
2017-06-02 09:17:22,418 DEBUG    RFM2Pi     370920    Values : [1952, 2009, 1046, 942, 120.56, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:22,419 DEBUG    RFM2Pi     370920      RSSI : -55
2017-06-02 09:17:22,420 DEBUG    RFM2Pi     370920 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:22,421 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 1952,2009,1046,942,120.56,0,0,0,0,0,0,1
2017-06-02 09:17:22,423 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -55
2017-06-02 09:17:22,425 DEBUG    RFM2Pi     370920 adding frame to buffer => [1496413042.412646, 10, 1952, 2009, 1046, 942, 120.56, 0, 0, 0, 0, 0, 0, 1, -55]
2017-06-02 09:17:22,426 DEBUG    RFM2Pi     370920 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:23,040 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 48 196 169 58 148 127 243 223 184 231 253 254 231 218 63 239 175 238 85 88 55 (-92)
2017-06-02 09:17:28,264 DEBUG    RFM2Pi     370921 NEW FRAME : OK 20 23 0 122 1 254 0 0 0 (-56)
2017-06-02 09:17:28,267 DEBUG    RFM2Pi     370921 Timestamp : 1496413048.26
2017-06-02 09:17:28,268 DEBUG    RFM2Pi     370921 From Node : 20
2017-06-02 09:17:28,269 DEBUG    RFM2Pi     370921    Values : [2.3000000000000003, 37.800000000000004, 25.400000000000002, 0]
2017-06-02 09:17:28,270 DEBUG    RFM2Pi     370921      RSSI : -56
2017-06-02 09:17:28,271 DEBUG    RFM2Pi     370921 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:28,272 INFO     RFM2Pi     Publishing: emonhub/rx/20/values 2.3,37.8,25.4,0
2017-06-02 09:17:28,274 INFO     RFM2Pi     Publishing: emonhub/rx/20/rssi -56
2017-06-02 09:17:28,276 DEBUG    RFM2Pi     370921 adding frame to buffer => [1496413048.264297, 20, 2.3000000000000003, 37.800000000000004, 25.400000000000002, 0, -56]
2017-06-02 09:17:28,277 DEBUG    RFM2Pi     370921 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:28,497 DEBUG    RFM2Pi     370922 NEW FRAME : OK 10 115 11 219 12 17 4 173 3 2 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-55)
2017-06-02 09:17:28,501 DEBUG    RFM2Pi     370922 Timestamp : 1496413048.5
2017-06-02 09:17:28,502 DEBUG    RFM2Pi     370922 From Node : 10
2017-06-02 09:17:28,503 DEBUG    RFM2Pi     370922    Values : [2931, 3291, 1041, 941, 120.34, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:28,503 DEBUG    RFM2Pi     370922      RSSI : -55
2017-06-02 09:17:28,504 DEBUG    RFM2Pi     370922 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:28,505 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 2931,3291,1041,941,120.34,0,0,0,0,0,0,1
2017-06-02 09:17:28,507 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -55
2017-06-02 09:17:28,510 DEBUG    RFM2Pi     370922 adding frame to buffer => [1496413048.497246, 10, 2931, 3291, 1041, 941, 120.34, 0, 0, 0, 0, 0, 0, 1, -55]
2017-06-02 09:17:28,511 DEBUG    RFM2Pi     370922 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:31,071 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496413023.931317,10,2943,3076,1049,973,120.58,0,0,0,0,0,0,1,-59],[1496413042.412646,10,1952,2009,1046,942,120.56,0,0,0,0,0,0,1,-55],[1496413048.264297,20,2.3000000000000003,37.800000000000004,25.400000000000002,0,-56],[1496413048.497246,10,2931,3291,1041,941,120.34,0,0,0,0,0,0,1,-55]]&sentat=1496413051
2017-06-02 09:17:31,193 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-02 09:17:31,194 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-02 09:17:34,662 DEBUG    RFM2Pi     370923 NEW FRAME : OK 10 184 7 27 9 37 4 187 3 27 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-54)
2017-06-02 09:17:34,665 DEBUG    RFM2Pi     370923 Timestamp : 1496413054.66
2017-06-02 09:17:34,666 DEBUG    RFM2Pi     370923 From Node : 10
2017-06-02 09:17:34,667 DEBUG    RFM2Pi     370923    Values : [1976, 2331, 1061, 955, 120.59, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:34,668 DEBUG    RFM2Pi     370923      RSSI : -54
2017-06-02 09:17:34,669 DEBUG    RFM2Pi     370923 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:34,670 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 1976,2331,1061,955,120.59,0,0,0,0,0,0,1
2017-06-02 09:17:34,672 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -54
2017-06-02 09:17:34,674 DEBUG    RFM2Pi     370923 adding frame to buffer => [1496413054.661655, 10, 1976, 2331, 1061, 955, 120.59, 0, 0, 0, 0, 0, 0, 1, -54]
2017-06-02 09:17:34,675 DEBUG    RFM2Pi     370923 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:35,991 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 62 217 225 94 222 253 111 189 179 239 189 95 247 118 30 119 171 255 122 47 249 (-91)
2017-06-02 09:17:39,645 DEBUG    RFM2Pi     370924 NEW FRAME : OK 21 0 0 28 2 205 0 206 0 246 0 (-60)
2017-06-02 09:17:39,648 DEBUG    RFM2Pi     370924 Timestamp : 1496413059.65
2017-06-02 09:17:39,649 DEBUG    RFM2Pi     370924 From Node : 21
2017-06-02 09:17:39,650 DEBUG    RFM2Pi     370924    Values : [0, 54, 20.5, 20.6, 24.6]
2017-06-02 09:17:39,650 DEBUG    RFM2Pi     370924      RSSI : -60
2017-06-02 09:17:39,651 DEBUG    RFM2Pi     370924 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:39,652 INFO     RFM2Pi     Publishing: emonhub/rx/21/values 0,54,20.5,20.6,24.6
2017-06-02 09:17:39,654 INFO     RFM2Pi     Publishing: emonhub/rx/21/rssi -60
2017-06-02 09:17:39,656 DEBUG    RFM2Pi     370924 adding frame to buffer => [1496413059.645058, 21, 0, 54, 20.5, 20.6, 24.6, -60]
2017-06-02 09:17:39,657 DEBUG    RFM2Pi     370924 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:40,877 DEBUG    RFM2Pi     370925 NEW FRAME : OK 10 164 7 215 7 25 4 217 3 26 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-54)
2017-06-02 09:17:40,881 DEBUG    RFM2Pi     370925 Timestamp : 1496413060.88
2017-06-02 09:17:40,882 DEBUG    RFM2Pi     370925 From Node : 10
2017-06-02 09:17:40,883 DEBUG    RFM2Pi     370925    Values : [1956, 2007, 1049, 985, 120.58, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:40,883 DEBUG    RFM2Pi     370925      RSSI : -54
2017-06-02 09:17:40,884 DEBUG    RFM2Pi     370925 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:40,885 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 1956,2007,1049,985,120.58,0,0,0,0,0,0,1
2017-06-02 09:17:40,887 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -54
2017-06-02 09:17:40,890 DEBUG    RFM2Pi     370925 adding frame to buffer => [1496413060.87723, 10, 1956, 2007, 1049, 985, 120.58, 0, 0, 0, 0, 0, 0, 1, -54]
2017-06-02 09:17:40,891 DEBUG    RFM2Pi     370925 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:46,634 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 191 66 44 135 214 123 113 125 255 204 114 186 239 191 42 135 125 98 118 144 196 (-96)
2017-06-02 09:17:46,956 DEBUG    RFM2Pi     370926 NEW FRAME : OK 10 153 7 181 9 16 4 196 3 40 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-55)
2017-06-02 09:17:46,960 DEBUG    RFM2Pi     370926 Timestamp : 1496413066.96
2017-06-02 09:17:46,961 DEBUG    RFM2Pi     370926 From Node : 10
2017-06-02 09:17:46,962 DEBUG    RFM2Pi     370926    Values : [1945, 2485, 1040, 964, 120.72, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:46,963 DEBUG    RFM2Pi     370926      RSSI : -55
2017-06-02 09:17:46,963 DEBUG    RFM2Pi     370926 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:46,965 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 1945,2485,1040,964,120.72,0,0,0,0,0,0,1
2017-06-02 09:17:46,967 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -55
2017-06-02 09:17:46,969 DEBUG    RFM2Pi     370926 adding frame to buffer => [1496413066.956372, 10, 1945, 2485, 1040, 964, 120.72, 0, 0, 0, 0, 0, 0, 1, -55]
2017-06-02 09:17:46,970 DEBUG    RFM2Pi     370926 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:53,185 DEBUG    RFM2Pi     370927 NEW FRAME : OK 10 96 11 82 13 21 4 198 3 14 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-55)
2017-06-02 09:17:53,189 DEBUG    RFM2Pi     370927 Timestamp : 1496413073.19
2017-06-02 09:17:53,190 DEBUG    RFM2Pi     370927 From Node : 10
2017-06-02 09:17:53,191 DEBUG    RFM2Pi     370927    Values : [2912, 3410, 1045, 966, 120.46000000000001, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:53,191 DEBUG    RFM2Pi     370927      RSSI : -55
2017-06-02 09:17:53,192 DEBUG    RFM2Pi     370927 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:53,193 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 2912,3410,1045,966,120.46,0,0,0,0,0,0,1
2017-06-02 09:17:53,196 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -55
2017-06-02 09:17:53,198 DEBUG    RFM2Pi     370927 adding frame to buffer => [1496413073.185063, 10, 2912, 3410, 1045, 966, 120.46000000000001, 0, 0, 0, 0, 0, 0, 1, -55]
2017-06-02 09:17:53,199 DEBUG    RFM2Pi     370927 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:17:56,151 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 187 241 199 95 111 157 67 199 188 93 124 6 184 253 195 187 179 191 254 159 255 (-94)
2017-06-02 09:17:59,271 DEBUG    RFM2Pi     370928 NEW FRAME : OK 10 163 7 219 7 21 4 193 3 37 47 0 0 0 0 0 0 0 0 0 0 0 0 1 0 (-59)
2017-06-02 09:17:59,275 DEBUG    RFM2Pi     370928 Timestamp : 1496413079.27
2017-06-02 09:17:59,276 DEBUG    RFM2Pi     370928 From Node : 10
2017-06-02 09:17:59,276 DEBUG    RFM2Pi     370928    Values : [1955, 2011, 1045, 961, 120.69, 0, 0, 0, 0, 0, 0, 1]
2017-06-02 09:17:59,277 DEBUG    RFM2Pi     370928      RSSI : -59
2017-06-02 09:17:59,278 DEBUG    RFM2Pi     370928 Sent to channel(start)' : ToEmonCMS
2017-06-02 09:17:59,279 INFO     RFM2Pi     Publishing: emonhub/rx/10/values 1955,2011,1045,961,120.69,0,0,0,0,0,0,1
2017-06-02 09:17:59,281 INFO     RFM2Pi     Publishing: emonhub/rx/10/rssi -59
2017-06-02 09:17:59,283 DEBUG    RFM2Pi     370928 adding frame to buffer => [1496413079.270912, 10, 1955, 2011, 1045, 961, 120.69, 0, 0, 0, 0, 0, 0, 1, -59]
2017-06-02 09:17:59,284 DEBUG    RFM2Pi     370928 Sent to channel(end)' : ToEmonCMS
2017-06-02 09:18:01,136 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496413054.661655,10,1976,2331,1061,955,120.59,0,0,0,0,0,0,1,-54],[1496413059.645058,21,0,54,20.5,20.6,24.6,-60],[1496413060.87723,10,1956,2007,1049,985,120.58,0,0,0,0,0,0,1,-54],[1496413066.956372,10,1945,2485,1040,964,120.72,0,0,0,0,0,0,1,-55],[1496413073.185063,10,2912,3410,1045,966,120.46000000000001,0,0,0,0,0,0,1,-55],[1496413079.270912,10,1955,2011,1045,961,120.69,0,0,0,0,0,0,1,-59]]&sentat=1496413081

I would recommend changing the emonTx firmware if you have a USB serial programmer.

All those zero values in the payload will most likely be causing a “bit-slip” issue, we are aware that emonTx’s running that firmware can “take a walk” and frequently do! The later firmware sends unused temperture values as a non-zero value to cure this exact issue.

We can wait to confirm if it’s that or not if you can post some log when it faults again, however if it’s not this that’s causing the current issue it could still cause an issue down the line. I would change the firmware and see if it still does it afterwards.

Let’s see what happens when it goes out first. The funny thing is it worked fine for quite a while until it started doing it randomly anywhere from a week to months apart.

Anything that could provoke a slight change in the data rate at either end could cause ‘bit-slip’, so temperature, voltage, ageing are all likely culprits (if that is indeed the problem).

Ok, it took a walk again. Here’s the same amount of the log from the hub:

2017-06-06 12:17:01,713 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 136 49 83 253 56 138 250 75 52 92 10 182 134 104 61 141 248 41 230 180 248 (-91)
2017-06-06 12:17:01,820 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 50 175 85 199 197 255 190 198 93 222 255 87 221 207 235 143 95 131 199 255 118 (-96)
2017-06-06 12:17:01,871 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496769417.338131,20,2.3000000000000003,41.2,23.5,0,-63],[1496769419.466078,19,21.900000000000002,0,45.1,2.6,-40]]&sentat=1496769421
2017-06-06 12:17:01,911 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-06 12:17:01,912 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-06 12:17:12,081 DEBUG    RFM2Pi     414799 NEW FRAME : OK 21 0 0 157 1 228 0 235 0 250 0 (-64)
2017-06-06 12:17:12,084 DEBUG    RFM2Pi     414799 Timestamp : 1496769432.08
2017-06-06 12:17:12,085 DEBUG    RFM2Pi     414799 From Node : 21
2017-06-06 12:17:12,085 DEBUG    RFM2Pi     414799    Values : [0, 41.300000000000004, 22.8, 23.5, 25]
2017-06-06 12:17:12,086 DEBUG    RFM2Pi     414799      RSSI : -64
2017-06-06 12:17:12,087 DEBUG    RFM2Pi     414799 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:17:12,088 INFO     RFM2Pi     Publishing: emonhub/rx/21/values 0,41.3,22.8,23.5,25
2017-06-06 12:17:12,091 INFO     RFM2Pi     Publishing: emonhub/rx/21/rssi -64
2017-06-06 12:17:12,093 DEBUG    RFM2Pi     414799 adding frame to buffer => [1496769432.081191, 21, 0, 41.300000000000004, 22.8, 23.5, 25, -64]
2017-06-06 12:17:12,094 DEBUG    RFM2Pi     414799 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:17:14,313 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 161 71 251 233 92 123 141 152 18 231 22 53 238 141 249 23 160 8 199 177 78 (-91)
2017-06-06 12:17:31,963 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496769432.081191,21,0,41.300000000000004,22.8,23.5,25,-64]]&sentat=1496769451
2017-06-06 12:17:32,006 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-06 12:17:32,007 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-06 12:17:32,888 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 159 127 199 26 87 76 254 253 21 190 239 213 249 191 158 192 250 255 111 127 90 (-94)
2017-06-06 12:17:35,002 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 146 240 123 89 226 52 203 53 219 160 217 26 147 67 167 216 104 142 167 227 227 (-91)
2017-06-06 12:17:38,227 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 40 203 89 231 31 49 45 127 63 57 154 248 175 38 29 112 180 242 234 214 223 (-88)
2017-06-06 12:17:45,478 DEBUG    RFM2Pi     414800 NEW FRAME : OK 21 0 0 163 1 229 0 236 0 249 0 (-67)
2017-06-06 12:17:45,481 DEBUG    RFM2Pi     414800 Timestamp : 1496769465.48
2017-06-06 12:17:45,482 DEBUG    RFM2Pi     414800 From Node : 21
2017-06-06 12:17:45,483 DEBUG    RFM2Pi     414800    Values : [0, 41.900000000000006, 22.900000000000002, 23.6, 24.900000000000002]
2017-06-06 12:17:45,484 DEBUG    RFM2Pi     414800      RSSI : -67
2017-06-06 12:17:45,484 DEBUG    RFM2Pi     414800 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:17:45,486 INFO     RFM2Pi     Publishing: emonhub/rx/21/values 0,41.9,22.9,23.6,24.9
2017-06-06 12:17:45,488 INFO     RFM2Pi     Publishing: emonhub/rx/21/rssi -67
2017-06-06 12:17:45,491 DEBUG    RFM2Pi     414800 adding frame to buffer => [1496769465.47825, 21, 0, 41.900000000000006, 22.900000000000002, 23.6, 24.900000000000002, -67]
2017-06-06 12:17:45,492 DEBUG    RFM2Pi     414800 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:17:53,633 DEBUG    RFM2Pi     414801 NEW FRAME : OK 20 23 0 156 1 235 0 0 0 (-63)
2017-06-06 12:17:53,635 DEBUG    RFM2Pi     414801 Timestamp : 1496769473.63
2017-06-06 12:17:53,636 DEBUG    RFM2Pi     414801 From Node : 20
2017-06-06 12:17:53,637 DEBUG    RFM2Pi     414801    Values : [2.3000000000000003, 41.2, 23.5, 0]
2017-06-06 12:17:53,638 DEBUG    RFM2Pi     414801      RSSI : -63
2017-06-06 12:17:53,639 DEBUG    RFM2Pi     414801 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:17:53,640 INFO     RFM2Pi     Publishing: emonhub/rx/20/values 2.3,41.2,23.5,0
2017-06-06 12:17:53,642 INFO     RFM2Pi     Publishing: emonhub/rx/20/rssi -63
2017-06-06 12:17:53,647 DEBUG    RFM2Pi     414801 adding frame to buffer => [1496769473.632877, 20, 2.3000000000000003, 41.2, 23.5, 0, -63]
2017-06-06 12:17:53,648 DEBUG    RFM2Pi     414801 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:17:57,480 DEBUG    RFM2Pi     414802 NEW FRAME : OK 19 219 0 0 0 195 1 26 0 (-35)
2017-06-06 12:17:57,482 DEBUG    RFM2Pi     414802 Timestamp : 1496769477.48
2017-06-06 12:17:57,483 DEBUG    RFM2Pi     414802 From Node : 19
2017-06-06 12:17:57,484 DEBUG    RFM2Pi     414802    Values : [21.900000000000002, 0, 45.1, 2.6]
2017-06-06 12:17:57,485 DEBUG    RFM2Pi     414802      RSSI : -35
2017-06-06 12:17:57,486 DEBUG    RFM2Pi     414802 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:17:57,487 INFO     RFM2Pi     Publishing: emonhub/rx/19/values 21.9,0,45.1,2.6
2017-06-06 12:17:57,490 INFO     RFM2Pi     Publishing: emonhub/rx/19/rssi -35
2017-06-06 12:17:57,492 DEBUG    RFM2Pi     414802 adding frame to buffer => [1496769477.479908, 19, 21.900000000000002, 0, 45.1, 2.6, -35]
2017-06-06 12:17:57,493 DEBUG    RFM2Pi     414802 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:18:02,020 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496769465.47825,21,0,41.900000000000006,22.900000000000002,23.6,24.900000000000002,-67],[1496769473.632877,20,2.3000000000000003,41.2,23.5,0,-63],[1496769477.479908,19,21.900000000000002,0,45.1,2.6,-35]]&sentat=1496769482
2017-06-06 12:18:02,051 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-06 12:18:02,052 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-06 12:18:07,184 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 175 220 221 191 79 255 61 75 247 126 111 243 233 107 214 147 127 119 59 122 231 (-93)
2017-06-06 12:18:16,960 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 164 61 23 221 232 58 211 92 222 145 185 234 170 79 108 122 241 104 73 251 114 (-90)
2017-06-06 12:18:18,873 DEBUG    RFM2Pi     414803 NEW FRAME : OK 21 0 0 166 1 228 0 236 0 248 0 (-63)
2017-06-06 12:18:18,876 DEBUG    RFM2Pi     414803 Timestamp : 1496769498.87
2017-06-06 12:18:18,877 DEBUG    RFM2Pi     414803 From Node : 21
2017-06-06 12:18:18,878 DEBUG    RFM2Pi     414803    Values : [0, 42.2, 22.8, 23.6, 24.8]
2017-06-06 12:18:18,878 DEBUG    RFM2Pi     414803      RSSI : -63
2017-06-06 12:18:18,879 DEBUG    RFM2Pi     414803 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:18:18,880 INFO     RFM2Pi     Publishing: emonhub/rx/21/values 0,42.2,22.8,23.6,24.8
2017-06-06 12:18:18,882 INFO     RFM2Pi     Publishing: emonhub/rx/21/rssi -63
2017-06-06 12:18:18,885 DEBUG    RFM2Pi     414803 adding frame to buffer => [1496769498.873337, 21, 0, 42.2, 22.8, 23.6, 24.8, -63]
2017-06-06 12:18:18,886 DEBUG    RFM2Pi     414803 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:18:28,135 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 197 237 159 247 216 211 253 223 135 207 110 249 123 239 245 254 94 219 82 237 43 (-93)
2017-06-06 12:18:32,068 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496769498.873337,21,0,42.2,22.8,23.6,24.8,-63]]&sentat=1496769512
2017-06-06 12:18:32,120 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-06 12:18:32,122 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-06 12:18:33,493 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 33 224 224 35 130 100 92 167 60 131 19 146 69 118 60 206 45 56 104 14 194 (-89)
2017-06-06 12:18:47,538 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 153 175 135 255 176 143 167 167 175 253 97 135 158 170 67 255 79 254 126 240 229 (-94)
2017-06-06 12:18:49,814 DEBUG    RFM2Pi     414804 NEW FRAME : OK 20 22 0 158 1 235 0 0 0 (-63)
2017-06-06 12:18:49,816 DEBUG    RFM2Pi     414804 Timestamp : 1496769529.81
2017-06-06 12:18:49,817 DEBUG    RFM2Pi     414804 From Node : 20
2017-06-06 12:18:49,818 DEBUG    RFM2Pi     414804    Values : [2.2, 41.400000000000006, 23.5, 0]
2017-06-06 12:18:49,819 DEBUG    RFM2Pi     414804      RSSI : -63
2017-06-06 12:18:49,820 DEBUG    RFM2Pi     414804 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:18:49,821 INFO     RFM2Pi     Publishing: emonhub/rx/20/values 2.2,41.4,23.5,0
2017-06-06 12:18:49,823 INFO     RFM2Pi     Publishing: emonhub/rx/20/rssi -63
2017-06-06 12:18:49,825 DEBUG    RFM2Pi     414804 adding frame to buffer => [1496769529.813979, 20, 2.2, 41.400000000000006, 23.5, 0, -63]
2017-06-06 12:18:49,826 DEBUG    RFM2Pi     414804 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:18:52,227 DEBUG    RFM2Pi     414805 NEW FRAME : OK 21 0 0 164 1 229 0 236 0 248 0 (-66)
2017-06-06 12:18:52,230 DEBUG    RFM2Pi     414805 Timestamp : 1496769532.23
2017-06-06 12:18:52,231 DEBUG    RFM2Pi     414805 From Node : 21
2017-06-06 12:18:52,232 DEBUG    RFM2Pi     414805    Values : [0, 42, 22.900000000000002, 23.6, 24.8]
2017-06-06 12:18:52,232 DEBUG    RFM2Pi     414805      RSSI : -66
2017-06-06 12:18:52,233 DEBUG    RFM2Pi     414805 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:18:52,234 INFO     RFM2Pi     Publishing: emonhub/rx/21/values 0,42,22.9,23.6,24.8
2017-06-06 12:18:52,236 INFO     RFM2Pi     Publishing: emonhub/rx/21/rssi -66
2017-06-06 12:18:52,239 DEBUG    RFM2Pi     414805 adding frame to buffer => [1496769532.227153, 21, 0, 42, 22.900000000000002, 23.6, 24.8, -66]
2017-06-06 12:18:52,240 DEBUG    RFM2Pi     414805 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:18:54,333 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 141 215 199 254 247 110 239 232 79 248 47 27 10 78 141 213 25 248 103 53 119 (-86)
2017-06-06 12:18:55,608 DEBUG    RFM2Pi     414806 NEW FRAME : OK 19 219 0 0 0 195 1 26 0 (-31)
2017-06-06 12:18:55,611 DEBUG    RFM2Pi     414806 Timestamp : 1496769535.61
2017-06-06 12:18:55,612 DEBUG    RFM2Pi     414806 From Node : 19
2017-06-06 12:18:55,613 DEBUG    RFM2Pi     414806    Values : [21.900000000000002, 0, 45.1, 2.6]
2017-06-06 12:18:55,614 DEBUG    RFM2Pi     414806      RSSI : -31
2017-06-06 12:18:55,614 DEBUG    RFM2Pi     414806 Sent to channel(start)' : ToEmonCMS
2017-06-06 12:18:55,615 INFO     RFM2Pi     Publishing: emonhub/rx/19/values 21.9,0,45.1,2.6
2017-06-06 12:18:55,617 INFO     RFM2Pi     Publishing: emonhub/rx/19/rssi -31
2017-06-06 12:18:55,620 DEBUG    RFM2Pi     414806 adding frame to buffer => [1496769535.608495, 19, 21.900000000000002, 0, 45.1, 2.6, -31]
2017-06-06 12:18:55,621 DEBUG    RFM2Pi     414806 Sent to channel(end)' : ToEmonCMS
2017-06-06 12:18:57,195 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 63 239 23 227 218 59 251 239 79 69 157 254 247 255 178 151 238 246 51 244 239 (-96)
2017-06-06 12:19:02,140 INFO     emoncmsorg sending: http://*.*.*.*/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1496769529.813979,20,2.2,41.400000000000006,23.5,0,-63],[1496769532.227153,21,0,42,22.900000000000002,23.6,24.8,-66],[1496769535.608495,19,21.900000000000002,0,45.1,2.6,-31]]&sentat=1496769542
2017-06-06 12:19:02,186 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://*.*.*.*
2017-06-06 12:19:02,187 INFO     emoncmsorg sending: http://*.*.*.*/myip/set.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y
2017-06-06 12:19:07,161 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 186 123 233 207 158 247 198 28 113 191 51 191 191 252 31 255 34 147 232 84 126 (-90)
2017-06-06 12:19:10,891 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 189 76 246 183 100 93 247 157 13 87 123 16 69 39 215 62 3 235 245 183 225 (-88)
2017-06-06 12:19:21,352 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content'? 129 119 119 137 225 247 124 121 225 87 138 39 47 251 221 241 4 199 249 87 251 (-90)

Can you tell from the log if it’s due to bit slip or something else?