New SD card = Null Inputs

SD card died after a couple of years, and I am getting Null Inputs. Via “new device” I am selecting EmonPI SolarPV type 2, and when I check EmonHub

Details:
I have an EmonPI on a Raspberry Pi Model B Rev 2.0 - 512MB (Sony UK)
New SD card written using the emonSD-17Oct19, with the change made as per the text.

2020-11-10 10:15:42,525 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1
2020-11-10 10:15:42,699 DEBUG    RFM2Pi     1 NEW FRAME : OK 10 242 2 4 0 0 0 0 0 211 93 0 0
2020-11-10 10:15:42,717 WARNING  RFM2Pi     1 RX data length: 12 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']
2020-11-10 10:15:42,845 DEBUG    RFM2Pi     acknowledged command: > 1p
2020-11-10 10:15:43,215 DEBUG    RFM2Pi     acknowledged command:  i     - set node ID (standard node ids are 1..30)
2020-11-10 10:15:43,363 DEBUG    RFM2Pi     acknowledged command:  b      - set MHz band (4 = 433, 8 = 868, 9 = 915)
2020-11-10 10:15:43,513 DEBUG    RFM2Pi     acknowledged command:  o   - change frequency offset within the band (default 1600)
2020-11-10 10:15:44,034 DEBUG    RFM2Pi     acknowledged command:  g    - set network group (RFM12 only allows 212, 0 = any)
2020-11-10 10:15:44,173 DEBUG    RFM2Pi     acknowledged command:  c      - set collect mode (advanced, normally 0)
2020-11-10 10:15:44,468 DEBUG    RFM2Pi     acknowledged command: ..., a - send data packet to node , request ack
2020-11-10 10:15:44,633 DEBUG    RFM2Pi     acknowledged command: ..., s - send data packet to node , no ack
2020-11-10 10:15:44,902 DEBUG    RFM2Pi     acknowledged command:  q      - set quiet mode (1 = don't report bad packets)
2020-11-10 10:15:45,122 DEBUG    RFM2Pi     acknowledged command:  x      - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2020-11-10 10:15:45,555 DEBUG    RFM2Pi     acknowledged command: ,,, f     - FS20 command (868 MHz)
2020-11-10 10:15:45,789 DEBUG    RFM2Pi     acknowledged command: ,, k              - KAKU command (433 MHz)
2020-11-10 10:15:46,109 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1
2020-11-10 10:15:51,366 DEBUG    RFM2Pi     8 NEW FRAME : OK 10 221 2 4 0 0 0 0 0 118 93 0 0
2020-11-10 10:15:51,370 WARNING  RFM2Pi     8 RX data length: 12 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']
2020-11-10 10:16:02,126 DEBUG    RFM2Pi     9 NEW FRAME : OK 10 240 2 4 0 0 0 0 0 197 93 0 0
2020-11-10 10:16:02,130 WARNING  RFM2Pi     9 RX data length: 12 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']

Config:

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,
    group = 210
    frequency = 433
    baseid = 5                              # emonPi / emonBase nodeID
    calibration = 230V                      # (UK/EU: 230V, US: 110V)
    quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
    # interval =  300                         # Interval to transmit time to emonGLCD (seconds)
[[9]]
   nodename = emontx2
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
      datacode = h
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

Lastly “minicom -b 38400 -o -D /dev/ttyAMA0” works fine:
> 0h

Available commands:
i - set node ID (standard node ids are 1…30)
b - set MHz band (4 = 433, 8 = 868, 9 = 915)
o - change frequency offset within the band (default 1600)
96…3903 is the range supported by the RFM12B
g - set network group (RFM12 only allows 212, 0 = any)
c - set collect mode (advanced, normally 0)
t - broadcast max-size test packet, request ack
…, a - send data packet to node , request ack
…, s - send data packet to node , no ack
q - set quiet mode (1 = don’t report bad packets)
x - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
123 z - total power down, needs a reset to start up again
Remote control commands:
,,, f - FS20 command (868 MHz)
,, k - KAKU command (433 MHz)
Current configuration:
E i5 g210 @ 433 MHz q1
OK 10 72 3 3 0 0 0 0 0 196 93 0 0
OK 10 80 3 6 0 0 0 0 0 129 93 0 0
OK 10 66 3 4 0 0 0 0 0 198 93 0 0
OK 10 65 3 3 0 0 0 0 0 234 93 0 0

Not sure what to do?

First, the latest version is the July 20 one, so I’d advise starting again with that. I have that running here on the same Pi as yours, so there’s no problem there.

That tells me that the data is coming from Node 10, what is Node 10? Until we know what that is, and what it’s sending, I can’t be sure. But it might be an emonTx sending 4 powers, voltage and something else. In that case, for the config for Node 10 you need:

[[10]]
   nodename = emontx
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp
      datacode = h
      scales = 1,1,1,1,0.01,0.1
      units =W,W,W,W,V,C

but I stress - this is a best guess and I could easily be wrong.

1 Like

As Robert says above, it’s your node definition that is incorrect

The first line shows node 10 is sending 12 byte values, and the second line is saying you have defined node 10 as having 26 byte values so they don’t match.

You haven’t shown us node 10 definition but I assume node 10 has a line datacodes = h, h, h, h, h, h, h, h, h, h, h, L and should be more like the example Robert has given just like the node 9 you have posted datacode = h or datacodes = h,h,h,h,h (note the datacode or datacodes, plural) to accommodate the 10 byte payload from your emontx.

1 Like

Do you mean you have an EmonPi (i.e. the one with an LCD screen)?

I second Paul’s comment - start again with the 2020 image.

Are you importing your old settings and data?

EmonPi should be using Node 5.

Looks to me like the EmonPi firmware might have got a little upset.

In the admin page, check the updates

image

Click on the pull down arrow.

select
image

and update the firmware (only if this really is an EmonPi).

@pb66, @Robert.Wall

Thankyou for the comments, it is now working but was still a bit fiddly, not sure why it had issues with a fresh install, I don’t remember it being quite that fiddly 6-7 years ago when I first got it going :slight_smile:

  1. used latest/new image (no changes made to sd card apart from ssh file)
  2. modified emoncfg as per robert.wall’s post (datacode = h)
  3. did full reboot
    Sometimes it is taking multiple minutes to update which I am working through.
    (it’s night here currently)

@borpin Thankyou, yes I provided wrong information after trying different things, not strictly speaking an EMONPI.

What you have I expect is an EmonBase plus the small RFM card plugged in, and an EmonTX.

I’m surprised you needed to. It should have just worked.

It should be showing data on the inputs every 10s.

Glad you got it sorted.

Things were significantly different back then, the first iteration of the current node definitions didn’t arrive until 2015 when the emonpi was launched. Before that emonhub effectively was datacodes = h by default since all the earlier sketches only sent integers and the payload was indeed as Robert described

\

As above.

Robert’s comment.

You can tell it isn’t an emonpi by the response to the 1p command in the log.

@Mark_Sydney, try setting quiet = 0 in the emonhub.conf file, it will let you see if there are any less perfect payloads being received from the emontx and discarded due to failing CRC checks.

1 Like

@borpin I was a very early buyer of the original solution… I can’t remember exactly what I bought. A quick google confirms that you are pretty close to the mark

me too :frowning:

Yeah, it never used to do this or better, but this is odd :frowning:

Can you repost the ‘emonhub.conf’ and the ‘emonhub.log’ from the point of a restart of the emonhub (from the UI), please?

@Mark_Sydney - if an early one, is the Baud rate set correctly in emonhub.conf - you probably have an RFM12Pi.

Can do that:

2020-11-11 10:56:27,025 DEBUG MQTT Publishing: emon/emontx1/vrms 238.94
2020-11-11 10:56:27,045 DEBUG MQTT Publishing: emon/emontx1/temp1 0
2020-11-11 10:56:27,050 INFO MQTT Publishing: emonhub/rx/10/values 1079,6,0,0,238.94,0
2020-11-11 10:56:35,091 DEBUG emoncmsorg Buffer size: 2
2020-11-11 10:56:37,454 DEBUG RFM2Pi 10 NEW FRAME : OK 10 51 4 6 0 0 0 0 0 59 93 0 0
2020-11-11 10:56:37,471 DEBUG RFM2Pi 10 Timestamp : 1605092197.453585
2020-11-11 10:56:37,511 DEBUG RFM2Pi 10 From Node : 10
2020-11-11 10:56:37,514 DEBUG RFM2Pi 10 Values : [1075, 6, 0, 0, 238.67000000000002, 0]
2020-11-11 10:56:37,546 DEBUG RFM2Pi 10 Sent to channel(start)’ : ToEmonCMS
2020-11-11 10:56:37,549 DEBUG RFM2Pi 10 Sent to channel(end)’ : ToEmonCMS
2020-11-11 10:56:37,737 DEBUG MQTT Publishing: emon/emontx1/power1 1075
2020-11-11 10:56:37,760 DEBUG MQTT Publishing: emon/emontx1/power2 6
2020-11-11 10:56:37,770 DEBUG MQTT Publishing: emon/emontx1/power3 0
2020-11-11 10:56:37,786 DEBUG MQTT Publishing: emon/emontx1/power4 0
2020-11-11 10:56:37,810 DEBUG MQTT Publishing: emon/emontx1/vrms 238.67000000000002
2020-11-11 10:56:37,815 DEBUG MQTT Publishing: emon/emontx1/temp1 0
2020-11-11 10:56:37,830 INFO MQTT Publishing: emonhub/rx/10/values 1075,6,0,0,238.67000000000002,0
2020-11-11 10:56:48,302 DEBUG RFM2Pi 11 NEW FRAME : OK 10 210 3 3 0 0 0 0 0 47 93 0 0
2020-11-11 10:56:48,309 DEBUG RFM2Pi 11 Timestamp : 1605092208.301349
2020-11-11 10:56:48,322 DEBUG RFM2Pi 11 From Node : 10
2020-11-11 10:56:48,324 DEBUG RFM2Pi 11 Values : [978, 3, 0, 0, 238.55, 0]
2020-11-11 10:56:48,327 DEBUG RFM2Pi 11 Sent to channel(start)’ : ToEmonCMS
2020-11-11 10:56:48,340 DEBUG RFM2Pi 11 Sent to channel(end)’ : ToEmonCMS
2020-11-11 10:56:48,720 DEBUG MQTT Publishing: emon/emontx1/power1 978
2020-11-11 10:56:48,730 DEBUG MQTT Publishing: emon/emontx1/power2 3
2020-11-11 10:56:48,757 DEBUG MQTT Publishing: emon/emontx1/power3 0
2020-11-11 10:56:48,770 DEBUG MQTT Publishing: emon/emontx1/power4 0
2020-11-11 10:56:48,780 DEBUG MQTT Publishing: emon/emontx1/vrms 238.55
2020-11-11 10:56:48,800 DEBUG MQTT Publishing: emon/emontx1/temp1 0
2020-11-11 10:56:48,820 INFO MQTT Publishing: emonhub/rx/10/values 978,3,0,0,238.55,0

I found an old backup DVD and it had an emon backup, within the old emonhub.conf it was set to 9600, so I have put that setting in place and restarted emonhub

What was it before you changed it? The logs you provided show valid serial comms, although the data was coming in very sporadically, the fact any data was coming in means the baud was correct!

“missing packets” as opposed to “no packets” suggest a rf (range/interference etc) or (if you have updated the rfm2pi FW) maybe something like “bitslip” due to the string of zeros in the values. either way this can be spotted by setting quiet to false as I previously suggested.

Ok I have turned off “quiet”, discarding frame unreliable content is showing up. I have power cycled the old emonTX

2020-11-11 11:49:26,164 DEBUG    emoncmsorg Buffer size: 2
2020-11-11 11:49:26,307 DEBUG    RFM2Pi     32 NEW FRAME : OK 10 63 4 2 0 0 0 0 0 70 93 0 0
2020-11-11 11:49:26,319 DEBUG    RFM2Pi     32 Timestamp : 1605095366.306587
2020-11-11 11:49:26,322 DEBUG    RFM2Pi     32 From Node : 10
2020-11-11 11:49:26,325 DEBUG    RFM2Pi     32    Values : [1087, 2, 0, 0, 238.78, 0]
2020-11-11 11:49:26,348 DEBUG    RFM2Pi     32 Sent to channel(start)' : ToEmonCMS
2020-11-11 11:49:26,350 DEBUG    RFM2Pi     32 Sent to channel(end)' : ToEmonCMS
2020-11-11 11:49:26,476 DEBUG    MQTT       Publishing: emon/emontx1/power1 1087
2020-11-11 11:49:26,488 DEBUG    MQTT       Publishing: emon/emontx1/power2 2
2020-11-11 11:49:26,502 DEBUG    MQTT       Publishing: emon/emontx1/power3 0
2020-11-11 11:49:26,516 DEBUG    MQTT       Publishing: emon/emontx1/power4 0
2020-11-11 11:49:26,527 DEBUG    MQTT       Publishing: emon/emontx1/vrms 238.78
2020-11-11 11:49:26,538 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2020-11-11 11:49:26,555 INFO     MQTT       Publishing: emonhub/rx/10/values 1087,2,0,0,238.78,0
2020-11-11 11:49:35,305 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 20 53 138 91 83 50 94 168 169 131 84 92 208 37 193 227 233 220 109 174 248

How frequently? Is it every 10s or more? Can you provide a longer log example?

How frequently are values getting through now? Your last log of almost 22s shows 2 successful packets just over10s apart.

about 25 times in 45 minutes, happy to paste a longer log, but don’t want to break anything?

Sorry, is that the number of discarded? or the number of successful? Are the discards 10s apart and where a successful packet should be?

You shouldn’t break anything, we don’t need hours of logs just a few minutes for now, you can paste more if needed.

Ok I hope this is sufficient, there is a correlation to your question regarding 10 seconds

2020-11-11 13:19:01,866 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 3 1 133 255 149 61 186 4 55 236 186 242 62 68 11 140 123 160 205 242 95
2020-11-11 13:19:06,573 DEBUG    emoncmsorg Buffer size: 3
2020-11-11 13:19:08,914 DEBUG    RFM2Pi     526 NEW FRAME : OK 10 231 3 5 0 0 0 0 0 14 94 0 0
2020-11-11 13:19:08,927 DEBUG    RFM2Pi     526 Timestamp : 1605100748.913388
2020-11-11 13:19:08,940 DEBUG    RFM2Pi     526 From Node : 10
2020-11-11 13:19:08,942 DEBUG    RFM2Pi     526    Values : [999, 5, 0, 0, 240.78, 0]
2020-11-11 13:19:08,945 DEBUG    RFM2Pi     526 Sent to channel(start)' : ToEmonCMS
2020-11-11 13:19:08,958 DEBUG    RFM2Pi     526 Sent to channel(end)' : ToEmonCMS
2020-11-11 13:19:09,185 DEBUG    MQTT       Publishing: emon/emontx1/power1 999
2020-11-11 13:19:09,195 DEBUG    MQTT       Publishing: emon/emontx1/power2 5
2020-11-11 13:19:09,210 DEBUG    MQTT       Publishing: emon/emontx1/power3 0
2020-11-11 13:19:09,228 DEBUG    MQTT       Publishing: emon/emontx1/power4 0
2020-11-11 13:19:09,234 DEBUG    MQTT       Publishing: emon/emontx1/vrms 240.78
2020-11-11 13:19:09,239 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2020-11-11 13:19:09,258 INFO     MQTT       Publishing: emonhub/rx/10/values 999,5,0,0,240.78,0
2020-11-11 13:19:11,601 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 20 238 248 149 150 137 49 88 223 155 238 48 232 183 247 47 9 144 185 31 109
2020-11-11 13:19:19,813 DEBUG    RFM2Pi     527 NEW FRAME : OK 10 229 3 7 0 0 0 0 0 155 93 0 0
2020-11-11 13:19:19,830 DEBUG    RFM2Pi     527 Timestamp : 1605100759.812412
2020-11-11 13:19:19,858 DEBUG    RFM2Pi     527 From Node : 10
2020-11-11 13:19:19,861 DEBUG    RFM2Pi     527    Values : [997, 7, 0, 0, 239.63, 0]
2020-11-11 13:19:19,864 DEBUG    RFM2Pi     527 Sent to channel(start)' : ToEmonCMS
2020-11-11 13:19:19,866 DEBUG    RFM2Pi     527 Sent to channel(end)' : ToEmonCMS
2020-11-11 13:19:19,981 DEBUG    MQTT       Publishing: emon/emontx1/power1 997
2020-11-11 13:19:19,998 DEBUG    MQTT       Publishing: emon/emontx1/power2 7
2020-11-11 13:19:20,004 DEBUG    MQTT       Publishing: emon/emontx1/power3 0
2020-11-11 13:19:20,021 DEBUG    MQTT       Publishing: emon/emontx1/power4 0
2020-11-11 13:19:20,027 DEBUG    MQTT       Publishing: emon/emontx1/vrms 239.63
2020-11-11 13:19:20,038 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2020-11-11 13:19:20,050 INFO     MQTT       Publishing: emonhub/rx/10/values 997,7,0,0,239.63,0
2020-11-11 13:19:30,684 DEBUG    RFM2Pi     528 NEW FRAME : OK 10 237 3 6 0 0 0 0 0 233 93 0 0
2020-11-11 13:19:30,693 DEBUG    RFM2Pi     528 Timestamp : 1605100770.683750
2020-11-11 13:19:30,740 DEBUG    RFM2Pi     528 From Node : 10
2020-11-11 13:19:30,742 DEBUG    RFM2Pi     528    Values : [1005, 6, 0, 0, 240.41, 0]
2020-11-11 13:19:30,745 DEBUG    RFM2Pi     528 Sent to channel(start)' : ToEmonCMS
2020-11-11 13:19:30,813 DEBUG    RFM2Pi     528 Sent to channel(end)' : ToEmonCMS
2020-11-11 13:19:30,980 DEBUG    MQTT       Publishing: emon/emontx1/power1 1005
2020-11-11 13:19:30,998 DEBUG    MQTT       Publishing: emon/emontx1/power2 6
2020-11-11 13:19:31,003 DEBUG    MQTT       Publishing: emon/emontx1/power3 0
2020-11-11 13:19:31,019 DEBUG    MQTT       Publishing: emon/emontx1/power4 0
2020-11-11 13:19:31,025 DEBUG    MQTT       Publishing: emon/emontx1/vrms 240.41
2020-11-11 13:19:31,042 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2020-11-11 13:19:31,058 INFO     MQTT       Publishing: emonhub/rx/10/values 1005,6,0,0,240.41,0
2020-11-11 13:19:35,564 DEBUG    RFM2Pi     Discarding RX frame 'unreliable content': ? 26 109 190 255 133 27 159 125 124 152 13 141 36 24 235 173 67 164 82 28 109
2020-11-11 13:19:36,632 DEBUG    emoncmsorg Buffer size: 3
2020-11-11 13:19:41,411 DEBUG    RFM2Pi     529 NEW FRAME : OK 10 173 3 5 0 0 0 0 0 161 93 0 0
2020-11-11 13:19:41,432 DEBUG    RFM2Pi     529 Timestamp : 1605100781.410677
2020-11-11 13:19:41,435 DEBUG    RFM2Pi     529 From Node : 10
2020-11-11 13:19:41,448 DEBUG    RFM2Pi     529    Values : [941, 5, 0, 0, 239.69, 0]
2020-11-11 13:19:41,450 DEBUG    RFM2Pi     529 Sent to channel(start)' : ToEmonCMS
2020-11-11 13:19:41,453 DEBUG    RFM2Pi     529 Sent to channel(end)' : ToEmonCMS
2020-11-11 13:19:41,678 DEBUG    MQTT       Publishing: emon/emontx1/power1 941
2020-11-11 13:19:41,700 DEBUG    MQTT       Publishing: emon/emontx1/power2 5
2020-11-11 13:19:41,718 DEBUG    MQTT       Publishing: emon/emontx1/power3 0
2020-11-11 13:19:41,737 DEBUG    MQTT       Publishing: emon/emontx1/power4 0
2020-11-11 13:19:41,748 DEBUG    MQTT       Publishing: emon/emontx1/vrms 239.69
2020-11-11 13:19:41,762 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2020-11-11 13:19:41,784 INFO     MQTT       Publishing: emonhub/rx/10/values 941,5,0,0,239.69,0

Yes, that was the number discarded, I opened up the log in Notepad++ and counted discarded in the timeframe

From the timing though, it doesn’t look like a frame from the EmonTX that was discarded, but some other received ‘noise’.

The gap is slightly over 10s, but that is normal and how the sketch is designed. It means you miss the occasional slot in the Feed data (which comes up as NULL).

If you

tail -f /var/log/emonhub/emonhub.log | grep " NEW FRAME \|Discarding "

you will get a running output which will summarise what is happening.

1 Like

Erm…
No.

The emonTx (and emonPi) send interval is slightly less than the feed interval, meaning there are no gaps in the database - rather, occasionally a value is overwritten before the next time slot comes along.

And that reinforces the argument that the errant data is not from the emonTx.

1 Like