EmonTX hardware 3.2 incorrect feed data

vk5ask · 2023-02-12T08:22:32.548Z

Hey everyone.

I have been using my OEM for a fair while now. A few months back i needed to reimage my emonpi and since then my EmonTX has not been working. All 4 of my emonTH are working well. I noticed that i was receiving this in the hub log.

2023-02-12 08:13:54,894 DEBUG    RFM2Pi     302 NEW FRAME : OK 10 133 2 0 0 124 2 43 0 9 5 0 0 178 0 0 0 13 0 0 0 103 2 0 0 0 0 0 0 (-62)
2023-02-12 08:13:54,895 WARNING  RFM2Pi     302 RX data length: 28 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']

So after digging on the forum i thought it might be a change in firmware needed, so I upgraded the emontx using the firmware update tool via usb in emonhub to emonTX v3.4 continuous and also tried discrete. The continuous image booted and looked like it was working (IDE debug) however there was no RF coming through the debug.
So after more reading i realized that i needed to use 3.2 firmware due to older hardware. I then proceeded to upload the 3.2 firmware via IDE onto the TX. This worked and booted as expected. I then noticed that the same data length error was still occurring. I then found it best to comment out the datacodes line, which i did and now i am getting readings coming through into the bug log. I thought i had finally sorted it after 6 hours of messing around! Alas i plugged it all back in and now see that the values coming through are incorrect. For example if i have the 9v AC unplugged all i get is an increasing count in the power1 field like 1, then next transmission 2, then 3 etc etc. all other values are 0. If i then plug in the 9v ac i start getting readings closer to the values, but quite incorrect still.
For example - when checking with a clamp meter power3 was 1400w and the vrms was 242 etc both steady (no temp sensors are plugged in)

2023-02-12 08:20:44,832 DEBUG    MQTT       Publishing: emon/emontx1/power1 850
2023-02-12 08:20:44,834 DEBUG    MQTT       Publishing: emon/emontx1/power2 0
2023-02-12 08:20:44,835 DEBUG    MQTT       Publishing: emon/emontx1/power3 622
2023-02-12 08:20:44,836 DEBUG    MQTT       Publishing: emon/emontx1/power4 42
2023-02-12 08:20:44,838 DEBUG    MQTT       Publishing: emon/emontx1/vrms 10.1
2023-02-12 08:20:44,839 DEBUG    MQTT       Publishing: emon/emontx1/temp1 0
2023-02-12 08:20:44,841 DEBUG    MQTT       Publishing: emon/emontx1/temp2 2.5
2023-02-12 08:20:44,843 DEBUG    MQTT       Publishing: emon/emontx1/temp3 0
2023-02-12 08:20:44,844 DEBUG    MQTT       Publishing: emon/emontx1/temp4 0.18
2023-02-12 08:20:44,845 DEBUG    MQTT       Publishing: emon/emontx1/temp5 0
2023-02-12 08:20:44,847 DEBUG    MQTT       Publishing: emon/emontx1/temp6 7.47
2023-02-12 08:20:44,848 DEBUG    MQTT       Publishing: emon/emontx1/pulse 0
2023-02-12 08:20:44,849 DEBUG    MQTT       Publishing: emon/emontx1/13 0
2023-02-12 08:20:44,851 DEBUG    MQTT       Publishing: emon/emontx1/14 0
2023-02-12 08:20:44,852 DEBUG    MQTT       Publishing: emon/emontx1/rssi -66
2023-02-12 08:20:44,853 INFO     MQTT       Publishing 'node' formatted msg
2023-02-12 08:20:44,854 DEBUG    MQTT       Publishing: emonhub/rx/10/values 850,0,622,42,10.1,0,2.5,0,0.18,0,7.47,0,0,0,-66

Any help would be greatly appreciated, i am pretty much out of ideas now!

Sam

[edit - mod]
Please use Ctl-E to format logs, shell etc

vk5ask · 2023-02-12T08:43:35.202Z

OK after chucking the values into a graph for 5 minutes i can see that they are really not correct. Power 1 is still actually incrementing by 1 each upload and the other values are pretty level even though are load is not a the moment.

image1143×634 25.2 KB

Ta

borpin · 2023-02-12T09:51:37.246Z

Can you connect up the TX as you would for flashing, but turn on the Serial Output (I think it is disabled by default for 3.2) and see what you are getting out of that, please?

Not really my area @TrystanLea or @Robert.Wall would be the best folk to chip in.

vk5ask:

A few months back i needed to reimage my emonpi and since then my EmonTX has not been working.

How did you restore the data? The USB import method?

Robert.Wall · 2023-02-12T13:23:15.511Z

vk5ask:

So after more reading i realized that i needed to use 3.2 firmware due to older hardware.

What is the firmware you loaded? At the top of the file, there should be a section of comment that defines the format of the radio output that emonHub needs to decode the message. You need to look at that and copy it into your emonhub.conf file, making sure that the NodeID is correct, and that you don’t have the NodeID definition duplicated in emonhub.conf.

If that’s not present, drop the whole .ino file into your post and I’ll make the section of emonhub.conf. It’s not hard, but searching for the information you need to do it will take you ages.

You haven’t posted enough information for me to be able to guess. 28 bytes is an emonTx V3.4 sketch.

When you commented out ‘datacodes = …’, you told it to use the default - which is unlikely to be completely correct.

The general rule is, we try to keep some sort of consistency, but with ongoing development, it’s impossible to do that all the time. In general terms, every different sketch could require an unique section for it’s data definition in emonhub.conf.

vk5ask · 2023-02-12T21:44:10.417Z

Thanks @borpin and @Robert.Wall for the quick replies!

I have had a quick look through this sketch to find the output, but its not jumping out at me so i would appreciate any assistance to build this.

sketch_feb13 emontx3.2.ino (28.8 KB)

Here is the debug from emonhub:

12 NEW FRAME : OK 10 5 25 0 0 244 0 44 0 151 1 0 0 49 2 0 0 39 0 0 0 73 4 0 0 0 0 0 0 (-57)
2023-02-12 21:48:55,542 WARNING  RFM2Pi     12 RX data length: 28 is not valid for datacodes ['h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'h', 'L']

Hopefully this is enough info atm?

Sam

[edit]
Please format with Ctl-E - second ask @vk5ask

Robert.Wall · 2023-02-12T22:29:45.734Z

That’s fine:

[[10]]
    nodename = emontx3_2
    [[[rx]]]
       names = MSG, P1, P2, P3, P4, E1, E2, E3, E4
       datacodes = L,h,h,h,h,l,l,l,l
       scales = 1,1,1,1,1,1,1,1,1
       units = n,W,W,W,W,Wh,Wh,Wh,Wh

For info:
[[10]] came from const int nodeID = 10; // emonTx RFM12B node ID
names = ... came from

 typedef struct { 
   unsigned long msgNumber; 
   int realPower_CT1;
   int realPower_CT2;
   int realPower_CT3;
   int realPower_CT4; 
   long wh_CT1;
   long wh_CT2;
   long wh_CT3;
   long wh_CT4; 
}

but with abbreviations: P = power, E = energy (Wh)
datacodes comes from there too, unsigned long = L, (signed) long is ‘l’, (signed) integer = h

You cannot send floating point (decimal) values. Voltage and temperatures are normally needed with one or two decimal places, had you had those, the sketch would have multiplied the value by 10 or 100 to send as an integer, you would have needed a ‘scales’ of 0.1 or 0.01 to put those right. I happen to know that all your values are not scaled like that, so a 1 is right for all the values - but you can look through the sketch to check.
“units” are obvious.

All four lines must have exactly the correct number of values for the data sent, and the number of bytes (integer = 2, long = 4) must add up to the number sent by the sketch - the count of numbers between “OK 10” and “(-57)”

vk5ask · 2023-02-12T23:01:29.597Z

Thanks! It is now feeding into inputs well.

image1733×384 6.32 KB

I will confirm that the values look good.

Is there a way to get the AC frq and temps in there for when in use?

Robert.Wall · 2023-02-13T00:08:18.331Z

vk5ask:

Is there a way to get the AC frq and temps in there for when in use?

Yes - there are 3 steps:

Add the values you want to send to the transmitted data structure.
Find the variables for them in the sketch, and assign the values to the new variables. For example, P1 is done like this:, tx_data.realPower_CT1 = realPower_long * powerCal_CT1;
Add them in emonhub.conf

But I can’t see that those you ask for are even measured by the sketch.

vk5ask · 2023-02-13T01:27:52.644Z

Thanks Robert. I had a suspicion that the data coming through was still not quite right and this has confirmed it.

Any thoughts what may be causing these static values? They are also a long way off the correct metering values, E3 is currently fluctuating between 6-9kw.

Robert.Wall · 2023-02-13T13:19:40.974Z

I have no idea - I don’t know what you are measuring nor what you expect to measure. Is the sketch running, is it sending data regularly, does your sketch give a serial output that you can see using the Arduino IDE, does that give “sensible” values?

Have you changed anything in the sketch since you downloaded it?

If I can find time later, I’ll load your sketch and see what I get.

Robert.Wall · 2023-02-13T15:11:29.693Z

I’ve taken a quick look at the sketch - it seems to be doing what it should for me (i.e. all 4 c.t’s produce varying numbers when I vary the current), but I don’t have a 433 MHz receiver with JeeLib available, so I can’t see exactly what is being sent.

Calibration isn’t far wrong either: with 2 A in my 20-turn coil, it’s reading almost exactly 10 kW on the first 3 inputs (actually about 450 W high with the particular c.t. & a.c. adapter I’m using).

borpin · 2023-02-13T17:17:57.346Z

I did suggest some things to try and asked a question.

EmonTX hardware 3.2 incorrect feed data emonTx

Can you connect up the TX as you would for flashing, but turn on the Serial Output (I think it is disabled by default for 3.2) and see what you are getting out of that, please? Not really my area @TrystanLea or @Robert.Wall would be the best folk to chip in. How did you restore the data? The USB import method?

vk5ask · 2023-02-13T21:04:27.397Z

Here is some output from bootup in IDE. I am unsure if this is normal for a TX as i have only looked at TH - however it seems to be missing info? I am monitoring power circuits in our house, so PV and variable load etc. It will generally be very varied. I had no issues with the sensors prior to this, so i do not think its that.

----------------------------------
Sketch ID:  emonTxV3_continuous_kwhtotals_noeeprom.ino
ADC mode:       free-running
Extra Features: none
powerCal_CT1 =      0.2619
phaseCal_CT1 =      0.22
powerCal_CT2 =      0.2619
phaseCal_CT2 =      0.41
powerCal_CT3 =      0.2619
phaseCal_CT3 =      0.60
powerCal_CT4 =      0.0478
phaseCal_CT4 =      1.25
----
voltage thresholds long:
  upper: 64000
  lower: -64000
Go!
samples per mains cycle = 50.81 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.00 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.70 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.93 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.36 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.93 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.86 
0:0, 0:0, 0:0, 1:0, samples per mains cycle = 4.41 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.86 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.90 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.21 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.18 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.07 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.84 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.01 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.18 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.01 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.06 
0:0, 0:0, 0:0, 1:0, samples per mains cycle = 4.01 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.03 
0:0, 0:0, 0:0, 1:0, samples per mains cycle = 4.53 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.80 
0:0, 0:0, 0:0, -1:0, samples per mains cycle = 4.10 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.32 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.20 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 4.33 
0:0, 0:0, 0:0, 0:0, samples per mains cycle = 3.98

Thanks again for the help.

Robert.Wall · 2023-02-13T22:37:12.387Z

There’s definitely something not right there. This is what I see, with one very minor change to your sketch - I’ve put a space and "MSG = " in front of the message number just to separate it:

2:25:14.060 -> 
22:25:14.060 -> ----------------------------------
22:25:14.093 -> Sketch ID:  emonTxV3_continuous_kwhtotals_noeeprom.ino
22:25:14.159 -> ADC mode:       free-running
22:25:14.192 -> Extra Features: none
22:25:14.192 -> powerCal_CT1 =      0.2619
22:25:14.226 -> phaseCal_CT1 =      0.22
22:25:14.259 -> powerCal_CT2 =      0.2619
22:25:14.292 -> phaseCal_CT2 =      0.41
22:25:14.325 -> powerCal_CT3 =      0.2619
22:25:14.358 -> phaseCal_CT3 =      0.60
22:25:14.358 -> powerCal_CT4 =      0.0478
22:25:14.391 -> phaseCal_CT4 =      1.25
22:25:14.425 -> ----
22:25:14.425 -> voltage thresholds long:
22:25:14.458 ->   upper: 64000
22:25:14.491 ->   lower: -64000
22:25:17.012 -> Go!
22:25:17.444 -> samples per mains cycle = 57.73 
22:25:17.477 -> 66:0, 374:0, -1:0, -8:0, samples per mains cycle = 38.46 
22:25:19.469 -> -31:0, 374:0, -19:0, -11:0, samples per mains cycle = 38.47 
22:25:21.461 -> -4:0, 374:0, 50:0, -3:0, samples per mains cycle = 38.47 
22:25:23.452 -> -77:0, 374:0  MSG=2
22:25:24.215 -> , -25:0, -14:0, samples per mains cycle = 38.33 
22:25:25.476 -> -82:0, 375:1, -48:0, 0:0, samples per mains cycle = 38.48 
22:25:27.468 -> 41:0, 375:1, 68:0, 14:0, samples per mains cycle = 38.49 
22:25:29.459 -> -112:0, 373:1, 48:0, 22:0, samples per mains cycle = 38.48 
22:25:31.450 -> 17:0, 375:1, 2:0, -1:0, samples per mains cycle = 38.49 
22:25:33.475 -> -51:0, 374:1  MSG=7
22:25:34.238 -> , 13:0, -6:0, samples per mains cycle = 38.34 
22:25:35.467 -> -29:0, 374:2, -1:0, 8:0, samples per mains cycle = 38.48 
22:25:37.458 -> -54:0, 373:2, 53:0, -4:0, samples per mains cycle = 38.49 
22:25:39.450 -> -70:0, 374:2, 34:0, 4:0, samples per mains cycle = 38.48

It’s sending the radio message every 2 s (not as expected), but I’m seeing a sensible number of samples 38 compared to your 3 or 4. Do you have the a.c. adapter connected? It completely stops reporting for me without the a.c. adapter - not surprising as it depends on seeing mains crossings to count cycles.

I had just one c.t. connected, in CT2.

vk5ask · 2023-02-13T23:03:23.385Z

I thought i had the AC adapter on, but it was off! Here is the output with it on. Original output and this output has all 4 clamps installed.

----------------------------------
Sketch ID:  emonTxV3_continuous_kwhtotals_noeeprom.ino
ADC mode:       free-running
Extra Features: none
powerCal_CT1 =      0.2619
phaseCal_CT1 =      0.22
powerCal_CT2 =      0.2619
phaseCal_CT2 =      0.41
powerCal_CT3 =      0.2619
phaseCal_CT3 =      0.60
powerCal_CT4 =      0.0478
phaseCal_CT4 =      1.25
----
voltage thresholds long:
  upper: 64000
  lower: -64000
Go!
samples per mains cycle = 57.79 
277:0, 30:0, 4881:2, -8:0, samples per mains cycle = 38.52 
279:0, 29:0, 4881:5, -8:0, samples per mains cycle = 38.52 
279:0, 30:0, 4881:8, -7:0, samples per mains cycle = 38.52 
278:0, 29:0, 4886:16, -8:0, samples per mains cycle = 38.53
277:1, 29:0, 4886:18, -8:0, samples per mains cycle = 38.53
275:1, 28:0, 4888:21, -8:0, samples per mains cycle = 38.53
277:1, 29:07, 4883:24, -7:0, samples per mains cycle = 38.38
276:1, 30:0, 4887:27, -8:0, samples per mains cycle = 38.53
277:1, 29:0, 4892:29, -8:0, samples per mains cycle = 38.53
278:1, 29:0, 4888:32, -8:0, samples per mains cycle = 38.53
278:2, 29:0, 4888:35, -8:0, samples per mains cycle = 38.53
278:2, 29:012, 4888:37, -8:0, samples per mains cycle = 38.37

– i turned off a solar inverter here to see if clamp 3 dropped, which it did.

277:2, 30:0, 3638:40, -5:0, samples per mains cycle = 38.52
277:2, 31:0, 2773:41, -5:0, samples per mains cycle = 38.52
277:2, 30:0, 2775:43, -4:0, samples per mains cycle = 38.52
274:2, 30:0, 2760:44, -5:0, samples per mains cycle = 38.52

So effectively it seems to be working since fixing the emonhub config yesterday (just needs the AC adapter on!). I also stupidly graphed the E1-4 not P1-4.

Ill have a crack at getting the AC frq into the msg now, i am fairly sure it came through originally so there must be a way!

Thanks for the help
Sam

Robert.Wall · 2023-02-13T23:21:23.582Z

vk5ask:

Ill have a crack at getting the AC frq into the msg now,

Are you sure you’re using the right sketch as a starting point?

vk5ask · 2023-02-13T23:39:15.076Z

That is a great question. I had a decent dig around on the github and that is what i thought was best, but yes i am totally not sure how to find the absolute best sketch?

Robert.Wall · 2023-02-13T23:57:38.623Z

What do you want? Obviously 3 powers, what else?
Continuous monitoring, kWh, frequency, voltage, temperatures, pulse count?

All those probably don’t exist together in an ‘old’ (contemporaneous) sketch, but a modern sketch converted back for the V 3.2 is probably the way to go.

vk5ask · 2023-02-14T00:05:52.189Z

No worries that makes sense. I am mainly looking for the 4 powers, frequency and voltage.

Robert.Wall · 2023-02-14T00:13:15.167Z

I did a special emonTx V3.4 sketch converted back to V3.2 in 2020 (according to the file date), which I’d forgotten about. This will give you

[[15]]
  nodename = emontx15
  [[[rx]]]
    names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
    datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
    scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
    units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
    whitening = 1

It uses emonLibCM and that has frequency available, so it’s trivial to add it.