Just wondering if the brains trust can please help me spot what’s going on here. There’s a huge difference between pulse count kWh and kWh recorded via CT and Ph1 VT inputs. Could this magnitude of error really be due to the (significant) imbalance in loads across the three phases? I was not able to isolate any of the loads at the premises, nor identify which sockets were attached to which phase (other than that the socket the voltage transformer is plugged into is obviously set to be Ph1).
I used a known resistive load (2000W radiant heater) to find which phase the VT socket was connected to, but had to attach the CTs for the other two phases according to labelled phase rotation at the utility meter, since it was not possible to easily identify any sockets connected to the other phases in order to use the test load to confirm. If I recall correctly I did try switching the Ph2 and Ph3 CTs around but I think that gave obviously gibberish results.
I followed the calibration instructions for the 3phase sketch on an isolated single phase test circuit and the maximum total power (all phases) error showing up after calibration was around 4W.
The optical pulse sensor is attached to “Pulse1” LED on an EDMI Atlas Mk10D labelled as having 1000imp/kWh and a visual check indicated that it was correctly picking up LED pulses prior to sticking the sensor on with the self-adhesive tape.
Here is a sample of data logged (exported from emoncms via csv with the raw pulse count converted to kWh in excel - the emoncms pulse count to kWh process is not working, but that’s probably a topic for a separate thread):
‘’’
Timestamp
P1 (W)
P1 (kWh)
P2 (W)
P2 (kWh)
P3 (W)
P3 (kWh)
Total (W)
Total (kWh)
P1 (V)
P1 (PF)
P2 (PF)
P3 (PF)
Pulse (Count)
Pulse (kWh)
07/05/2021 20:00
496
1.78
190
0.26
652
1.2
1339
3.24
239.76
0.89
0.96
0.78
2437
2.437
07/05/2021 20:01
494
1.79
189
0.26
650
1.21
1332
3.26
239.37
0.89
0.96
0.78
2459
2.459
07/05/2021 20:02
492
1.8
188
0.27
650
1.22
1330
3.28
239.24
0.89
0.96
0.78
2481
2.481
07/05/2021 20:03
745
1.81
187
0.27
648
1.23
1580
3.31
239.11
0.77
0.96
0.78
2503
2.503
07/05/2021 20:04
556
1.82
187
0.27
653
1.24
1396
3.33
239.32
0.92
0.96
0.78
2527
2.527
07/05/2021 20:05
556
1.83
186
0.27
649
1.25
1392
3.35
239.42
0.92
0.96
0.78
2550
2.55
07/05/2021 20:06
555
1.83
186
0.28
642
1.26
1383
3.37
239.77
0.91
0.96
0.78
2573
2.573
07/05/2021 20:07
550
1.84
136
0.28
643
1.27
1329
3.4
239.86
0.91
0.94
0.78
2596
2.596
07/05/2021 20:08
546
1.85
33
0.28
647
1.28
1226
3.42
240.26
0.91
0.17
0.78
2617
2.617
07/05/2021 20:09
552
1.86
33
0.28
993
1.3
1579
3.44
240.24
0.91
0.17
0.81
2641
2.641
07/05/2021 20:10
553
1.87
34
0.28
633
1.31
1220
3.46
239.83
0.91
0.17
0.77
2662
2.662
07/05/2021 20:11
554
1.88
33
0.28
982
1.32
1568
3.48
239.12
0.91
0.17
0.81
2683
2.683
07/05/2021 20:12
555
1.89
32
0.28
980
1.34
1567
3.51
239.11
0.91
0.17
0.81
2709
2.709
07/05/2021 20:13
566
1.9
64
0.29
620
1.35
1250
3.53
239.51
0.92
0.32
0.77
2734
2.734
07/05/2021 20:14
552
1.91
289
0.29
619
1.36
1461
3.56
239.76
0.91
0.84
0.77
2757
2.757
07/05/2021 20:15
549
1.92
289
0.29
621
1.37
1458
3.58
239.57
0.91
0.83
0.77
2784
2.784
07/05/2021 20:16
563
1.93
288
0.3
964
1.39
1815
3.61
239.71
0.92
0.83
0.81
2813
2.813
07/05/2021 20:17
550
1.94
158
0.3
1100
1.4
1808
3.64
239.94
0.91
0.64
0.83
2839
2.839
07/05/2021 20:18
486
1.94
32
0.3
1083
1.42
1601
3.67
237.71
0.89
0.17
0.83
2867
2.867
07/05/2021 20:19
491
1.95
33
0.3
1157
1.44
1680
3.7
241.49
0.89
0.17
0.82
2894
2.894
07/05/2021 20:20
490
1.96
31
0.3
1609
1.46
2131
3.72
241.59
0.89
0.16
0.86
2923
2.923
07/05/2021 20:21
488
1.97
34
0.31
1024
1.48
1547
3.75
241.73
0.89
0.18
0.82
2949
2.949
07/05/2021 20:22
487
1.98
35
0.31
1015
1.49
1537
3.77
241.57
0.89
0.18
0.82
2972
2.972
07/05/2021 20:23
493
1.99
36
0.31
726
1.5
1255
3.79
243.71
0.89
0.18
0.79
2994
2.994
07/05/2021 20:24
491
1.99
37
0.31
730
1.51
1258
3.82
244.23
0.89
0.18
0.79
3016
3.016
07/05/2021 20:25
496
2
37
0.31
755
1.53
1288
3.84
245.06
0.89
0.19
0.77
3039
3.039
07/05/2021 20:26
490
2.01
36
0.31
962
1.54
1488
3.86
242.98
0.89
0.18
0.8
3061
3.061
07/05/2021 20:27
488
2.02
36
0.31
845
1.55
1368
3.88
242.67
0.89
0.18
0.79
3082
3.082
07/05/2021 20:28
488
2.03
36
0.31
701
1.57
1225
3.91
242.72
0.89
0.18
0.78
3104
3.104
07/05/2021 20:29
488
2.03
35
0.31
745
1.58
1267
3.93
242.16
0.89
0.18
0.79
3127
3.127
07/05/2021 20:30
484
2.04
35
0.31
763
1.6
1282
3.95
242.2
0.89
0.18
0.8
3150
3.15
‘’’
I’d be extremely grateful for any thoughts anyone might have (@Robert.Wall)?.
It looks very much to me as if you started the two recordings (pulses from the meter and Wh from the c.t’s) at different times, because over the limited range that I can see, having subtracted the starting value from each, the slopes (the calibration) agree fairly closely - not that I’d lay much store on comparing over a difference of only 700 Wh.
The red line is 1:1, the blue is meter pulses (y-axis) vs c.t kWh (x-axis)
Putting a trend line on the blue curve, I get f(x) = 1.005798 x + 0.0022473, so your calibration is wrong by 0.58%. I wouldn’t complain at that.
Have you fallen foul of emonCMS carrying on from where it left off when accumulating values? Have you started counting pulses after you started accumulating energy? Those are two areas to look at.
Thanks @Robert - sorry I probably should have included more spread out data to show the error as it grows over time.
AHHHHH!!! When I exported the whole period of data logging from Friday till last night at wider datapoint intervals to show you - it all became clear! For some inexplicable reason the raw pulse count feed managed to reset itself on Saturday night!
How might that have happened and how might I prevent that happening in the future?
I did delete all stored data at the same time under the setup feeds screen (both feeds ticked together) to try to prevent this from happening… perhaps there was just enough of an electrical load at the time that the script operating sequentially didn’t manage to negate that initial difference…
Like I said, you fell foul of emonCMS carrying on with accumulating the energy. But as you’re not accumulating the energy from the pulse count inside emonCMS, that started from zero.
That is what you intended (but didn’t quite work), the same would happen if you or a power failure restarted the emonTx.
Normally of course, you want it to carry on accumulating, that’s the behaviour of your supplier’s meter.
Not sure I follow @Robert - am I not accumulating the raw pulse count inside emoncms? Why did the raw pulse count feed reset - would a power outage have somehow caused this? If so, why did all the other cumulative feeds not also reset?
PS: I can’t explain why the kWh accumulator from the raw pulse count process previously wasn’t working - a short while ago I repeated exactly the same procedure to add the process that I’d already tried a dozen times before and it just magically started working!
Did you reset/restart the emonTx at around the time you tried to reset emonCMS? Or at some other time before 20:00 on the 7th?
What happens inside emonCMS is, if it sees the energy drop to zero, then it saves the last value and perpetually adds that to the incoming numbers. The 3-phase sketch only sends the present power, so you invoke that behaviour when you use the power to Wh process.
The 3-phase sketch internally accumulates the pulse count, but does not remember it across a restart. (I was forgetting for a moment you were getting the pulses from the emonTx - presumably). So I think the emonTx has restarted after you reset emonCMS.
I can’t remember exact details ('twas a long day!) but yes it was restarted at some point prior to 20:00 on the 7th (it was being set up and commissioned that evening).
Ah - this is how it avoids cumulative data stored in emontx memory being lost when the emontx resets?
Is there a mechanism to achieve this with the pulse counter too? EDIT: obviously this is what the “kWh Accumulator” process is meant to do, it just wasn’t working (for some unknown reason) when the reset happened… (please correct me if I’ve misunderstood)
I just realised I hadn’t properly thought through how this works - the “Log to feed” process simply passes the current cumulative raw pulse count from the emontx memory into the emoncms data log system (regardless of whether it’s higher or lower than the previous figure)?
Emoncms is running on a RPi powered by the emontx 5V rail, so not sure how the emontx could have reset without the Pi also rebooting (unless someone managed to push the reset button, but that almost certainly did not happen at the time in question)?
Ok, just tried to manually correct the lost pulse count temporarily by adding in what was lost when the emontx spat the dummy using the “+” process but much like the first dozen times I tried to get the kWh accumulator process to work it’s not behaving as I would have expected:
Why is the manual calibration factor added to the raw log to feed process not being passed on to the subsequent processes?
Am I missing a step here to achieve what I would like to do?
EDIT: Ahh! Herein lies the clue I think - "it also filter’s out spikes in energy use that are larger than a max power threshold set in the processor, assuming these are error’s, the max power threshold is set to 25kW. " So how do I add a calibration factor bigger than the equivalent of 25kW since the last input? Although, somehow (after a dozen or so attempts) I managed to get the kWh accumulator to start at 7x.xxkWh based solely on the current state of the raw pulse counter feed…
Exactly - Log to feed doesn’t know and doesn’t care what the units are or what they mean. In fact, none of the processes inherently know that. It’s the context in which you use them that determines the outcome, and whether the result is a sensible unit or nonsense. (e.g. You could pick up an energy feed from another input and do “power to kWh” on it - what would that then mean in practical units? kWh² )
You’ll need someone who understands the inner workings of emonCMS if you need a more detailed explanation.
Don’t suppose you know a way around the constraint of the kWh Accumulator filtering out “spikes in energy use that are larger than a max power [25kW] threshold set in the processor” so I can manually add a large calibration factor (equal to the pulse count before the emontx lost the plot) to get the kWh Accumulator feed to match its correct value without resetting the rest of the data logs?
Where is the 25kW limit to be found in the code? Can it be temporarily altered then changed back again, or is there a better way to achieve this outcome?
And I’m guessing here that you mean the emonTx is itself supplied at 5 V via the USB connector.
If the Pi pulled enough current to dip the 5V rail far enough, either when rebooting or it momentarily demanded more current, that could be your answer.
I’ve not delved into the precise circuit design of the emontx, but thought it would be far more tolerant of voltage dips than the Pi since at ATMEGA328p can operate down to below three volts I think (from memory)?
Now that I have the kWh accumulator process working (albeit without the correction calibration for the first couple of days of logging), the raw counter resetting shouldn’t cause a big problem in future I guess.
If anyone else knows how to calibrate the kWh accumulator process by more than the equivalent of a 25kW load occurring between sampling intervals, I’d be grateful to learn…
When I exported the whole period of data logging from Friday till last night at wider datapoint intervals to show you - it all became clear! For some inexplicable reason the raw pulse count feed managed to reset itself on Saturday night!
)