@Jon are you able to provide a section of emonhub.log with the matching emoncms data as above?
Paul - Enclosed is a section of the above. It starts near 2016-11-06 15:17:04 (CST) and ends near 2016-11-06 16:17:00
Hi @Jon, I have had a bit of a look and it seems the data is being transported, processed and saved faithfully, the packets recieved from the emonPi firmware in emonhub seem to correlate to the graph so I do think the issue is in the emonPi sketch or the pulse counter itself, when you get alerted to a non-incrementing pulse count, do you check the led on the optical pulse counter?
If, as you say the the pulses are being “delayed” that would suggest a “hold up” somewhere, but I would have expected to see an instant “batch” increase in pulsecount rather than the gradual increase seen when the pulsecount starts moving. I’ll take another look tomorrow and let you know if I spot anything.
Yes, I’ve watched the Opto Pulse sensor green LED blink without a pulse counter increment.
[quote=“pb66, post:5, topic:2139”]
but I would have expected to see an instant “batch” increase in pulsecount rather than the gradual increase seen when the pulsecount starts moving[/quote]
I re-wrote my logging code to keep track of the Count, the Increment, and the Delay. Sometimes the increase is gradual, sometimes a bunch and sometimes an instant batch. Makes no sense…
Date (CST) epoch (ms) count inc delay (sec)
Fri Nov 11 2016 15:06:06 GMT-0600 (CST) 1478898366222 1728 5 5.110
Fri Nov 11 2016 15:06:11 GMT-0600 (CST) 1478898371181 1736 8 4.959
Fri Nov 11 2016 15:06:16 GMT-0600 (CST) 1478898376015 1744 8 4.834
Fri Nov 11 2016 15:06:41 GMT-0600 (CST) 1478898401129 1754 10 25.110 diffTime = 25.110
Fri Nov 11 2016 15:06:46 GMT-0600 (CST) 1478898406155 1761 7 5.030
Fri Nov 11 2016 15:06:51 GMT-0600 (CST) 1478898411036 1764 3 4.881
Fri Nov 11 2016 15:07:01 GMT-0600 (CST) 1478898421155 1769 5 10.119
Fri Nov 11 2016 15:07:06 GMT-0600 (CST) 1478898426054 1773 4 4.898
Fri Nov 11 2016 15:07:16 GMT-0600 (CST) 1478898436184 1788 15 10.130
Fri Nov 11 2016 15:07:21 GMT-0600 (CST) 1478898441152 1790 2 4.968
Fri Nov 11 2016 15:07:26 GMT-0600 (CST) 1478898446144 1797 7 4.993
Fri Nov 11 2016 15:07:31 GMT-0600 (CST) 1478898451099 1798 1 4.954
Fri Nov 11 2016 15:10:01 GMT-0600 (CST) 1478898601142 1799 1 150.030 diffTime = 150.030
Fri Nov 11 2016 15:10:26 GMT-0600 (CST) 1478898626152 1802 3 25.023 diffTime = 25.023
Fri Nov 11 2016 15:10:31 GMT-0600 (CST) 1478898631165 1809 7 5.014
Fri Nov 11 2016 15:10:36 GMT-0600 (CST) 1478898636165 1810 1 5.000
Fri Nov 11 2016 15:10:41 GMT-0600 (CST) 1478898641143 1816 6 4.977
Fri Nov 11 2016 15:10:46 GMT-0600 (CST) 1478898646087 1820 4 4.944
@Jon, do you have any CT based power readings for the same period and circuit?
Looking at the increment values at each of the delays in your latest data, I now wonder if the issue is as bad as it looks, I’ve added some notes to the data, (see below, these are not conclusions, just observations for consideration)
The biggest “delay” is actually the least concern as it only returns an increment of 1 after the “quiet spell”, so it could be accurate.
What do you calculate your maximum pulse rate to be? and how does that compare to the possible rate set by the “pulse duration” in the sketch? Perhaps some of the quiet spells are high use that is pulsing too fast for the sketch and the “catch up” value on resume is the tail of that high consumption as it comes down to a readable level again???
Date (CST) epoch (ms) count inc delay (sec) missing updates
Fri Nov 11 2016 15:06:06 GMT-0600 (CST) 1478898366222 1728 5 5.11
Fri Nov 11 2016 15:06:11 GMT-0600 (CST) 1478898371181 1736 8 4.959
Fri Nov 11 2016 15:06:16 GMT-0600 (CST) 1478898376015 1744 8 4.834
Fri Nov 11 2016 15:06:41 GMT-0600 (CST) 1478898401129 1754 10 25.11 diffTime = 25.110 4
4 updates missing, possibly at the peak of consumption as it increased way beyond 8p/5s until it reduced close to 10p/5s, the next count is lower again too. OR did consumption drop so low and then pick up again at the end? A lot can change in 25secs at this level.
Fri Nov 11 2016 15:06:46 GMT-0600 (CST) 1478898406155 1761 7 5.03
Fri Nov 11 2016 15:06:51 GMT-0600 (CST) 1478898411036 1764 3 4.881
Fri Nov 11 2016 15:07:01 GMT-0600 (CST) 1478898421155 1769 5 10.119 ??? 1
Fri Nov 11 2016 15:07:06 GMT-0600 (CST) 1478898426054 1773 4 4.898
Fri Nov 11 2016 15:07:16 GMT-0600 (CST) 1478898436184 1788 15 10.13 ??? 1
Fri Nov 11 2016 15:07:21 GMT-0600 (CST) 1478898441152 1790 2 4.968
Fri Nov 11 2016 15:07:26 GMT-0600 (CST) 1478898446144 1797 7 4.993
Fri Nov 11 2016 15:07:31 GMT-0600 (CST) 1478898451099 1798 1 4.954
Fri Nov 11 2016 15:10:01 GMT-0600 (CST) 1478898601142 1799 1 150.03 diffTime = 150.030 29
increment of 1 pulse after 150secs might suggest this was a genuine low consumption period as the previous increment was only 1 pulse for 5secs too
Fri Nov 11 2016 15:10:26 GMT-0600 (CST) 1478898626152 1802 3 25.023 diffTime = 25.023 4
3 pulses over 25 deconds imediately after a long slow increment of 1 maybe the tail of the lowest consumption period with a burst of 3 pulses at the end just as the consumption increases to 7 pulses in the subsequent 5secs
Fri Nov 11 2016 15:10:31 GMT-0600 (CST) 1478898631165 1809 7 5.014
Fri Nov 11 2016 15:10:36 GMT-0600 (CST) 1478898636165 1810 1 5
Fri Nov 11 2016 15:10:41 GMT-0600 (CST) 1478898641143 1816 6 4.977
Fri Nov 11 2016 15:10:46 GMT-0600 (CST) 1478898646087 1820 4 4.944
Paul -
Yes, see files below. Keep in mind the power is averaging about 450 Watts over the one hour. See the image below.
2016-11-06 at 16-17-01 - emoncms 60 sec interval data.txt (1.8 KB)
2016-11-06 at 16-17-01 - emoncms 10 sec data.txt (10.5 KB)
CT Watts vs Opto Pulses:
My electricity meter is 1Wh per pulse.
During this time frame? A max pulse rate of 12 pulses per minute or 1 to 2 pulse per emonpi update.
The emonpi pulsecount value is updated every 5 seconds. So that would be 1 to 2 pulse increments per emonpi update (or per 5 seconds).
During any time frame? Summer would be the worst with Air Conditioning + appliances running at the same time. Maybe 6000 Watts for a short duration. So 100 pulses per minute or 8 to 9 pulses per emonpi update max.
I can’t figure out if this is a cockpit error (my error in calculations) or what…
At first glance those graphs do not seem to correlate.
Interesting that you say the power averages 450w for that hour as there were 571 pulses counted, which CT is grid?
I just pulled the power data out of the emonhub.log (using the 3rd power “power1pluspower2”) for that hour and found the average power across 3600secs was 439.8watts which is therefore 439.8Wh (0.4398 kWh) or 439 pulses if your meter pulses for each Wh. There were 571 pulses recorded across the same hour which is a 23% to 30% discrepancy.
I think you should do some tests to compare the actual meter readings. At the power levels you are working at I would expect the power factor to be pretty poor with a lot of standby and electrinic devices, perhaps the CT reading is out? or perhaps the way your meter measures or calculates the “sum accros the 2 legs” isn’t as straight forward as summing the 2 individual powers?
Paul - the emonhub.log file I sent starts at Sun Nov 06 2016 15:07:34 GMT-0600 (CST) or 1478466454796. See the red outline in the image below:
Could some of the extra pulses be carry over from that large delay?
Sun Nov 06 2016 15:44:20 GMT-0600 (CST) 1478468660325 diffTime = 2205.528
How did you do that? Did you add up all of these lines by hand:
2016-11-06 15:17:04,826 INFO RFM2Pi Publishing: emon/emonpi/power1pluspower2 529
I will do that!
The electric company installed a smart meter a few months ago and I downloaded the data from it. This is the emoncms data and it matches up (mostly) with the emonhub.log file above. The RFenergy_kWh is the smart meter and the data interval is every 5 minutes
emoncms data 4a.csv (2.0 KB)
Here are the totals for that emonhub.log duration (about 1 hour):
durat RFenergy_kWh ePi_P1_P2_kWh1 ePi_Opt_Pulse_kWh
3300 0.399 0.391 0.563
and here is the emoncms data for the past month:
emoncms data 7 (past month) 2.csv (10.8 KB)
duration RFenergy_kWh ePi_P1_P2_kWh1 ePi_Opt_Pulse_kWh
2505600 445.651 444.535 444.482
so over a longer duration it seems like my discrepancy drops to < 1%. does this seem correct?
I don’t think anything can be ruled out yet but the first 1600secs of that hour only counted 1 pulse, that would suggest the pulses are being delayed by half an hour or so, which doesn’t sound that likely.
No I dumped the data into excel from emonhub.log.
Is that an additional smart meter or is it the same meter you have your pulse counter attached to?
Do you have the raw data from the smart meter for that same period I looked at?
When you are questioning the data shown in emoncms you need to look at the source data, these prepared lists you provide just show numbers without any proof or working to verify their validity
Without verification of those numbers all I can say is “emoncms data 4a.csv” suggests the pulsecounting is over counting by 40% over that 3300secs and the CT looks pretty accurate, underreading by just 2%. But I have no idea of how those numbers came about and how faithful or typical they are.
The “emoncms data 7 (past month)2.csv” looks suspicious because the interval is all over the place, varying from 300 to 142200 secs in no particular pattern.
Yes, same meter the Opto Pulse sensor is connected to.
Dumb error on my part - I clicked this without thinking:
This is how I got started down this path - I was trying to validate what I was seeing and things didn’t make sense (like the long delays with no data).
Let me mull this over a bit. Even when looking at all of the data (Null values = Show) on the Graph screen things don’t look quite right. The interval is still “odd”.
Try not to use the graph page for debugging, that introduces it’s own errors. Firstly if you are using FI feeds, things get altered and then running queries via emoncms returns values nearest to the timestamp intervals derived from start time, end time and screen size and there is also a quirk that only uses the first selected feeds timestamps and not the other feeds timestamps for the CSV output.
See this post in Emoncms Graph module developments
I’ve spent a lot of time searching… could you explain how you got those feeds into emoncms? I assume you have some code in your emonpi_interrupt_pulse.ino. How did you implement the result? Is there a guide or fitting keywords i should look for?
Daniel - I think you’re looking for a more technical answer than I can supply. I’m still learning about the emon/OEM code.
I have an emonPi and I use the standard Nov 2016 image. So the Inputs and Feeds and pulse count software are all part of this build. I did not add anything extra to the emonpi_interrupt_pulse.ino code.
I am guessing you are not using the standard image. Is this correct?
The emonPi runs the emonPi firmware: emonpi/firmware at master · openenergymonitor/emonpi · GitHub
Pulse data along with CT and VRMS power readings are passed to the RasPi in JeeLabs RF packet format via tttyAMA0 GPIO serial port. EmonHub is then used to decode the data and post to MQTT and remote emoncms
MQTT input script then takes the MQTT data and posts to emoncms.
See emonPi technical overview: https://guide.openenergymonitor.org/technical/
Jon! Ah I thought ePi_opt_pulse_watts was a value calculated in the onPulse routine! Yeah, I also have the emonPi!
Glyn: Thanks! Not that I know exactly how to do all of that, but I now I know what to learn and what path to take! Great!
Daniel - The ePi_opt_pulse_watts was an experiment. It doesn’t seem correct to me and I probably constructed it wrong.
This is the process list for the pulsecount (ePi_opt_pulse_watts is line #4):
I have a 10.000 pulses per kwh meter and I think it seems pretty accurate actually. The only thing is that the rate of change-process only gives one decimal, multiplying it with 1000 (to not have the emoncms app round off the decimal and only give you whole numbers) you get 100w steps. I bet this way of doing it is way more precise than 100w up to the 12kw limit where the fuse has blown for sure. I had to recompile the firmware with a lower lowest time between pulses though (turned it down from 110ms to 10ms). The last process I dont know about. Perhaps its wrong, but I put it there to get rid of missed pulses giving a crazy value). It seems to work.
If I could have used the amp meters I would have done that though.
