I’ve got some dipoles on order; I’ll let you all know how they work after they arrive.
In the meantime I’m curious about how the optical pulse sensor data is interpreted. There’s a glitch in the graph of my solar generation; perhaps the easiest way to show it is wih the CSV data:
There isn’t anything obviously wrong with this data (to us at least) it is increasing only and the values seem believable (without any context to consider).
You have posted a link to a graph on your local emonpi (http://emonpi/emoncms/graph/3) which obviously we cannot view. I these instances we ask you post a screenshot rather than a (working) link because as time passes the link to live data would change and the context of the discussion is lost.
Can you specify what is wrong with the current value, ie what it is and what you think it should be and why?
Me too, I do not think a whaccumulator would fallback like that, it would usually drop to zero or spike when misbehaving and the fact it springs back to counter the missing 200 is too coincidental to believe without confirmation, I think the 12028 should have been 12228. Perhaps @djh can confirm.
I’m sorry but I think that 12028.0 is less than 12213.0?
And no, I didn’t copy the values wrong! It’s difficult to do that with copy-and-paste
I just posted what emoncms displays, to try to be sure I wasn’t mistaken when I said ‘whacc’. I am assuming that 10464.00 is the current value. If not, I’d be happy if you could explain what that number actually is?
Little more info. Firstly the heading above the 10464.00 is ‘Latest’ and there’s a tooltip that says ‘Last recorded 2 value’. The number is still the same despite it being a day later.
Second I copied the data from /home/pi/data/phpfina/ and I wrote a perl program to read the .dat files. In 3.dat I can see the 12028.00 value sitting there:
So it’s definitely emon-whatever doing this, not my imagination!
Now the graph isn’t showing me the same data today, it’s obviously resampling or somesuch. Where is the explanation of how the graphs work? A link on the page would be useful.
The energy feed is calculated from the CT sensor. My dump of the actual data files confirms that it’s a single bad value timed at 14:26:00 in the optical stream.
Is this a data collection error (bug) in the emonTx or a data recording error (bug) in the emonBase? Or is there some other possible explanation?
I gather from some googling that there is no user documentation for the graph app, so I’m still in the dark as to whether I’m using it right. Why are the colours shown as black in the ‘feeds in view’ for example?
I’ve got other oddities in the data that I’ll post later.
This one concerns the electricity imported from the mains, measured by the other emonTx in my garage. There’s the optical sensor data, the raw power readings from the CT on the meter tail, and the integrated energy readings computed from the power. The readings have been scaled by different factors to make the graph show the problem.
The CT has a single missing data point at the critical time; the raw data in 4.dat confirms that:
2018-12-10 03:41:40 2160.000000
2018-12-10 03:41:50 2163.000000
2018-12-10 03:42:00 NaN
2018-12-10 03:42:10 2167.000000
2018-12-10 03:42:20 2184.000000
The computed energy has a single retrograde value at the same time and apparently catches up afterwards:
2018-12-10 03:41:40 113.359528
2018-12-10 03:41:50 113.365540
2018-12-10 03:42:00 112.734421
2018-12-10 03:42:10 113.377548
2018-12-10 03:42:20 113.389030
I don’t know what I expect the energy integration to do when there’s missing power data, but pretending that there was a sudden switch to energy outflow is definitely not it!
The optical sensor also has a retrograde measurement at the same time:
2018-12-10 03:41:40 115049.000000
2018-12-10 03:41:50 115055.000000
2018-12-10 03:42:00 114418.500000
2018-12-10 03:42:10 115067.000000
2018-12-10 03:42:20 115079.000000
This is very strange. How does a whacc feed produce retrograde data? And no, my meter doesn’t record exported power! And why does it occur at this precise time? What’s special about -636.5, approximately 100 times the magnitude of the actual import at that time?
I believe the problem occurs as a result of NaN data, but I haven’t got to the bottom of it yet. (and apologies but the lack of comments in the code or overview documentation is one of the main things in slowing me down diagnosing the problem, IMHO)
Thanks for noticing this thread and posting the cross-link. We do indeed seem to be experiencing the same problem. It’s clearly a bug that’s been around for a while but doesn’t seem to have been cleared. I raised a bug about it: incorrect data recorded in feeds · Issue #1137 · emoncms/emoncms · GitHub but it doesn’t seem to be attracting any attention.
I read the thread about your experiences and whilst a lot of it is still over my head I did notice some points that seem interesting:
(1) this problem seems to occur with the phpfina engine but not with the timeseries engine.
(2) I’ve also seen spikes in power_to_kwh data but they’re slightly different, in that they first go down when there’s null data underlying, then spike upwards by the same height above as the initial spike below, and then return to a sensible value. In each case, the underlying trend of the data is also accounted for.
(3) I haven’t got a clue what determines the size of the spikes, but I suspect it will prove to be significant. Here’s an interesting example with two spikes, one being three nulls wide:
(4) In your thread a year ago there was a suggestion that there is a better technique for recording the data (optical pulse counters on revenue meters, in my case) instead of wh_accumulator, but I chose wh_accumulator precisely because it is the technique recommended in the current documentation. So what’s the situation here?
I had an issue with nulls, you can test in the process list and skip the log to feed if the value is null.
A null means that, for whatever reason, the input value was not sent correctly. Unfortunately, by default, Emoncms logs nulls to feed. Seems odd, but that is what it does.
The nulls are not the problem. The whole point of phpfina, as I understand it is to minimise writes and increase efficiency. It does that by writing nulls, which are essential for it to keep the time stamps accurate. So nulls are to be be expected in the underlying data.
The problem is what wh_accumulator (in conjunction with the phpfina engine) does with those nulls.
I expect in my case the nulls are due to some co-channel interference from some device or other.
You mean I can fudge the results and sweep the bug under the carpet?
Shame on you! I’d rather try to see it fixed.
How to do that is part of the puzzle that goes over my head. Plus I’m doing more than enough fiddling with my system as it is and getting myself into a muddle.
