Energy readings out of sync with power usage?

When would it not?

You demonstrated that in this picture - it’s where the gaps are, before you moved the emonBase:

If emonCMS didn’t record the RSSI, it wouldn’t have recorded the data either.

True, so the condition you’re trying to protect against is loss of RF signal, not a power cut?

Accumulating the energy values at source, and saving those accumulated values in the event of a power cut, protects against both conditions. Really, I’d have thought that should have been obvious, especially as you’ve complained earlier in the thread that the values were reset following a power failure and were (further?) out of step with your supplier’s meter.

I think the only case where it preserves information that can’t be reconstructed by the emoncms system is when there is a power cut whilst RF communications have been lost. And that only matters if the RF loss is for a significant period. So I think it’s a fairly rare case where it would be notably beneficial.

In the cases of my power cuts, I can reconstruct the true readings by postprocessing the files on the emoncms system; I haven’t lost much information. But I would also want to reprocess multiple feeds to get derived feeds that were all calculated on the same bases (a) import only (b) export only (c) net import, where exports are subtracted. That’s the only way to compare the various sources of information I have - monthly revenue meter readings, optical sensor readings and CT integration readings.

Not entirely true, when using emoncms to accumulate from a source that resets some data is lost, only a small amount, but none the less, it can be avoided by accumulating at source as Robert suggests. This is because (say) you have a reset when the counter is at 100, when the first value following a reset is transmitted of (say) 5, emoncms will spot 5 is less than the last received value, assume a reset and simply use the new value as a starting point to count from, eg when 2nd value sent of (say) 10, emoncms will calc the diff between the 5 and 10 = 5, when in actual fact the device has reported 10 units since reset. Had the last value been stored at source, it would have sent 105 and 110 (not 5 and 10) so emoncms would not have missed 5 units.

You wouldn’t, when a (revenue) meter is fitted, it doesn’t get set to the current reading of the old meter, that’s not how it works, any meter generally meters what passes through it, not what came before. If you wanted to specifically follow the actual revenue meter reading, you could use a emoncms process to add the now static offset, ie the reading when the new device was fitted/initialised (or reset to zero). But I would not recommend doing that as float arithmetic has a certain precision, dependent on the HW/OS/SW etc and in emoncms IIRC that precision is 7 places so if you start counting at a high number you could loss precision in the not too distant future, it’s better to just manually deduct the static “meter reading at install” from the actual meter reading whenever you read it to compare them.

Good to hear to have tried the emonhub.logs, for future ref with issues regarding missing payloads the best starting point is to use a search for “NEW FRAME : OK 8” (where 8 is your misbehaving node) to pull out all the lines where new “good” packets have arrived, check the timestamps to see if these are all at the correct intervals (eg 10s apart) any anomalies should point you in the right dirrection. eg missing packets will result in gaps of double the interval or more, you can check the log to see if there were any discarded packets around the time expected, and if so, was the first value “8” ? Was the RSSI ok? Was the rest of the payload similar to the “good” payloads from the same node? etc. Even a complete absence of frames around the time expected can reveal loads, like the source is dead, signal is seriously low etc

Well, clearly since you can describe that issue, you can code the solution to it :slight_smile: Don’t store zero when you receive 5, store 105, or +5, however the data is encoded.

Well I was speaking provocatively I admit, but I do hope that I haven’t just learned that emoncms stores digital readings in floating format, especially not single-precision float? Sorry, it’s late and I’m not going to check now, but that would be a great reason to resurrect a project I had started to rewrite the emoncms storage engines.

Comparing things is a whole lot easier when they are THE SAME, rather than offset by a constant or derived on a different basis or stored in an inappropriate storage system or whatever.

If only it were that simple, since the 100 value was saved in the devices dying breath it probably didn’t send it on. So the last known good value received might have been 95, or the device might have missed 5, 10, 15 units whilst offline resetting. Perhaps the device has only a small data type for this value and a massive surge of consumption caused it to roll over and there are no missing unit’s how could emoncms possibly know without lots of additional settings and checks in place? The fact is that the only place to reliably count what passes through a device/sensor is at source, beyond that it becomes “best guess” and yes ploughing a lot of effort and complexity into it might one day give slightly better results it can never be perfect, so the simple system in place is considered “good enough” for the purpose. If you want more accuracy or robust readings, do it at source because all emoncms can do is accumulate the difference between the current reading and the last reading, and if that is negative then it can only assume that to NOT to be a continuation of the same accumulating value and therefore starts counting again, from that unexpected value.

See https://learn.openenergymonitor.org/electricity-monitoring/timeseries/Fixed-interval

Sorry, I’m not sure why you started talking about float arithmetic and seven digits then?

Well that’s not true for all feed types I believe. And certainly needn’t be true for any new feed types. It’s like the PHPFina argument that it uses half the space of a time series that stores the time. That’s only true because PHPTimeSeries is very naive and stores interleaved time values and data values for every data stream. But almost all data streams come in groups from a Node so storing the time stamps for the Node separately instead would vastly reduce the space overhead.

I agree that the source device is in general a better place to preserve information, but the number of complex cases have persuaded me that the complexity is better placed at the centre in the case of simple, hardware-limited devices without battery backup. Obviously you disagree, I don’t think it will be very productive to explore much further.

He’s probably talking about this?

https://www.php.net/manual/en/function.pack.php

Ah, thanks. I hadn’t absorbed that bit.