Community
OpenEnergyMonitor

Community

Calculation of feeds from different IoTaWatt

I have 4 (homemade) IotaWatt running, covering 45 CT’s (a 3phase setup).
Some feeds require input from different IoTaWatt units.
I was wondering how the feeds are calculated: are they updated when additional data with the same timestamp arrives?
And if so, how specific does this timestamp need to be (exact to the second?)

Cross device input processing can be tricky in emoncms and getting it right depends on several different elements.

Virtual feeds may be useful here as they can be quite forgiving and accommodating when it comes to merging data from different sources as they retrospectively calculate the values from the latest data at the time of query, whereas conventional input processing will use the data available at the time when the input that contains the processlist is updated and persists the result, so any changes to any value after that point in time is effectively ignored as it is “too late”.

I do not know the specifics of the FW you are running nor of the standard IoTawatt FW, but IIRC the IoTaWatt uses the bulk upload api. This can create difficulties if the data is buffered and therefore not allowed to arrive at emoncms in absolute chronological order. For example if the update intervals are 10s, but data is sent to emoncms every 30s, when the inputs are updated with 3 sets of data, the chances are that some of the other inputs could either be up to 3x 10s intervals behind or ahead at that time depending where in the update order that device is.

IIRC the IoTaWatt has a RTC but I do not know if that is used to syncronise the sampling or sending of data across devices, ie does the IoTaWatt sample and send every n secs from start up or are the sample and sends aligned to certain times eg is “every 60s” at the top of every minute or just 60s after power up and every 60s after that?

What types of feeds are you using? ie are they fixed interval?
What type of emoncms instance are you sending to? ie is it using feedwriter?

The timestamps for all data arriving at emoncms are truncated to whole seconds eg 1564350615.999 will be truncated to 1564350615, not rounded up!

When using a “low-write” emoncms (feedwriter service) feeds will not be “edited” that is the first data written cannot change where as the standard emoncms can update (overwrite) data with the same timestamp.

The important thing to bear in mind is that the input processing is “request driven” there is no timing or regular loops etc, if device A’s data always arrives just before device B’s data and you want to sum A+B, put the processing in input B so that when device B updates input B and sums A+B, input A will be the latest value. If you were to have the processing in input A instead, then when device A updates input A, input B’s value is stale as it is not yet updated. However! if the order changes due to a network outage, a slightly different send interval time or power interruption etc etc etc it will upset things.

A couple of related threads to read

and there are many others on this subject.

Virtual feeds do take much of the hard work out of cross-device processing but they can be slower to graph data and populate dials and widgets etc since they calculate on the fly rather than using pre-written static data. Plus they do not work with the app module (without hacking).

You might also get further/better advice from the IoTaWatt forum since how and when the IoTaWatts post their data will play a large part in determining how easy and reliable your input processing might be.

The exact timestamps will play a much smaller part in this than you think, the biggest hurdle is the order of arrival rather than the timestamps, it doesn’t matter much if data is a couple of seconds out, but if you are summing “old” and “new” input values interchangeably, that can cause havoc.