Community
OpenEnergyMonitor

OpenEnergyMonitor Community

Delayed data on emoncms.org?

Been scratching my head on this for a while now and discovered it must be at the emoncms.org server as when I send duplicate data to another server it has the correct times.

Below shows nodes 29 and 30 being sent to 2 emoncms instances with timestamps. node 29 at emoncms.org has stopped updating or is running very slow.

One for @TrystanLea I suspect?

Seems to be catching up now!

It will probably have sorted itself before Trystan see’s this

Interesting! The queues all look fine over the last couple of days. If one node is ahead and the other on time within a single account that suggests an issue outside of emoncms.org as the emoncms queues are divided in blocks of users, so any slow down should affect all inputs in a given account. Has it all caught up now?

Yes, thanks, it all caught up later that evening. Node 29 was something I was testing and not running 24/7 so I can’t say whether it would have remained that way or not. But as odd as it might seem, the node 29 and node 30 frames were delivered to emoncms.org via the same single bulk upload with timestamps within a couple of seconds of each other. That’s how I came to conclude it was at the server end, I was watching the emonhub.logs and could see the frames arrive, get processed with similar timestamps and dispatched as 2 identical bulk upload requests to 2 emoncms targets and got “ok” replies from each.

The emoncms.org target showing this delay whilst the other showing correctly.

If not the queuing, the only other thing I could speculate towards might be IF during my tests I might of accidentally sent a frame with an incorrect timestamp in the future, perhaps the different servers handled the subsequent inputs differently??? I’ve no evidence or suspicion of using incorrect timestamps, but I did change from untimestamped to timestamped data at some point in writing the script. I had also checked the clocks on all the devices and found them ok during debugging this.

The other server is running older v9.??? emoncms without any queue management.

It isn’t a problem for me, but the delayed input times displayed at emoncms.org were definitely incorrect for node 29 whilst correct for 30. node 29 was being updated alongside 30 so the update times should have been the same (within a couple of seconds).

In fact, the fact it slowly caught up would suggest a queuing issue too, if the timestamps were to blame it would correct instantly, either immediately the correction was made or after passing the future timestamp if the inputs do now function like a low-write phpfina feeds in some way.

Oh well, not a problem for me, so don’t spend any time on it.

1 Like

Thanks @pb66 interesting I can see how youd think it points to emoncms.org, perhaps there’s something Im missing, I will keep an eye out.

1 Like

Hi @TrystanLea this has happened again

Identical data posted via http bulk api from the same emonhub source to 2 emoncms instances and emoncms.org is showing as last updated 2 hrs ago.

This time I can confirm that the clock was out on one of the Pi’s and there most likely was a period were the timestamp supplied was wrong. However to be 2 hrs out after correction, I would have expected to see the inputs page showing “last updated” minus 2 hours ago prior to correction? And that certainly didn’t happen.

It would seem there is something funky going on when submitted timestamps change. In this instance “sentat” would have been 2 hours behind so (old) emoncms would have added 2hrs to all the timestamps, putting them in the future. But that doesn’t explain why now that all timestamps are correct, emoncms shows last updated 2 hrs ago unless last updated is based on the difference between supplied timestamps.

The data values are static at the values presented with “2hr slow” timestamps and I suspect it will remain that way for 2 hour until the new data timestamps pass the stuck timestamp.

This is a very old emonhub setup that still uses “sentat=” in the bulk api, I now always use “time=0” so that’s probably why I haven’t seen this before. But this error doesn’t occur on emoncms version 10.1.6-beta (without input queuing), it’s just a redundant test server that happens to still be live, it’s quite by chance that we have this to compare against.

This bug really doesn’t bother me, but I thought you should be aware and I can perhaps recreate to test if needed.