5 hours of no updating of Feeds

The feeds one of which is 202107 were being updated on emoncms.org fine until around 4 or 5 hours ago. From my end the scripts appear to be working and even a by hand command line feed update doesn’t seem to work now. It is the same command from my command line history that worked previously.
Regards
-Jim

1 Like

I have the same problem. The server accepts the data but the inputs don’t refresh. It’s going on for 6 hours now.

@TrystanLea, @glyn.hudson or @Gwil are you aware of any emoncms.org issues today?

Yes, there is an issue with one of the queues - @TrystanLea is currently looking into it.

Cheers @Gwil

I noticed that the feeds have not stopped they just seem to be taking 5 hours to show up in my emon interface.

Perhaps this is some sort of server or SQL time issue?

Probably a lot of requests were stuck in a queue and now there is a 5 hour backlog to process but the processing does not seem to be fast enough to reduce the backlog so now there is a 5 hour constant delay.

Interestingly I’ve looked at two other (and older) installations and they are still working fine

The OEM hosted emoncms is operating across multiple linked servers to handle the traffic, so it’s totally possible for some users to be effected but not others, When I checked my account earlier it was ok too, I didn’t post that as it wasn’t helpful to you guys.

I experienced same problem today. My feeds were also 5 hours delayed but it is now 4 hours late. The graphs are correct and the same as the local graphs but just delayed. On the emonhub the data send to https://emoncms.org is getting an OK. Anything that I can check or do?

Hi, I,m the same, says last update 4 hours ago, Emonhub also says data sent OK.

Down to 2 hours now, waiting in anticipation :grin:

Now its below 1 hour.

Hi there.
I am still having problems too, with 53 min last updated.
Yesterday MQTT wasn’t able to connect.
Local connection was working always fine.
I did a full reset to my emonpy (didn’t mind about the data) but results were the same.

Apologies for the long queue, the issue seemed to have been caused by an IotaWatt uploading a couple of months of history yesterday - it should really be able to handle it so I will need to look into that once the queue clears.

The queue should be fully clear in just over 12 hours from now.

The only problem was the issue was making me think my emonpi was having some problem until i check this thread. Otherwise, everything is fine since now i know the problem.
I wonder, maybe emoncms.org need some solution to prevent a single upload to cause this big slowdown. Imagine more of this happens.
Best regards

I was aware of one IoTaWatt user that was affected by the backlog, but that was less than a day backup and seemed to be more of a result than a cause.

Nevertheless, it’s to be expected that one might upload several months, or even a years worth of data. I’d like to know more about what it is that triggers the problem. Whether it’s simply volume or has to do with specific input processes employed by this user.

Right now, there isn’t any feedback system to indicate any problem. In fact, I had recently posted an issue where EmonCMS apparently responded OK to a POST message, then couldn’t parse the data and discard it.

So we should investigate some method of validating successful posting, both to verify that it is done, and possibly as a mechanism to throttle large upload streams.

I still do not see changes I made to the Emonbase Inputs and Feeds show up in EmonCMS Inputs and Feeds. I am new at this so I may have done something wrong, but it approaching 2 days since I made the changes and the Updated column is reporting seconds rather than hours like it was.

Has the database worked through its backlog or have I done something incorrect in my setup?

The other thread is
here

thanks
rearden

Hi Rearden,
So, you have an EmonBase, that’s configured to send the data to 2 Emoncms instances - one local (either within the Pi that’s running the data acquisition or separate), and one on emoncms.org. And the emonHuib piece seems to be working; the data values are indeed being sent to both the local & .org instances - correct?
I think the issue might be expectations! These 2 Emoncms instances are independent - they get fed the same data, but they need to be configured separately.
Thanks,
Sandy

It would be useful if someone could confirm the backlog is now clear and normal service has resumed, but in the absence of that confirmation it sounds like it has from the fact the update column values you are seeing are now green and just seconds, previously you were seeing orange values of 4hrs. In which case, yes further debugging should return to your original thread, I had only pointed you here as it was pointless trying to debug your input processing using input data that was 4hrs old.