Import historical data from iotawatt

This afternoon i encountered a fatal crash of my emonpi. The sd card is completely dead (does not even show up on another pc).
So i had to reinstall emoncms and us a backup. However the backup is almost 1 year old.
Since i also have some Iotawatt that are feeding the emonpi the data, i was wondering if i could insert the missing data?

I tried to find more info on the forum, but i could not find how to do it when there is already new data (last hour) added to the feed.
Can anyone show me where to look or how to create a script?

If you’re using the Iotawatt uploader that will upload historical data.
Go to /Setup/Data Uploaders/Emoncms and put the date you want history uploaded from in the “upload history from” box.

For some of my units i’m using the uploader, but i couldn’t get it to work.
Can it be that it does not work when there is already newer data available?
I have data up to 17/04/2021 and some new data from after the rebuild.

I’m not an emonCMS expert, but I believe that is indeed the case. I don’t know if or how you can save the existing data, empty the feed, upload the old data and then continue and upload the saved data.

You might like to look in the ‘Learn’ section where there is a description of the databases used by emonCMS.

Thanks for your reply. I was able to get another pi running and update the data.
But for 1 iota it didn’t work.
@glyn.hudson : would there be a way to delete the recent data (last 5 min or so) for a feed so i can retry
edit1:
it seems to be somehow different:
the lastest data in the feeds is from 17/04/2021.
i ask iotawatt to update from 18/04/2021. the status page on the iotawatt show that it tries to update on 17/04/2021 but instantly jumps to 20/05/2022
but with this one i get an error in the log “padding max block size exeeded”. This did not occur with the other 3 iotawatt i synced.

edit2: got it working after a reboot! but the question remains :wink:

  • can you delete a part of recent values?
  • is there a way to put new values between already present values (ex if a month is missing)

What feed engines are you using to save the data? You may be able to use Vis > Edit realtime for some of this. It’s a tool that needs a bit of work, but may do what you need.

I’m using the php fina engine (default one).
I looked into the “viz → edit realtime” and it seems that i could use it to change the ‘null’ value to a correct one.
Is there a way to automate this (api)? (because one-by-one for 1 month @10sec for 30feeds is a bit much :slight_smile:

Yes you can just post values to be amended with:

/feed/post.json?id=FEEDID&time=TIMESTAMP&value=VALUE
1 Like