I have feeds going back 3 months. At the beginning I have some bad data that I want to remove. Sadly I did not use new feeds once I had things working.
So I have PPFINA and PHPTIMESERIES feeds with data I would like to delete. All data before a particular date.
The delete feed data option in the ‘edit real time’ visualisation does not work in edge or firefox - I have never got it to work.
I have found no clear API guidance on deleting feed data before a date or over a date range! Does anyone have solutions tested as working on emoncms.org?
Sadly the promise of deletion from the edit real-time visualisation does not actually work. Older posts suggest that this is being experenced by others. You can ask it to delete all of the data in view, it asks for confirmation, it does not work on emoncms.org.
My feeling is an API call of some sort is the way forward.
Which suggest json action ‘deletedatarange’ is available only for the MYSQL engine and not the PHPTIMESERIES or PHPFINA engines. I have also read that the PHPTIMESERIES uses the same format as MYSQL but does not support use of the delete bit, so only data amend is possible.
So delete appears not to be supported for a PHPTIMESERIES or PHPFINA feed and this is probably the case for emoncms.org.
Perhaps @TrystanLea can advise on what can be done to remove or clear (NaN) old data on emoncms.org, or a robust way to generate a new feed from the truncated original feed data?
Also, if the feed type does not support deletedatarange then can I suggest the editrealtime visualisaton does not offer to delete a range of data?
The management of historic data is not really supported.
GUI editrealtime works for individual points but is unrealistic for all but small numbers of changes. It is also prone to error with the update value applied as the current value rather than at the entered timestamp. So edits risk causing problems by overwrite if the current value! Seen many times. It needs fixing.
Editrealtime 'visualization features ‘delete all in window’s and ’ multiply all’ features do not work.
It appears to me that if you have any sizeable data issues in emoncms.org then you just have to live with it or throw the data away and start a fresh feed. Ouch.
Obviously emoncms.Org cannot do it all, it is after all a contended service. Poor data handling is perhaps inevitable as large amounts of data manipulation for corrections etc. demands lots of limited resources.
There may be a painful API way to NaN or 0 data. I have a feeling limits on activity rates in emoncms.org will prevent this.
My conclusion is that it is what it is. It is currently useful with limitations.
I think it is now possible to delete data older than a certain age, by trimming a feed. Go to Feeds, select the feed and click on the Delete button, then choose the timestamp that you want to trim before and click “Trim Data”:
