Manually edit feed data (to delete bad data) - Is it possible? How?

sroof · 2019-11-16T19:33:14.794Z

Hi - I’m running an emonpi install (ver low-write 10.1.11) collecting water meter pulses. One of my sensors went crazy and sent thousands of extra counts. I’ve got the same question posted in 2018 at https://community.openenergymonitor.org/t/manually-edit-feed-data-to-delete-bad-data-what-tools-to-use/7668:
“I now have points in my data that are many orders of magnitude higher than what they ought to be, and it’s completely trashing my views (because of the zoom) and causing havoc with my historical data. Is there any way I can go in and scrub the insanely high values out of the database?”
Specifically, are there are tools to edit phpfina data? Any pointers to get started will be greatly appreciated.

Robert.Wall · 2019-11-25T23:21:00.056Z

It used to be possible, but I don’t have a running emonPi with the latest version at present, so I’m not able to confirm that it’s still there.

However, I did a search and I’ve found this post from 15 months ago, which does show (roughly) where it is:

Unearthed a quirk with editrealtimedata timestamps Emoncms

Today I spotted a freak value in some temperature data and decided to try and correct the one bad datapoint using vis/editrealdata. This way of editing values has always been a bit temperamental and I may have stumbled across some useful info whilst making my correction. For this particular project I am recording csv locally and also sending to a self-hosted emoncms. It was in emoncms that I noticed that one of 34 temp sensors momentarily read -2043.125°C, thinking that was a tad chilly for thi…

sroof · 2019-11-27T23:41:54.461Z

Excellent tip - thank you!
I found “EditRealTime” in the list of Visualizations in the Setup (I’m running emonPi low-write 10.1.11). I chose my feed that had bad data and zoomed in the time period with bad values. I could not edit single values using the “Edit_feed @ time” option, but after carefully zooming so just the bad data showed in the window, I used the “Multiply data in the wind by …” option and entered NAN then pressed Save. Instantly my bad data was gone! Just be careful - there is no confirmation after pressing Save and apparently no undo.

Robert.Wall · 2019-11-27T23:56:22.828Z

Hopefully (but maybe not) a minor point: If when viewing the data, you choose a scale and a window such that the graph wants to pick one of those “NAN” points to plot, it won’t have any data to display, and I suspect it will treat it like a NULL value - it’s going to either show as missing data, or draw a straight line between the values it’s using either side, even if there are valid values between the two it chose.

borpin · 2019-11-28T07:51:39.900Z

sroof:

I could not edit single values using the “Edit_feed @ time” option

The issue here is that currently, the edit does not actually select valid data points. You can use the graphs functionality to identify the actual data point timestamp, then the edit function will work.