Community
OpenEnergyMonitor

Community

Manually edit feed data (to delete bad data) - Is it possible? How?

Tags: #<Tag:0x00007fc928249b88>

Hi - I’m running an emonpi install (ver low-write 10.1.11) collecting water meter pulses. One of my sensors went crazy and sent thousands of extra counts. I’ve got the same question posted in 2018 at https://community.openenergymonitor.org/t/manually-edit-feed-data-to-delete-bad-data-what-tools-to-use/7668:
“I now have points in my data that are many orders of magnitude higher than what they ought to be, and it’s completely trashing my views (because of the zoom) and causing havoc with my historical data. Is there any way I can go in and scrub the insanely high values out of the database?”
Specifically, are there are tools to edit phpfina data? Any pointers to get started will be greatly appreciated.

It used to be possible, but I don’t have a running emonPi with the latest version at present, so I’m not able to confirm that it’s still there.

However, I did a search and I’ve found this post from 15 months ago, which does show (roughly) where it is:

Excellent tip - thank you!
I found “EditRealTime” in the list of Visualizations in the Setup (I’m running emonPi low-write 10.1.11). I chose my feed that had bad data and zoomed in the time period with bad values. I could not edit single values using the “Edit_feed @ time” option, but after carefully zooming so just the bad data showed in the window, I used the “Multiply data in the wind by …” option and entered NAN then pressed Save. Instantly my bad data was gone! Just be careful - there is no confirmation after pressing Save and apparently no undo.

Hopefully (but maybe not) a minor point: If when viewing the data, you choose a scale and a window such that the graph wants to pick one of those “NAN” points to plot, it won’t have any data to display, and I suspect it will treat it like a NULL value - it’s going to either show as missing data, or draw a straight line between the values it’s using either side, even if there are valid values between the two it chose.

The issue here is that currently, the edit does not actually select valid data points. You can use the graphs functionality to identify the actual data point timestamp, then the edit function will work.