Losing? or not displaying? data

Hi, a quick preface to let you know how I got to the current situation…
I had a power cut and the emonpi would not re-boot. Searching on these forums and the suggestion was that the card had been corrupted by said power cut.
I bought a new Industrial SD card from the shop and a surge protector from Amazon and installed both. System booted and all seemed fine.
However, I’m now losing data. There seems to be something that resets some arbitrary amout of data back to zero. I can’t see a pattern to either the time between ‘resets’ or the length of the ‘reset’.
Any help would be much appreciated.
If any more info is required (I’m sure it will be as I’m a newbie to this) please let me know and how to get it.
Many thanks
Garry

Welcome back.

Is it just an emonPi that you have?

Did you copy or try to copy anything from the corrupted SD card to the new one? Or did you rebuild everything?

What have you got in terms of Inputs, Feeds, etc?

Can you flesh that out a bit more. Do you see data appearing - where, on a dashboard or a graph - and then when you look later - what? Has it really reset to zero or is there just no record of it having been there? Is it one Feed, or all? If more than one, does the same thing happen at exactly the same time for all?

Many thanks for getting back to me @Robert.Wall

I have an EmonPi and an EmonTx. I use all 4 inputs on the EmonTx and only 1 on the EmonPi. Both have the voltage sensors.

I had a backup of the ‘old’ system on a NAS, so I used that to recover some of the feeds/inputs and then modified them as I changed some of the inputs around.

Just looked at the CSV output for the graph and it is showing NULL values, not zero, for ALL feeds sometimes, as follows…

2022-05-11 08:08:00, 319.4, 0, -94.2, -243, 8.2, 0
2022-05-11 08:08:10, 154.9, 11, -27.9, -145, -8.4, 0
2022-05-11 08:08:20, 0.0, 1, 10.5, -13, 1.4, 0
2022-05-11 08:08:30, 0.0, -9, 26.0, -14, 3.2, 0
2022-05-11 08:08:40, 0.0, 1, 26.0, -31, -2.1, 0
2022-05-11 08:08:50, 0.0, 6, 23.2, -46, -20.0, 0
2022-05-11 08:09:00, -27.1, -10, 80.6, -60, -6.1, 0
2022-05-11 08:09:10, 27.1, 28, -142.6, 122, 8.5, 0
2022-05-11 08:09:20, 0.0, -20, -28.5, 29, -1.8, 0
2022-05-11 08:09:30, null, null, 30.8, -51, -2.8, 0
2022-05-11 08:09:40, null, null, 28.9, -8, 4.4, 0
2022-05-11 08:09:50, 0.0, -6, 28.8, -28, -4.0, 0
2022-05-11 08:10:00, -213.9, -4, 267.0, -66, -7.8, 0
2022-05-11 08:10:10, 151.0, 0, -52.4, -88, 1.6, 0
2022-05-11 08:10:20, 62.9, 13, -175.6, 117, 2.9, 0
2022-05-11 08:10:30, null, null, null, null, null, null
2022-05-11 08:10:40, null, null, null, null, null, null
2022-05-11 08:10:50, null, null, null, null, null, null
2022-05-11 08:11:00, null, null, null, null, null, null
2022-05-11 08:11:10, null, null, null, null, null, null
2022-05-11 08:11:20, null, null, null, null, null, null
2022-05-11 08:11:30, null, null, null, null, null, null
2022-05-11 08:11:40, null, null, null, null, null, null
2022-05-11 08:11:50, null, null, null, null, null, null
2022-05-11 08:12:00, null, null, null, null, null, null
2022-05-11 08:12:10, null, null, null, null, null, null
2022-05-11 08:12:20, null, null, null, null, null, null
2022-05-11 08:12:30, null, null, null, null, null, null
2022-05-11 08:12:40, null, null, null, null, null, null
2022-05-11 08:12:50, null, null, null, null, null, null
2022-05-11 08:13:00, null, null, null, null, null, null
2022-05-11 08:13:10, null, null, null, null, null, null
2022-05-11 08:13:20, null, null, null, null, null, null
2022-05-11 08:13:30, null, null, null, null, null, null
2022-05-11 08:13:40, null, null, null, null, null, null
2022-05-11 08:13:50, null, null, null, null, null, null
2022-05-11 08:14:00, null, null, null, null, null, null
2022-05-11 08:14:10, null, null, null, null, null, null
2022-05-11 08:14:20, null, null, null, null, null, null
2022-05-11 08:14:30, null, null, null, null, null, null
2022-05-11 08:14:40, null, null, null, null, null, null
2022-05-11 08:14:50, null, null, null, null, null, null
2022-05-11 08:15:00, null, null, null, null, null, null
2022-05-11 08:15:10, null, null, null, null, null, null
2022-05-11 08:15:20, null, null, null, null, null, null
2022-05-11 08:15:30, null, null, null, null, null, null
2022-05-11 08:15:40, null, null, null, null, null, null
2022-05-11 08:15:50, null, null, null, null, null, null
2022-05-11 08:16:00, null, null, null, null, null, null
2022-05-11 08:16:10, null, null, null, null, null, null
2022-05-11 08:16:20, null, null, null, null, null, null
2022-05-11 08:16:30, null, null, null, null, null, null
2022-05-11 08:16:40, null, null, null, null, null, null
2022-05-11 08:16:50, null, null, null, null, null, null
2022-05-11 08:17:00, null, null, null, null, null, null
2022-05-11 08:17:10, null, null, null, null, null, null
2022-05-11 08:17:20, null, null, null, null, null, null
2022-05-11 08:17:30, null, null, null, null, null, null
2022-05-11 08:17:40, null, null, null, null, null, null
2022-05-11 08:17:50, null, null, null, null, null, null
2022-05-11 08:18:00, null, null, null, null, null, null
2022-05-11 08:18:10, null, null, null, null, null, null
2022-05-11 08:18:20, null, null, null, null, null, null
2022-05-11 08:18:30, null, null, null, null, null, null
2022-05-11 08:18:40, null, null, null, null, null, null
2022-05-11 08:18:50, null, null, null, null, null, null
2022-05-11 08:19:00, null, null, null, null, null, null
2022-05-11 08:19:10, null, null, null, null, null, null
2022-05-11 08:19:20, null, null, null, null, null, null
2022-05-11 08:19:30, null, null, null, null, null, null
2022-05-11 08:19:40, null, null, null, null, null, null
2022-05-11 08:19:50, null, null, null, null, null, null
2022-05-11 08:20:00, null, null, null, null, null, null
2022-05-11 08:20:10, null, null, null, null, null, null
2022-05-11 08:20:20, null, null, null, null, null, null
2022-05-11 08:20:30, null, null, null, null, null, null
2022-05-11 08:20:40, null, null, null, null, null, null
2022-05-11 08:20:50, null, null, null, null, null, null
2022-05-11 08:21:00, null, null, null, null, null, null
2022-05-11 08:21:10, null, null, null, null, null, null
2022-05-11 08:21:20, null, null, null, null, null, null
2022-05-11 08:21:30, null, null, null, null, null, null
2022-05-11 08:21:40, null, null, null, null, null, null
2022-05-11 08:21:50, null, null, null, null, null, null
2022-05-11 08:22:00, null, null, null, null, null, null
2022-05-11 08:22:10, null, null, null, null, null, null
2022-05-11 08:22:20, null, null, null, null, null, null
2022-05-11 08:22:30, null, null, null, null, null, null
2022-05-11 08:22:40, null, null, null, null, null, null
2022-05-11 08:22:50, null, null, null, null, null, null
2022-05-11 08:23:00, null, null, null, null, null, null
2022-05-11 08:23:10, null, null, null, null, null, null
2022-05-11 08:23:20, null, null, null, null, null, null
2022-05-11 08:23:30, null, null, null, null, null, null
2022-05-11 08:23:40, null, null, null, null, null, null
2022-05-11 08:23:50, null, null, null, null, null, null
2022-05-11 08:24:00, null, null, null, null, null, null
2022-05-11 08:24:10, null, null, null, null, null, null
2022-05-11 08:24:20, null, null, null, null, null, null
2022-05-11 08:24:30, null, null, null, null, null, null
2022-05-11 08:24:40, null, null, null, null, null, null
2022-05-11 08:24:50, null, null, null, null, null, null
2022-05-11 08:25:00, null, null, null, null, null, null
2022-05-11 08:25:10, null, null, null, null, null, null
2022-05-11 08:25:20, null, null, null, null, null, null
2022-05-11 08:25:30, null, null, null, null, null, null
2022-05-11 08:25:40, null, null, null, null, null, null
2022-05-11 08:25:50, null, null, null, null, null, null
2022-05-11 08:26:00, null, null, null, null, null, null
2022-05-11 08:26:10, null, null, null, null, null, null
2022-05-11 08:26:20, null, null, null, null, null, null
2022-05-11 08:26:30, null, null, null, null, null, null
2022-05-11 08:26:40, null, null, null, null, null, null
2022-05-11 08:26:50, null, null, null, null, null, null
2022-05-11 08:27:00, null, null, null, null, null, null
2022-05-11 08:27:10, null, null, null, null, null, null
2022-05-11 08:27:20, null, null, null, null, null, null
2022-05-11 08:27:30, null, null, null, null, null, null
2022-05-11 08:27:40, null, null, null, null, null, null
2022-05-11 08:27:50, null, null, null, null, null, null
2022-05-11 08:28:00, null, null, null, null, null, null
2022-05-11 08:28:10, null, null, null, null, null, null
2022-05-11 08:28:20, null, null, null, null, null, null
2022-05-11 08:28:30, null, null, null, null, null, null
2022-05-11 08:28:40, null, null, null, null, null, null
2022-05-11 08:28:50, null, null, null, null, null, null
2022-05-11 08:29:00, null, null, null, null, null, null
2022-05-11 08:29:10, null, null, null, null, null, null
2022-05-11 08:29:20, null, null, null, null, null, null
2022-05-11 08:29:30, null, null, null, null, null, null
2022-05-11 08:29:40, null, null, null, null, null, null
2022-05-11 08:29:50, null, null, null, null, null, null
2022-05-11 08:30:00, null, null, null, null, null, null
2022-05-11 08:30:10, null, null, null, null, null, null
2022-05-11 08:30:20, null, null, null, null, null, null
2022-05-11 08:30:30, null, null, null, null, null, null
2022-05-11 08:30:40, null, null, null, null, null, null
2022-05-11 08:30:50, null, null, null, null, null, null
2022-05-11 08:31:00, null, null, null, null, null, null
2022-05-11 08:31:10, null, null, null, null, null, null
2022-05-11 08:31:20, null, null, null, null, null, null
2022-05-11 08:31:30, null, null, null, null, null, null
2022-05-11 08:31:40, null, null, null, null, null, null
2022-05-11 08:31:50, null, null, null, null, null, null
2022-05-11 08:32:00, null, null, null, null, null, null
2022-05-11 08:32:10, null, null, null, null, null, null
2022-05-11 08:32:20, null, null, null, null, null, null
2022-05-11 08:32:30, null, null, null, null, null, null
2022-05-11 08:32:40, null, null, null, null, null, null
2022-05-11 08:32:50, null, null, null, null, null, null
2022-05-11 08:33:00, null, null, null, null, null, null
2022-05-11 08:33:10, null, null, null, null, null, null
2022-05-11 08:33:20, null, null, null, null, null, null
2022-05-11 08:33:30, null, null, null, null, null, null
2022-05-11 08:33:40, null, null, null, null, null, null
2022-05-11 08:33:50, null, null, null, null, null, null
2022-05-11 08:34:00, null, null, null, null, null, null
2022-05-11 08:34:10, null, null, null, null, null, null
2022-05-11 08:34:20, null, null, null, null, null, null
2022-05-11 08:34:30, null, null, null, null, null, null
2022-05-11 08:34:40, null, null, null, null, null, null
2022-05-11 08:34:50, null, null, null, null, null, null
2022-05-11 08:35:00, null, null, null, null, null, null
2022-05-11 08:35:10, null, null, null, null, null, null
2022-05-11 08:35:20, null, null, null, null, null, null
2022-05-11 08:35:30, null, null, null, null, null, null
2022-05-11 08:35:40, null, null, null, null, null, null
2022-05-11 08:35:50, null, null, null, null, null, null
2022-05-11 08:36:00, null, null, null, null, null, null
2022-05-11 08:36:10, null, null, null, null, null, null
2022-05-11 08:36:20, null, null, null, null, null, null
2022-05-11 08:36:30, null, null, null, null, null, null
2022-05-11 08:36:40, null, null, null, null, null, null
2022-05-11 08:36:50, null, null, null, null, null, null
2022-05-11 08:37:00, null, null, null, null, null, null
2022-05-11 08:37:10, null, null, null, null, null, null
2022-05-11 08:37:20, null, null, null, null, null, null
2022-05-11 08:37:30, null, null, null, null, null, null
2022-05-11 08:37:40, null, null, null, null, null, null
2022-05-11 08:37:50, null, null, null, null, null, null
2022-05-11 08:38:00, -75.5, 0, 63.4, 14, 5.9, 0
2022-05-11 08:38:10, 78.4, 0, -60.3, 8, 15.1, 0
2022-05-11 08:38:20, -40.7, 0, -0.3, 6, -27.9, 0
2022-05-11 08:38:30, -30.0, 0, 56.6, 0, 30.6, 0
2022-05-11 08:38:40, -94.9, 0, 54.9, 3, -36.0, 0
2022-05-11 08:38:50, -175.2, 0, 177.8, 8, 19.6, 0
2022-05-11 08:39:00, 19.4, 0, -12.9, 1, -7.5, 0

If I continually refresh the graph on, either on my dashboard or in the ‘graph’ section of ‘setup’, there are values, and then they will disappear and be replaced by NULL.

Please let me know if you need any other info.
Your help is much appreciated.
Garry

Disclaimer - I’m not an emonCMS expert.

A zero means that data was received, but the value was - zero. A null means no data was received prior to that time.

The problem is, the data you see on the graph is second-hand - it’s arrived in emonHub, it’s gone through the input processing, it’s been written to a feed, and then it’s being read back, selected by the graph and finally written to the CSV file. There’s a lot to go wrong.

It seems to me that it’s possible there’s a mixup between the database and the metadata that defines it (there are two separate files. I suspect the quickest and easiest way to sort this is to download a fresh image for your SD Card and then recreate the input processing and the feeds manually, rather than copying files. There’s a guide on how to do that here Update & Upgrade - Guide | OpenEnergyMonitor

The first step - before you do the reset - will be to look at the emonHub page and/or the emonhub log to check that the data is being received correctly and consistently. Without that, you’ll get nowhere.
Then move on to the Inputs page, and check that the data is still there. If that’s the case, then I think it’s time for a reset and start rebuilding the input processes by hand (rather than trying to copy or import the backup - because it’s that which I suspect has gone wrong).