I recently shutdown my custom emonpi and used win32diskimager to make a back on my pc.
After I replaced the SD card I ran a full upgrade, but it exited with errors relating to theme files I changed that it doesn’t want to overwrite. Fine, so I restarted again. All the feeds are working as expected, graphs update and all appears normal BUT after reboot, all data back to the back up point is lost, like it’s only in memory but not actually writing to the card or something. This happen at every reboot now. No new data is saved.
Have you put a new card in and put the old image back on the new card? You need to be more precise as to where you are at and how you got there.
As the old image is probably on an older version of Raspbian, the recommended update/upgrade path would be to use a new image and use the USB import method to copy all your data. You can then modify the themes as well (although creating a new theme might be a better route as it should not block and future updates).
Thanks for replying. This is the same SD card. I only removed it to make a copy and then put it back and tried an in place upgrade. But I don’t think the upgrade actually did much because the file compare test detected some theme files I customised and kicked it out.
Right now, the phpfina/*.dat files are growing and data is being displayed on the dashboard, but the mysql file date has not changed since I first removed the card. So it seems like the database isn’t storing the data. Does that chime with all recent data disappearing if I reboot? What is the data flow? Is it from the .dat files into the database?
I also did a back up using the backup module and downloaded the zip, so I have that too. My concern right now is that I’m losing data so want to figure out how to move forward.
Here’s how it looks right now. On the left of the graph, you can see that I took it down to make the backup image at 2pm yesterday. It was only down for about a hour and then appeared normal after I restarted about 3pm, with just one hour of missing data. But at the next reboot, the data added after the restart is missing from the graph. I repeated this several times in the evening, last rebooting a little after 11pm, which is where this graph picks up. Each time, all data back to 2pm disappears.
The feeds do look normal, so I’m hoping that data loss is minimal, but are you able to see what I mean about it appearing to not store new data?
My guess is that after the attempted upgrade, one of the services did not start correctly. Did you check the service state in the Admin page (if it had not updated it might not show up).
It be the feedwriter was the issue, so yes the data is not written to disk.
What does your Admin page look like (screenshot)? What version does it say you are on?
Here it is. Version is still low-write 10.1.0 but some modules did update. Like, I previously downgraded Backup to get that working but it’s up to date again now. Feedwriter looks fairly normal doesn’t it?
Hello @cmcg in the screenshot above, next to feedwriter, it states ‘63 feed points pending write’ I assume that does keep resetting to 0 every 5minutes or so? The fact that the feeds show that they are updated on disk and that I can see both the disk usage and write update time updated suggest that that is happening.
I havent seen this behaviour before, so makes me wonder if there has been some sort of disk corruption. It might be worth writing the latest SD card image to a fresh SD card and then importing your historic data using this approach: Update & Upgrade - Guide | OpenEnergyMonitor
I’m guessing the default ownership must have changed with a previous version upgrade. The five that have not changed since August align with five feeds I stopped using when I added my EmonTX.
An update to this. I’ve realised that the entire /home/pi/data partition is acting as tempfs. No changes to files in there remain after a reboot. The phpfina dat files give the impression that they are always growing, even through reboots, but I wonder if they just fill the time gap with zeros each time they start up again?
Could something have happened with an overlay file system to cause this?
I have a replacement sdcard arriving tomorrow so I’ll be able to take your advice then @TrystanLea
Does this look normal?
Appreciate this probably looks pretty normal.
Here’s an example where I just did “touch data/test.txt” and then rebooted. As well as test.txt disappearing, notice how wificheck.log also reverts to a version from just before this issue began.
pi@emonpi:~ $ ls -al data/
total 87
drwxrwxrwx 12 pi pi 1024 Apr 2 12:45 .
drwxr-xr-x 27 pi pi 4096 Oct 3 19:58 ..
-rw-r--r-- 1 root root 0 Oct 30 2018 dhcpd.leases
drwxrwxrwx 3 root root 1024 Oct 30 2018 @eaDir
-rw-r--r-- 1 root root 0 Jun 21 2019 emonbase
-rw-r--r-- 1 pi pi 8761 Jun 21 2019 emoncms-backup-2019-06-21.tar.gz
-rw-rw-r-- 1 pi www-data 0 Oct 30 2018 emoncms.conf
-rw-r--r-- 1 pi pi 1348 Jun 21 2019 emoncms-wifiscan.log
-rw-rw-rw- 1 pi www-data 8063 Oct 3 19:57 emonhub.conf
-rw-r--r-- 1 pi pi 40286 Oct 6 15:53 emonpiupdate.log
drwx------ 2 pi pi 12288 Aug 16 2018 lost+found
drwxrwxrwx 5 mysql mysql 1024 Mar 31 13:03 mysql
drwxrwxrwx 2 pi pi 1024 Mar 27 12:16 phpfina
drwxr-xr-x 2 www-data root 1024 Jun 21 2019 phpfina1
drwxr-xr-x 2 pi pi 1024 Jun 22 2019 phpfina2
drwxr-xr-x 2 www-data root 1024 Mar 27 12:17 phptimeseries
drwxrwxrwx 2 pi pi 1024 Jun 21 2019 System Volume Information
-rw-r--r-- 1 pi pi 0 Apr 2 12:45 test.txt
drwxrwxrwt 2 root root 1024 Oct 18 2018 @tmp
drwxr-xr-x 2 www-data root 1024 Oct 22 2018 uploads
-rw-r--r-- 1 root root 184 Apr 2 12:45 wificheck.log
pi@emonpi:~ $ ls -al data/
total 87
drwxrwxrwx 12 pi pi 1024 Mar 30 08:18 .
drwxr-xr-x 27 pi pi 4096 Oct 3 19:58 ..
-rw-r--r-- 1 root root 0 Oct 30 2018 dhcpd.leases
drwxrwxrwx 3 root root 1024 Oct 30 2018 @eaDir
-rw-r--r-- 1 root root 0 Jun 21 2019 emonbase
-rw-r--r-- 1 pi pi 8761 Jun 21 2019 emoncms-backup-2019-06-21.tar.gz
-rw-rw-r-- 1 pi www-data 0 Oct 30 2018 emoncms.conf
-rw-r--r-- 1 pi pi 1348 Jun 21 2019 emoncms-wifiscan.log
-rw-rw-rw- 1 pi www-data 8063 Oct 3 19:57 emonhub.conf
-rw-r--r-- 1 pi pi 40286 Oct 6 15:53 emonpiupdate.log
drwx------ 2 pi pi 12288 Aug 16 2018 lost+found
drwxrwxrwx 5 mysql mysql 1024 Mar 31 13:03 mysql
drwxrwxrwx 2 pi pi 1024 Mar 27 12:16 phpfina
drwxr-xr-x 2 www-data root 1024 Jun 21 2019 phpfina1
drwxr-xr-x 2 pi pi 1024 Jun 22 2019 phpfina2
drwxr-xr-x 2 www-data root 1024 Mar 27 12:17 phptimeseries
drwxrwxrwx 2 pi pi 1024 Jun 21 2019 System Volume Information
drwxrwxrwt 2 root root 1024 Oct 18 2018 @tmp
drwxr-xr-x 2 www-data root 1024 Oct 22 2018 uploads
-rw-r--r-- 1 root root 184 Mar 31 13:00 wificheck.log
