I just noticed two gaps in my emonCMS graphs today. One was a minute around 12:30. And then a few minute later a gap of about eight hours… at which point I rebooted the emonBase which allowed new data to arrive in emonCMS graphs again.
I think the gaps are a side effect of going apt-gets about the time that the gaps started. Is this not allowed? Do we have an alternative to apt-get? Or a special apt-get process we use on our emonBase machines?
I was really counting on emonCMS to be reliable. Is there a process in place to recover from gaps like this?
In my case, much, but not all, of my data is coming from an IotaWatt which still has a copy of the data. I’ve tried to tell the IotaWatt to resend the data starting the day before the gap. I don’t know of any way to confirm that it has done that, but as far as I can tell, the emsCMS is not filling the gaps even after 30 minutes. Do you have any other suggestions?
Finally… it seems like gaps in emonCMS data is fairly common. I have found myself posting questions like this several times in the last few years. The apparent need for me to monitor and baby my emonCMS machine is the reason I stopped using the open energy monitor solution a few months ago. I restarted using it again recently to try the IotaWatt, which so far seems pretty reliable. But alas, I’m still being bit by emonCMS problems which undermines my IotaWatt approach to some degree. Is there a fairly fool proof approach I can use to get most of the benefits of everything offered by the oem team? (I’m not exactly a fool, but I feel like I do not have time to drill deep to deal with unreliable systems.)
Running apt-gets should be allowed and its how most of the software is installed and updated on the image. But I guess it does depend on what your installing/updating? E.g if it caused one of the services: Apache, redis, mosquito that either the emonhub, mqtt_input or feedwriter services depend on to become unavailable that could cause an issue.
If this happens again, could you copy the content of Server information here prior to rebooting?
I do not see that - it does depend to some extent on what else the machine it is running on is being asked to do. If I leave it alone, I find it runs just fine.
If you are installing other packages / software, you do risk upsetting some part of the EmonCMS system. It is always worth rebooting after any install just to reset where everything is at but also be aware of any side effects your actions may be having.
+1 from me on this… I’ve got many months of data without any gaps. The only time I get a gap is when I update anything (using apt-get, or the update button from EmonCMS) and I make sure to reboot the Pi after I’m done and make sure everything has started up correctly again. Any such gap is usually no more than a few minutes.
FWIW… On previous instances, I had a SSD. On this one I don’t install anything extra except emacs. What might be somewhat unusual is I have a LOT of TX’s (4) and TH’s. (Only two TH’s are active now but once the system seems stable I’ll put new batteries in the rest.) Also I think the TX’s sample every 5 seconds.
I’ll make a note to myself to do as you’ve suggested.
Thanks Brian and Greebo. It’s good, but frustrating, to know that it’s probably just me. Of note, I might also be biased by my experience with a prior installation that was spontaneously losing feeds about once a week. I’ve recently abandoned that and started with a freshly downloaded microSD card.
For the brief gap you all have during the reboot, do you make any effort to refill that gap? I suppose I could update only at night with no solar and most circuits turned off, so that my kwh figures remain fairly accurate, but I’m wondering what you do to keep you kwh numbers accurate.
In the scheme of things, its such a small “error” in the numbers that I’m not bothered by it.
I’ll try and put it into perspective:
Lets say you update once a month and it takes 5 minutes. That’s 1/12th of an hour, so lets assume you’re using 500 W of power continuously during that update time, you’ll have failed to account for 41 Wh (500/12) of energy, or 0.041 kWh. Assuming you wait until your power usage is lower (say 150 W), then you’ve instead failed to account for 0.0125 kWh (150/12) of energy in your 5 minutes.
For me using anything from 25 - 40 kWh/day depending on the weather (all electric house, no solar), 0.04kWh isn’t even worth thinking about - we’re talking about a roughly 0.1% error per day, or a 0.003333% error if I update monthly.
And to put it into perspective even more pointedly, take a look at para. 3.2 of Electricity Meters – Disputed Meter Accuracy Report. OK, this is UK practice, but it gives you an idea of the tolerances that are “acceptable” - at least to the supplier.
Thanks Robert. Verifying the accuracy of my solar and utility numbers is a motivator for my desire to be accurate, so your link is particularly interesting.
That makes sense. Nevertheless I’m still looking for a good way to recover from gaps though given that I just experienced an 8hr gap. I’d like to fix that. And I will try to be more careful with my next apt-get, but I am fallible and might not get it right every time.
I’ve read that that can be done via the API, but I’m also seeing that the IotaWatt, which must be using that API, didn’t seem to fill the gap.
You can only fill the gap if the data was collected.
If the error that caused the gap was in your local EmonCMS, but you somehow managed to push data into emoncms.org, then you can probably back-fill that data from emoncms.org, although I don’t use emoncms.org so I don’t know how that would be done precisely.
However, if the problem was caused by, say, emonhub being down, and no data got stored anywhere, then that gap can only be filled by your best guess at reality
My IotaWatt has the most important data without a gap. Might you know how to successfully use it to fill the gap in the local emonCMS graphs? Might it have already succeeded and I just need to nudge the emonCMS is some way to see the data that was sent to fill the gap?
Hello @jasonnet, my understanding is that the IotaWatt does try to upload missing data to emoncms. I’ve seen this to work successfully on emoncms.org. If your feeds on the emonpi now contain newer data, it may have tried to upload the missing data, but for some reason failed. Unfortunately there’s no easy way to re-transfer the data. Im not that familiar with the detail of how IotaWatt works unfortunatly. It may be worth asking Bob Lemaire over at http://community.iotawatt.com/.
The standard configuration on the emonpi/emonbase is for append only writes, in order to reduce the risk of disk wear. This means that if you have posted newer data and it has recorded successfully in emoncms you can’t then go back and edit or add missing datapoints. You can turn this off by setting redisbuffer enabled to false in settings.php and then if the IotaWatt supports it reupload data to fill a gap.
Trystan. This sounds very useful. Your posting definitely gets a “heart” for that.
I’ve also adjusted the apache conf on the emonBase so that I can see requests come in. It looks like the IotaWatt is sending data to catch up, but before doing so it seems to ask the emonCMS input API for the timestamp of the most recent data it has. It looks to me that it then sends only the data since that time. (aka APPEND). Alas, that doesn’t help me because somehow subsequent data has already arrived at the emonCMS inputs so the IotaWatt isn’t sending data back from the day that had the gap. (Yes, perhaps emonCMS did at some point report a “last data” time that did not consider that it had not fully recorded the recent data… and thus I have a gap.)
Hopefully there is a way to have the IotaWatt send (just one time?) all the data rather than just appending from what appears to the data most recently received by the emonCMS server.
I’d like to follow up with the IotaWatt Community using the link you shared. But right now I am having trouble registering on their website. The verification email never arrives. I’ll mention @overeasy here in hope that Bob will see this posting. (I’ll also message him.)
