I need to better understand how RedisBuffer behaves in the emonPi environment. I am running with the 20Feb2016 image in a Raspberry Pi 2, and my hobby horse is getting the pulse-based gas usage properly represented in the feed. This means being able to update earlier datapoints in the feed, perhaps as far back as 15 minutes from the current datapoint. This appears to be problematic or unreliable, and I am trying to understand why. I think, though not sure, that RedisBuffer is struggling a bit with updates further back than perhaps about three to five 10s intervals in my gas Watts feed, possibly needing more time before the next value comes along.
Is there a way to have Redisbuffer retain at least 12 or 16 minutes of a given feed in memory, so that all updates can take place in RAM without having to reload older datapoints from storage?
Thanks Bill Thomson for your response. I am puzzled why you answered me by email, rather than here, where it might have helped other readers, and which might have lead to some discussion and debate about the merits or otherwise of what I am doing and why I think it is worth doing. I think that discussion should still happen.
The retrowriting function is now working well, both in the gas Watts Retrofeed and in the graph display in the modified mysolarpv web app, where the solar Watts are actually my gas Watts, showing alongside the electric Watts.
Extracts in .csv from the gas Feed show that the solitary pulse-derived Watts figure is now preceded by a series of the same Watts values as far back as the time of the previous pulse, or only 10 minutes earlier, whichever is shorter, and not going back further than they should, which was the problem I was having.
This makes a huge difference in the display graph of the gas usage, because the sampling system of the app rarely picked up the solitary Watts values, so they were severely under-represented in the graph, and in the kWh count.
@Bill.Thomson - for some reason your post of 11th May above is shown as ‘hidden’, and therefore Mike may not be able to see it. He may have received the discourse email alert, but when checking the forum there would be no reply showing to him.
Paul
UPDATE - it was showing as ‘hidden’ because it had been deleted, I’ve just restored the post so it can be seen again (I presume/hope that it wasn’t deleted on purpose?)
Well, that explains the email. I did reply to it, even though it suggested it was a no-reply one, because it was worth the attempt, because there was nothing really happening in the thread.
My reply was just that the save settings for redis were all commented out meaning no writes to disc were being done, so could not have contributed to the problem I was seeing. But that just meant it was even less clear just what redisbuffer was actually doing. When I did amend the redisbuffer sleep setting to 1200, my graph updates stopped happening, until after 20 minutes they did, but only after refreshing the web page. I put it down to 90, and have left it at that.
The fix that worked was simply reversing the order of the updates in my log to retrofeed function. Instead of writing the new value into the last value, and working back through older datapoints, I started with the oldest one and worked back up to the last value. This let redisbuffer reload anything it needed in one operation, instead of several, if its buffer didn’t contain any of the datapoints I wanted to overwrite.
Basically, everything is working pretty much as I want, the only detail I still have to crack now is writing a new scale on the right hand of the graph area, as my gas Watts are displayed at 1/4 height alongside full height electric Watts. This is a cosmetic measure, because the highest gas peaks are about 38000W, and common, while my electric Watts rarely breach 10000W, mostly varying up to around 4000-5000W. Thus the scaling function was rescaling all my peaks to fit in the tallest gas peaks, and so scaling down all my electric ones, so those mostly appeared as random noise at the bottom of the graph, with yellow gas peaks standing proud and tall above the blue undergrowth.
Nevertheless, Bill, yours was still the first meaningful response to any queries I’ve made since I started this project, and when I said “Thanks, Bill”, I meant it, and still do.
