PHPFINA value drop to zero

Hello community members

my name is giuseppe and i am using OpenEnergyMonitor since a year with success at my home.

Recently I experienced the following problem in one of my feed : values drop to zero .
I attach a picture to show this :

Could you help to find the reason of this ? Is it possible to fix the values ?

Thanks a lot
ciao

Hi Giuseppe

Can you provide more info?

What is “F1Reading” ? How does the emonTx data reach emoncms?

Is this emoncms.org or self hosted? What input processing is behind it?

How long has it been running correctly?

Can you zoom in and see exactly what date and time it happened? Do you recall anything specific about that date and time eg power outage or home improvements etc? Are there any other anomalies at that time and date on any other feeds when you look closely?

Hi
I’m experiencing the same issue.

The feeds are working fine in normal conditions. I recently upgraded my system from ubuntu 15.10 to 16.04 and during the upgrade the webserver was disabled for (quite) some time. After the final reboot, all systems were back online and the locally hosted emoncms was available again, but all my phpfina feeds dropped to zero. This issue renders all the applications useless, as now in my electric, the report Year : -15930 kwh, month: -19115 kwh. I tried fixing the feeds, by recreating them from my allpower feed, but the recreation loses information.

Hey @Bill.Thomson I see you tried to add something in my notifications but I don’t find anything in the thread ?

Bill corrected a typo that you made, as you misspelled ‘loses’.

Paul

@ziporah can you provide more info.

What are the feeds that reset?

Where does the data come from?

What processing do you have on the inputs that post to those feeds?

hello @pb66, here is the additional info.
The feeds that reset are the Power to kWh feeds. The feeds are phpfina feeds. I’ve been using these feeds since jan.2015 and have migrated them over different versions of emoncms. I think they were created with a 10s fixed time, since my nodes log data every 10 seconds.
I have several inputs logging to kwh. And they all reset to zero.
The data comes from a single emontx v3 and from 3 combined emontx V3 nodes that monitor 3phase usage using the I2C bus.The data is pushed to an Open-Kontrol-Gateway and then sent to my ubuntu server over the ethernet. I ‘hacked’ the current emoncms version to support the push to api functionality again as I didn’t have time to try and upgrade the Gateway to a more recent version of the firmware that doesn’t use the api to push the data.
The processing on these feeds are typically in this order:
1 log to feed
2 power to kWh
3 power to kWh/d

if you want I can put one of the phpfina files available for you to download if that should help.

Ok, so typically the incoming 10s data is power in watts and the power to kwh process is used to post to a 10s phpfina feed, that sounds fine. I just wanted to check there was nothing odd in the way the data arrived.

This isn’t something I’ve experienced so I’m not sure how much I can help, especially as I have never run emoncms on Ubuntu either.

When new input data arrives the power to kwh process takes the last datapoint from the feed and uses that time and value pair to establish the time interval between this and the last datapoint, if that time interval is less than 2hrs it updates as usual, by calculating the energy over that time interval from the power and adding it to the last value, or if the interval is over 2hrs, it will fallback to using the last value without any adjustment for this updated input to avoid any spikes caused by absent inputs.

So in those instances you should have seen either uninterrupted data or at worse, a flat where the same value was reused. For it to drop to “zero” that would mean the last value was not accessible at the time and the last time and value have been assumed to be zero’s. Then the above code would determine the time interval to be over 2hrs (now - 0 = >2hrs) and reuse the last value, (assumed to be zero).

Why the last values were un-available at that time I can only speculate, but do you use redis? if you do not then I cannot think why this should happen if you migrated your data, obviously permission wasn’t an issue as it has written to the db.

Did you have the Redis settings in emoncms/settings.php set correctly before first use?

If you are using Redis, Was the Redis server running before the apache2 server came on-line? Did you have the phpredis PHP extension enabled? Did you flush Redis?

A change from Ubuntu 15.10 to 16.04 usually involves a change from php 5.6 to 7.0, so the phpredis extension could need changing as well, depending on what version you were previously running.<=2.2.8 won’t work with php7.0.

Sorry I can’t be of more help. I would be good to find out the reason this happened to try and avoid it happening again, but I do not think we can rectify your data by finding the issue.

The only way to fix your data is to find someway of manually adding the last good value to every single subsequent value, there is no mechanism to do this within emoncms that I’m aware of, so you would need to edit the phpfina files offline and directly by writing a custom script, there are some scripts you could look at for ideas in the emoncms/useful_scripts github repo though.

thanks for the info; now I understand how it is supposed to work. I needed to fix a lot of stuff after the upgrade. Redis was working, so I don’t see an issue there. The old php version was still there with the right plugins, but the site was loaded with the new php without the right plugins.

I also had to hack in the api command again, so it took me over 2 hours to fix all the issues and have a working system again. That probably reset the feed.

I tried rebooting the machine yesterday, this went fine without any impact on the feed.

I already checked the scripts folder, but I will have to add a new script to fix the feed probably since I couldn’t find one that tackles this issue.

@pb66 what would be the correct procedure to restart the application and make sure the phpfina files are reloaded from disk ? I tried redis-cli flush, stop apache, alter the files and start apache, but the feeds drop to zero again after fixing them. I will try a full system restart after altering the feeds to see if that helps, but knowing the right procedure would also be nice :slight_smile:

I don’t think there is a documented “start up procedure” as such. The things you need to be wary of is whether the correct modules are installed and active, plus if there is any stray old Redis tables in memory that can cause havoc.

A full restart/reboot should do it, but I would try stopping apache2, edit the files, flush Redis from the command line and then restart apache2. In theory that should ensure only the newly edited data is loaded in to Redis.

It might be worth stopping your source data whilst you do this so that when you restart apache2 there is no incoming data. The process that caused the feed to zero is triggered when new data arrives, before that happens you should check the loaded modules are present and correct via the admin page and check you php files are being read correctly etc, then once your happy post 1 or 2 frames of data to one feed as a test.

The new datapoints should be equal or greater (ref the 2hr rule) if it is zerod that is only one feed to fix and try again,