Help, Emoncms not responding

What have I done? I now cannot get any response from the web interface. I am fairly new with my emonPi, so my level of understanding the inner workings is limited. I have up until now, been successfully operating for ~3 mo, logging data locally.

I was trying to implement a virtual feed to estimate total energy usage; without reading any relevant documentation, that I have not been able to find, I might add. When I tried to save the virtual feed, the interface displayed a number of error messages. The interface became unresponsive and I had to shut down the tab in my browser (FF 56).

When I now attempt to run the web interface now, it starts up on the ‘feeds’ page and but does not display the feeds or complete the update. It hangs there only displaying the circular waiting widget. If I wait long enough, it may allow me to select a different page, such as inputs or graph. These pages will start to load, then the wait begins again.

What do I need to do to get this working again? A complete power off has not rectified the situation. The emonPi seems to be merrily logging data. All readings indicated on the display are normal. I can log in to the system using putty, but I don’t want to mess around with the software without instruction.

You will probably need to delete the virtual feed via a back door, ie from the command-line with mySQL or by installing phpmyadmin and doing it via a webpage gui. See Corrupted virtual feed breaks the feed list

Thank you for replying. But, beyond having managed to log into my emonPi using Putty and having reset the time server, I know nothing about mySQL or installing/using phpmyadmin.

I would like to try using mySQL before installing anything new, but I need somewhat more detailed instruction.
I have managed to get to the mysql command line, but I have no idea of what to do now.

I think you will find it much easier to install phpmyadmin and do it that way. I certainly wouldn’t be able to walk you through the mysql commands without research or trying it myself.

phpmyadmin is relatively self-self explanitory, it’s the way I would do it and I do have some mysql command-line experience.

Installing is easy using these commands

rpi-rw
sudo apt-get update
sudo apt-get install phpmyadmin

the installer will ask you 3 or 4 questions along the way. From memory use the space bar to select “Apache2” when asked which server and accept all the defaults except for setting up a password for you to access phpmyadmin.
Once done you can navigate to http://whateveryourserveriscalledorip/phpmyadmin, login, select emoncms in the left panel to dropdown a list of tables in emoncms, select “feed” and in the main panel you should see all your feeds, theoretically the one with the highest id should be the one you want but check the name and type fields before deleting. It will also be one of the few with anything in the processlist field.

You may also need to flush redis, either from the admin page of emoncms or from the command line via ssh with

sudo redis-cli FLUSHALL

OK. Followed these instruction until the installation generated an error after accepting the default option to create a default database. This generated an error.


To get past this point, I selected the alternate, that was to use a pre-existing data base. Which one, I don’t know, there was no selection option provided. Anyway, the installation completed.

But now, I cannot login to phpmyadmin. The login screen appears but it will not accept the password I used. What is the username? The installation did not ask for one. Is there a default?

Okay, so I tried installing phpmyadmin to an emonpi and fell down the same hole as you. The only time I have ever experienced any problem installing phpmyadmin is when it hasn’t previously uninstalled correctly and there has been an existing table blocking the creation of a new one.

Following the failed installation I fully uninstalled it again, using “purge” and “clean” to get rid of any traces and then reinstalled again. Second time around I had to enter the root mysql password and entered through the phpmyadmin to mysql password page. It did not ask me to set up any phpmyadmin user password, but I could then login to phpmyadmin using the mysql “root” username and password (emonpimysql2016).

rpi-rw
sudo apt-get purge phpmyadmin
sudo apt-get clean

sudo apt-get install phpmyadmin

Say yes to removing the old table when uninstalling and yes to create a new table when reinstalling. other options as previous buit enter the mysql admin password of “emonpimysql2016” when asked.

This is a bit of a security risk as that password is common and published so if your emonpi is visible outside the LAN (or possibly even if it’s not) you should either change the mysql root password or remove phpmyadmin once the job is done.

It took me three tries to get it right, I think.I have logged into phpmyadmin and was able to delete the offending feed. But then it was not obvious how to save the configuration and exit from the program. I used the logout button.

The emoncms webserver now loads as before. The presumably offending virtual feed is gone. The process list for input Line1 was missing, so I re-entered that, Identical to Line2.

None of the feed values is updating and the ‘updated’ column shows 13.6 hrs of data pending. (?). There is no new data displayed on any graph. The gap is presumably due to the 2nd installation attempt of phpmyadmin being interrupted and left overnight for completion.

What now? A reboot?

phpmyadmin is issuing mysql queries/commands behind the scenes in real time, if it didn’t complain and splash red banners up then it did what you asked, loging out will close the connection but the changes were made as you selected them.

This is a backdoor operation, there is no tried and tested procedure, we are doing this because something went wrong and conventional method were blocked by the lack of access to the input/feed list pages. I would think a reboot is probably the best thing to try at this point.

There may yet be some further issues to iron out if there is an input process still trying to use the deleted virtual feed, but it sound like you’ve address that in the “empty line” perhaps.

If you do reboot you will lose the 13.6hrs data (it’s probably lost already) it isn’t really “pending”, the 13.6hrs ago is telling you when the feed was last updated, not how much data is “pending”. you would see the same thing if there was no data coming in for 13.6hrs.

I suspect the processing is being blocked due to the rogue virtual feed (or its ghost) in ram or redis, did you try flushing redis? One would hope a reboot would get rid of any traces, if flushing redis doesn’t.

This seems to be a re-occurring theme, and really the errors should be better handled in code without users having to resort to this.

I had practically the same issue a couple of months ago, and unfortunately I trashed the system trying to fix it - hence I rebuilt my system on stretch.

Paul

OK, Paul. Thank you very much for the help. We are operational again. Yes, I did try flushing redis, but I must admit I don’t know exactly what that means. But a reboot of the emonPi, ie. I pulled the power plug, did the trick and yes 13.6hrs of data is now missing. As the emonPi LCD has shown updated data the whole time, I hoped that it would somehow magically end up in the database. No such luck.

At this point, I would ask for help in understanding the concept of virtual feeds, how to use them and what they are for? And the same could be said for the process lists, Is there any succinct documentation? I have many issues that I need to resolve to proceed with confidence.

I got myself into this mess, by trying to create a virtual feed that summed the two Line feeds to provide the total energy consumption (W), not unlike the total power feed (KWh). I don’t recall how I failed, I was simply playing around, confident that it would be simple to recover from any mistake.

The processes that are attached to the various [node][keys] of the input panel are minimally documented in the setup screen when you are attaching a new process. I do not understand some of the terminologies. ‘UPSERT’ (?) for example. I would like to have feeds of total power and total energy consumed, for instance, that reset to zero at a specified scheduled time. I would also like to be able to generate a feed of the accumulated power cost, that resets to zero on a monthly basis, and is based on the scheduled rates in my locale. I sense that all this should be possible, but I don’t want to get into another mess like I just got out of.

Calibration is another issue. I understand that I can use a simple multiplication factor to scale the power1 input before it is logged. This works but needs to be done as well to the power2 input and power1pluspower2 input. But the scaling is not reflected on the LCD display of the emonPi. It remains unscaled. To achieve that, I had to edit the scaling numbers for the emonPi in the emonhub.conf file as below:

nodename = emonpi
[[[rx]]]
    names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
    datacodes = h, h, h, h, h, h, h, h, h, h, L
    scales = 1.093,1.093,1,0.01093,0.1,0.1,0.1,0.1,0.1,0.1,1
    units = W,W,W,V,C,C,C,C,C,C,p

This is where I am right now. The scalar for power1pluspower2 remains in the process list for that input. The value 1.093 comes from the measurement of AC voltage at my panel compared to the value that the emonPi displayed. This is nearly the value of 120/110 (1.0909). I recall reading somewhere on the forum that these scaling factors were not the intended place to calibrate the readings. But I have found no other way of making the emonPi display and database be consistent with each other. I would have thought that once the VRMS measurement has been calibrated, then the power measurments should follow from that, since the power is simply the product of the voltage and the current, as measured by the CT’s. There is probably something about the architecture of the system that requires this value be entered in multiple places. And, this would be a design error, in my opinion.

My apologies for this tour-de-force and again, thank you for the help. You seem like more than an interested, helpful 3rd party. Are you more deeply involved in the OEM project? I don’t necessarily expect you to answer all these questions. I had to get it out. If you feel it should be posted elsewhere, I will do that.

Great!

I will try and answer some of your question, but I will probably do it over a couple of posts as I’m here there and everywhere over the next few days. Just heading out now so I will come back to it later in the day.

In a way, you’re right. The emonPi front end processor is a stripped-down emonTx, running what is almost exactly the same sketch. You can calibrate that sketch as described in the ‘Learn’ section, but the ‘easy’ instructions Glyn sent to me for editing and reloading it didn’t work, and the old-fashioned way is a pain. The method of tweaking the scale factors in emonhub.conf is a fudge to allow you to get the reported and displayed values more-or-less right using only the web browser interface. Knowing that, and realising that real power is calculated in the front end sketch, you can see why a voltage calibration error affects all three power values, and a current calibration error affects only two of the three.

Both Pauls and I are moderators here. Paul (pb66) developed the original emonhub that the emonPi variant is based on; Paul Reed is very close to being a founder member, and is an expert on emoncms.