All inputs are gone

[jasonnet (Jason Crawford)]

A few months ago my OpenEnergyMonitor stopped receiving input. I have an emonBase and many TH’s and TX’s. When this happened, I became busy and shutdown my emonBase and moved on to urgent tasks. I’m back now. I don’t recall if anything in particular triggered the change. My notes indicate that I did try to fix it by the emonCMS update and via apt update/upgrade. Neither helped.

As I debug now I see that emonHub is running because “top -c” shows:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                      
971 emonhub   20   0   49620  16520   6456 S   3.9  1.7   2:19.52 python /usr/share/emonhub/emonhub.py --config-file /home/pi/data/emonhub.conf --logfile /var/log/emonhub/em+ 

The /var/log/emonhub/emonhub.log lists a continuous stream of the following and nothing else:

2018-10-27 19:00:20,931 DEBUG    RFM2Pi     device settings updated: [RF12demo.14] E i5 g210 @ 433 MHz
2018-10-27 19:00:23,651 DEBUG    RFM2Pi     device settings updated: [RF12demo.14] E i5 g210 @ 433 MHz

It looks to me like maybe the red light on the daughter board on the emonBase flashes each time an entry appears in this log file. I have a lot of TH’s and TX’s in the house. I suspect most of the TH’s are no longer running. There is an emonTH right next to the emonBase and it’s red LED flashes about every 58 seconds.

FWIW the emonhub config for that TH is

[[22]]
   nodename = TH22_Kitchen
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

Where should I be looking to fix this?

[borpin (Brian Orpin)]

Could you share a bit more of the setup is please?

• Where is EmonCMS hosted - locally on an RPi?
• How was it installed? Self install or the SD card image from OEM?

Have you done an update of the database (on the admin screen)?

On the administration screen, to the right of the heading ‘Server Information’ you should see a ‘copy to clipboard’ button (depending on the version you are running). If it is there, please press that, then paste the result into a post here (no other formatting required).

Also the whole emonhub config will be useful.

[jasonnet (Jason Crawford)]

Hello Brian. Thanks for your reply.

The EmonCMS is hosted on the EmonBase in my house. It was installed from downloaded SD card image about a year ago. I think the last update (from the Admin screen) was on April 17th according to the emonpiupdate.log file.

That file also has the following line near the top of it:

emonSD version: emonSD-07Nov16

I’m also seeing the following in emoncms.log:

2018-10-27 18:00:45.809|WARN|phpmqtt_input.php|Not connected, retrying connection
2018-10-27 18:00:45.820|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0

I’ll post the two files you just mentioned…

[jasonnet (Jason Crawford)]

Server Information

Emoncms
Version
low-write 9.8.28 : 2018.01.27



Modules
Administration : App v1.1.0 : Backup v1.1.2 : EmonHub Config v1.0.0 : Dashboard v1.0.1 : EventProcesses : Feed : Graph v1.2.0 : Input : postprocess : CoreProcess : Schedule : setup : Time : User : Visualisation : WiFi v1.0.0



Buffer
loading…



Writer
Daemon is running with sleep 60s


Server
OS
Linux 4.9.35-v7+



Host
emonpi emonpi (127.0.1.1)



Date
2018-10-28 18:18:13 UTC



Uptime
18:18:13 up 1 day, 18 min,  0 users,  load average: 0.08, 0.09, 0.03


HTTP
Server
Apache/2.4.10 (Raspbian) HTTP/1.1 CGI/1.1 80


MySQL
Version
5.5.59-0+deb8u1



Host
localhost (127.0.0.1)



Date
2018-10-28 18:18:13 (UTC 00:00‌​)



Stats
Uptime: 15092078  Threads: 2  Questions: 1802  Slow queries: 0  Opens: 58  Flush tables: 1  Open tables: 51  Queries per second avg: 0.000


Redis
Version
2.8.17



Host
localhost:6379 (127.0.0.1)



Size
124 keys  (493.82K)



Uptime
1 days


MQTT
Version
1.4.14



Host
localhost:1883 (127.0.0.1)


Pi
CPU Temp
45.08°C



Release
emonSD-07Nov16



File-system
Set root file-system temporarily to read-write, (default read-only)


Memory
RAM
Used: 25.48% Total: 970.93 MB Used: 247.35 MB Free: 723.58 MB


Disk
Mount
Stats



/
Used: 68.66% Total: 3.33 GB Used: 2.29 GB Free: 903.54 MB



/mnt/usb/sda1
Used: 0.53% Total: 109.91 GB Used: 592.9 MB Free: 103.72 GB



/home/pi/data
Used: 19.12% Total: 193.66 MB Used: 37.02 MB Free: 146.64 MB



/boot
Used: 36.36% Total: 59.95 MB Used: 21.8 MB Free: 38.15 MB


PHP
Version
5.6.33-0+deb8u1 (Zend Version 2.6.0)



Modules
apache2handler : bcmath : bz2 : calendar : Core v5.6.33-0+deb8u1 : ctype : curl : date v5.6.33-0+deb8u1 : dba : dio v0.0.4RC4 : dom v20031129 : ereg : exif v1.4 : fileinfo v1.0.5 : filter v0.11.0 : ftp : gettext : hash v1.0 : iconv : json v1.3.6 : libxml : mbstring : mcrypt : mhash : mosquitto v0.3.0 : mysql v1.0 : mysqli v0.1 : openssl : pcre : PDO v1.0.4dev : pdo_mysql v1.0.2 : Phar v2.0.2 : posix : readline v5.6.33-0+deb8u1 : redis v2.2.7 : Reflection : session : shmop : SimpleXML v0.1 : soap : sockets : SPL v0.2 : standard v5.6.33-0+deb8u1 : sysvmsg : sysvsem : sysvshm : tokenizer v0.1 : wddx : xml : xmlreader v0.1 : xmlwriter v0.1 : Zend OPcache v7.0.6-devFE : zip v1.12.5 : zlib v2.0 :

Client Information

HTTP
Browser
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0


Screen
Resolution
1728 x 1152


Window
Size
1711 x 1038

[jasonnet (Jason Crawford)]

emonhub.conf.txt (7.7 KB)

[borpin (Brian Orpin)]

Can you run the following command and post the output please.

sudo systemctl status {feedwriter,emonhub,mosquitto,redis-server,mqtt_input}

[jasonnet (Jason Crawford)]

Thanks Brian.
It looks like all of those processes are active. Here is the listing you suggested:
system_status.txt (2.9 KB)

[jasonnet (Jason Crawford)]

In case it’s helpful, here is the top of the emonhub.log shortly after a reboot
emonhub.log.txt (4.2 KB)

[borpin (Brian Orpin)]

This sounds similar to this thread Feeds are missing after reboot

The suggestion elsewhere is that it is a redis problem. There are a couple of suggestions to modify the wait period in the cron script to solve it. Redis does not startup after a reboot - #7 by brandock

[jasonnet (Jason Crawford)]

Thanks Brian.
I’ve fixed the problem.
My hypothesis was that something on the RFM69 or between it and the emonhub was preventing the data that the RFM69 certainly was collecting (given the irregular red LED flashes) from reaching emonhub… given that the emonhub.log file listed pretty much nothing coming in. I further hypothesized that I must have just did an emonBase update just before the failure a few months ago and that that the updated software or some sort of problem during the update process might be responsible for this feed problem.

I was having trouble finding documentation for how I should debug this and was becoming impatient so I ran “emonBase Update” again, hoping that there had been a software update recently that would fix the problem or that updating would reset whatever went wrong with the last update. (But I was also fearing that this update would fail and also wipe away evidence of the original problem and cause you and I to start over debugging this.)

That update did cause a “Database error, you may need to run database update” login problem, but finding a community forum posting describing a fix to that was easy. And once my browser was logged in to emonCMS, I could see the feed data was again reaching emonCMS.

So I’m now back in business.

Thanks for your help through this. Is there a way I can give you some karma points?

Jason.

[Greebo]

@jasonnet

Is there a way I can give you some karma points

Hit the little heart icon under the post(s) of his that you liked or that helped the most…