So the issue came up again. Feeds and inputs stopped recording 3 days ago. I was not able to catch it when it stopped working. I will try to restart it and catch it faster when the issue comes up.
Note that I also pull data constantly (every 1 second) by reading the feeds via the API from an external application.
In the meantime, the /var/log/emonhub/emonhub.log content is tens of thousands of lines as follows:
2016-10-25 16:17:02,125 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:02,126 WARNING MainThread MQTT thread is dead
2016-10-25 16:17:02,377 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:02,379 WARNING MainThread MQTT thread is dead
2016-10-25 16:17:02,614 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:02,615 WARNING MainThread MQTT thread is dead
2016-10-25 16:17:02,851 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:02,852 WARNING MainThread MQTT thread is dead
2016-10-25 16:17:03,088 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:03,089 WARNING MainThread MQTT thread is dead
2016-10-25 16:17:03,313 WARNING MainThread RFM2Pi thread is dead
2016-10-25 16:17:03,314 WARNING MainThread MQTT thread is dead
Could it be a problem similar to this thread.
I followed the thread around and reading this thread I tried a soft reboot. I did a ‘sudo reboot’ and it recovered. The inputs and feeds are now recording again. Any idea where the issue could be coming from to fix it?
Here is the log after reboot:
2016-10-25 17:10:53,818 INFO MainThread EmonHub emonHub 'emon-pi' variant v1.1
2016-10-25 17:10:53,819 INFO MainThread Opening hub...
2016-10-25 17:10:53,821 INFO MainThread Logging level set to DEBUG
2016-10-25 17:10:53,823 INFO MainThread Creating EmonHubJeeInterfacer 'RFM2Pi'
2016-10-25 17:10:53,830 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 38400 bits/s
2016-10-25 17:10:55,835 WARNING MainThread Device communication error - check settings
2016-10-25 17:10:55,837 INFO MainThread Setting RFM2Pi frequency: 433 (4b)
2016-10-25 17:10:56,840 INFO MainThread Setting RFM2Pi group: 210 (210g)
2016-10-25 17:10:57,842 INFO MainThread Setting RFM2Pi quiet: 0 (0q)
2016-10-25 17:10:58,844 INFO MainThread Setting RFM2Pi baseid: 5 (5i)
2016-10-25 17:10:59,847 INFO MainThread Setting RFM2Pi calibration: 110V (2p)
2016-10-25 17:11:00,850 DEBUG MainThread Setting RFM2Pi subchannels: ['ToRFM12']
2016-10-25 17:11:00,853 DEBUG MainThread Interfacer: Subscribed to channel' : ToRFM12
2016-10-25 17:11:00,855 DEBUG MainThread Setting RFM2Pi pubchannels: ['ToEmonCMS']
2016-10-25 17:11:00,856 DEBUG MainThread Interfacer: Subscribed to channel' : ToRFM12
2016-10-25 17:11:00,860 INFO MainThread Creating EmonHubMqttInterfacer 'MQTT'
2016-10-25 17:11:00,862 INFO MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2016-10-25 17:11:00,888 DEBUG RFM2Pi 1 NEW FRAME : OK 5 242 0 0 0 242 0 16 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2016-10-25 17:11:00,892 DEBUG RFM2Pi 1 Timestamp : 1477415460.89
2016-10-25 17:11:00,893 DEBUG RFM2Pi 1 From Node : 5
2016-10-25 17:11:00,908 DEBUG MainThread MQTT Subscribed to channel' : ToEmonCMS
2016-10-25 17:11:00,911 DEBUG RFM2Pi 1 Values : [242, 0, 242, 11280, 0, 0, 0, 0, 0, 0, 0, 0]
2016-10-25 17:11:00,922 DEBUG RFM2Pi 1 Sent to channel' : ToEmonCMS
2016-10-25 17:11:00,926 INFO MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg'
2016-10-25 17:11:00,930 DEBUG MainThread emoncmsorg Subscribed to channel' : ToEmonCMS
2016-10-25 17:11:01,025 INFO MQTT Connecting to MQTT Server
2016-10-25 17:11:01,040 DEBUG RFM2Pi device settings updated: E i5 g211 @ 433 MHz USA 1
2016-10-25 17:11:01,146 DEBUG RFM2Pi device settings updated: [emonPi.2.60] E i5 g211 @ 433 MHz USA 1
2016-10-25 17:11:01,148 INFO MQTT Connecting to MQTT Server
2016-10-25 17:11:01,159 INFO MQTT connection status: Connection successful
2016-10-25 17:11:01,162 DEBUG MQTT CONACK => Return code: 0
2016-10-25 17:11:01,255 DEBUG RFM2Pi 2 NEW FRAME : OK 5 243 0 0 0 243 0 51 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2016-10-25 17:11:01,292 INFO MQTT on_subscribe
2016-10-25 17:11:01,367 DEBUG RFM2Pi 2 Timestamp : 1477415461.25
2016-10-25 17:11:01,369 DEBUG RFM2Pi 2 From Node : 5
2016-10-25 17:11:01,371 DEBUG RFM2Pi 2 Values : [243, 0, 243, 113.15, 0, 0, 0, 0, 0, 0, 0]
2016-10-25 17:11:01,373 INFO RFM2Pi Publishing: emon/emonpi/power1 243
2016-10-25 17:11:01,382 INFO RFM2Pi Publishing: emon/emonpi/power2 0
2016-10-25 17:11:01,386 INFO RFM2Pi Publishing: emon/emonpi/power1pluspower2 243
2016-10-25 17:11:01,389 INFO RFM2Pi Publishing: emon/emonpi/vrms 113.15
2016-10-25 17:11:01,393 INFO RFM2Pi Publishing: emon/emonpi/t1 0
2016-10-25 17:11:01,409 INFO RFM2Pi Publishing: emon/emonpi/t2 0
2016-10-25 17:11:01,412 INFO RFM2Pi Publishing: emon/emonpi/t3 0
2016-10-25 17:11:01,416 INFO RFM2Pi Publishing: emon/emonpi/t4 0
2016-10-25 17:11:01,419 INFO RFM2Pi Publishing: emon/emonpi/t5 0
2016-10-25 17:11:01,422 INFO RFM2Pi Publishing: emon/emonpi/t6 0
2016-10-25 17:11:01,425 INFO RFM2Pi Publishing: emon/emonpi/pulsecount 0
2016-10-25 17:11:01,428 INFO RFM2Pi Publishing: emon/emonpi/rssi 0
2016-10-25 17:11:01,430 INFO RFM2Pi Publishing: emonhub/rx/5/values 243,0,243,113.15,0,0,0,0,0,0,0
2016-10-25 17:11:01,433 INFO RFM2Pi Publishing: emonhub/rx/5/rssi 0
2016-10-25 17:11:01,436 DEBUG RFM2Pi 2 adding frame to buffer => [1477415461, 5, 243, 0, 243, 113.15, 0, 0, 0, 0, 0, 0, 0]
2016-10-25 17:11:01,438 DEBUG RFM2Pi 2 Sent to channel' : ToEmonCMS
2016-10-25 17:11:01,547 DEBUG RFM2Pi device settings updated: E i5 g211 @ 433 MHz USA 1
2016-10-25 17:11:01,702 DEBUG RFM2Pi 3 NEW FRAME : OK 5 244 0 0 0 244 0 50 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2016-10-25 17:11:01,715 DEBUG RFM2Pi 3 Timestamp : 1477415461.7
2016-10-25 17:11:01,717 DEBUG RFM2Pi 3 From Node : 5
2016-10-25 17:11:01,718 DEBUG RFM2Pi 3 Values : [244, 0, 244, 113.14, 0, 0, 0, 0, 0, 0, 0]
2016-10-25 17:11:01,721 INFO RFM2Pi Publishing: emon/emonpi/power1 244
2016-10-25 17:11:01,725 INFO RFM2Pi Publishing: emon/emonpi/power2 0
2016-10-25 17:11:01,729 INFO RFM2Pi Publishing: emon/emonpi/power1pluspower2 244
2016-10-25 17:11:01,733 INFO RFM2Pi Publishing: emon/emonpi/vrms 113.14
2016-10-25 17:11:01,740 INFO RFM2Pi Publishing: emon/emonpi/t1 0
2016-10-25 17:11:01,755 INFO RFM2Pi Publishing: emon/emonpi/t2 0
2016-10-25 17:11:01,775 INFO RFM2Pi Publishing: emon/emonpi/t3 0
2016-10-25 17:11:01,782 INFO RFM2Pi Publishing: emon/emonpi/t4 0
2016-10-25 17:11:01,788 INFO RFM2Pi Publishing: emon/emonpi/t5 0
2016-10-25 17:11:01,793 INFO RFM2Pi Publishing: emon/emonpi/t6 0
2016-10-25 17:11:01,802 INFO RFM2Pi Publishing: emon/emonpi/pulsecount 0
2016-10-25 17:11:01,812 INFO RFM2Pi Publishing: emon/emonpi/rssi 0
2016-10-25 17:11:01,816 INFO RFM2Pi Publishing: emonhub/rx/5/values 244,0,244,113.14,0,0,0,0,0,0,0
2016-10-25 17:11:01,820 INFO RFM2Pi Publishing: emonhub/rx/5/rssi 0
2016-10-25 17:11:01,827 DEBUG RFM2Pi 3 adding frame to buffer => [1477415461, 5, 244, 0, 244, 113.14, 0, 0, 0, 0, 0, 0, 0]