Emoncms.org broken today

Emoncms
1

Morning

It would appear that emoncms.org is down this morning and returning the following error:

Fatal error: Uncaught RedisException: LOADING Redis is loading the dataset in memory in /var/www/emoncms/index.php:73 Stack trace: #0 /var/www/emoncms/index.php(73): Redis->incr() #1 {main} thrown in /var/www/emoncms/index.php on line 73

I’m seeing this across multiple devices running Edge and Chrome.

Tim

2

I’m getting the same.

3

Same here - Fatal error: Uncaught RedisException: LOADING Redis is loading the dataset in memory in /var/www/emoncms/index.php:73 Stack trace: #0 /var/www/emoncms/index.php(73): Redis->incr() #1 {main} thrown in /var/www/emoncms/index.php on line 73

4

Back online here .. although, no data since 6:36 this morning (28th)

5

I’m back online as well now

6

I have no data logged to emoncms.org since about 07:31 (BST).

The emonhub log on my EmonPI looks normal - getting data from the sensors and sending it without any obvious errors at my end.

7

Hello all, apologies about this down time. I have resolved the core issue and the site is now back online. We do however now have a very long input queue as data for the last couple of hours is processed. A lot of devices will be posting a few hours of historic data at the same time and so it look like that’s going to take a few hours to clear through. I will keep this thread updated with progress.

8

Yes - data on emoncms.org is gradually catching up - it’s now showing my feeds as 18 minutes old, rather than the 2 hours it had been.

9

Nice! Looks like local emonhub is working as it should, pushing up the data it was not able to upload during the outage now.

10

Looks like the emoncms.org input processing queue should be fully processed by just after 10 am (in one hour from now).

11

Thanks all. Mine is now up and running but with a data outage shown. I am still trying to understand how to post logically on this site so I am glad this was raised by some else.

12

Thanks Nick, out of interest did you reboot your emonPi/base when you saw this issue - that would clear the temporary local store, it could also be a version or settings difference in the local emonhub software that may give different results for different people.

13

Queues are all caught up.

Checking a few systems it looks like it’s quite common to have the outage between 6:36am and 8am? Interestingly yours @Martin_Robinson shows no such gap. It would be great to understand why. What version of emonhub are you running and have you changed any of the emonhub settings?

Edit: the error was cleared at about 8:45am and so systems that are showing data from 8am did still post up part of the gap, just not quite enough!

14

I’ve just checked my feeds and am seeing the gap between 06:30 and 08:45.

15

Interesting…

All feeds now showing green and data is up to date with no obvious gaps:

image
image940×343 48.1 KB

I did reboot my EmonPi at around 08:30, because, of course, I suspected a local issue first.

Versions on my EmonPi are:

image
image1395×253 10.5 KB

I don’t think there’s anything odd about my emonhub.conf file: here it is:

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################

emonHub configuration file, for info see documentation:






#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
autoconf = 1

Uncomment this to also send to syslog

use_syslog = yes

#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[EmonPi2]]
Type = EmonHubOEMInterfacer
[[[init_settings]]]
com_port = /dev/ttyAMA0
com_baud = 115200
[[[runtimesettings]]]
pubchannels = ToEmonCMS,
subchannels = ToRFM12,

[[USB0]]

Type = EmonHubOEMInterfacer

[[[init_settings]]]

com_port = /dev/ttyUSB0

com_baud = 115200

[[[runtimesettings]]]

pubchannels = ToEmonCMS,

subchannels = ToRFM12,

nodename = emonTx4

[[SPI]]
    Type = EmonHubRFM69LPLInterfacer
    [[[init_settings]]]
        nodeid = 5
        networkID = 210
        resetPin = 24
        selPin = 16
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,


[[MQTT]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016
    
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        
        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 0
        node_format_basetopic = emonhub/
        
        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/
        
        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 0
        node_JSON_basetopic = emon/

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXX
        senddata = 1    # Enable sending data to Emoncms.org
        sendnames = 1    # Send full input names (compression will be automatically enabled)
        interval = 30    # Bulk send interval to Emoncms.org in seconds

[[DS18B20]]
    Type = EmonHubDS18B20Interfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        read_interval = 10
        nodename = sensors

ids = 28-000008e2db06, 28-000009770529, 28-0000096a49b4

names = ambient, cyl_bot, cyl_top

[[MBUS]]
Type = EmonHubMBUSInterfacer
[[[init_settings]]]
device = /dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_EHBUb143M07-if00-port0
baud = 2400
[[[runtimesettings]]]
pubchannels = ToEmonCMS,
read_interval = 10
validate_checksum = False
nodename = HeatMeter
[[[[meters]]]]
[[[[[heatmeter]]]]]
address = 1
type = standard

#######################################################################
#######################          Nodes          #######################
#######################################################################

See config user guide: 


If autoconf is enabled above, node configuration will automatically

populate based on templates listed in available.conf

[nodes]
[[23]]
nodename = emonth2_23
[[[rx]]]
names = temperature, external temperature, humidity, battery, pulsecount
datacodes = h, h, h, h, L
scales = 0.1, 0.1, 0.1, 0.1, 1.0
units = C, C, %, V, p
16

Thanks Martin! interesting that it still recovered for you even after the reboot .. that is unexpected! I edited out your apikey in the config post.

17

Hi Tristan No the system just came back online. Thanks

18

Thanks Trystan.