Cannot change post interval to external Emoncms CMS

spallared · 29 May 2018 12:49

Hello, i’ve an EmonTX that transmit data to an emon-base every 10 seconds and everything works fine.
Now i’ve setup an external EmonCMS server on a linux box and i’m trying to post data to that server.
It almost works, but data is posted every 30 seconds and not every 10 seconds as in the configuration file.

This is the configuration:

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = 	http://10.1.3.242/html/emoncms
        apikey = *******************************
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 0                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 10                # Bulk send interval to Emoncms.org in seconds

It seems that sendinterval is ignored, why? what i’m doing wrong?

Emonbase version is: low-write 9.8.28 | 2018.01.27

Thank you in advance,
Regards
Luca

pb66 · 29 May 2018 13:00

Are there any errors reported in emonhub.log? If you change the interval can you see that change being applied in the emonhub.log?

spallared · 29 May 2018 13:26

No, there are no errors … from the log i can see that data is received and posted locally every 10 seconds as setup in the input but is posted to my cms every 3 reading:

2018-05-29 13:23:12,518 INFO     MQTT       Publishing: emonhub/rx/8/values 583,558,3465,0,263.36,300,300,300,300,300,300,0,-80
2018-05-29 13:23:15,000 DEBUG    emoncmsorg Buffer size: 3
2018-05-29 13:23:15,001 INFO     emoncmsorg sending: http://10.1.3.242/html/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1527600172.476953,8,582,556,3474,0,263,300,300,300,3                                             00,300,300,0,-81],[1527600182.410771,8,582,571,3467,0,263.2,300,300,300,300,300,300,0,-80],[1527600192.300197,8,583,558,3465,0,263.36,300,300,300,300,300,300,0,-80]]&sentat=1527600195
2018-05-29 13:23:15,039 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://10.1.3.242/html/emoncms
2018-05-29 13:23:22,151 DEBUG    RFM2Pi     13208 NEW FRAME : OK 8 69 2 58 2 149 13 0 0 144 102 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-80)
2018-05-29 13:23:22,154 DEBUG    RFM2Pi     13208 Timestamp : 1527600202.15
2018-05-29 13:23:22,155 DEBUG    RFM2Pi     13208 From Node : 8
2018-05-29 13:23:22,156 DEBUG    RFM2Pi     13208    Values : [581, 570, 3477, 0, 262.56, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:23:22,157 DEBUG    RFM2Pi     13208      RSSI : -80
2018-05-29 13:23:22,158 DEBUG    RFM2Pi     13208 Sent to channel(start)' : ToEmonCMS
2018-05-29 13:23:22,159 DEBUG    RFM2Pi     13208 Sent to channel(end)' : ToEmonCMS
2018-05-29 13:23:22,376 DEBUG    MQTT       Publishing: emon/emontx3/power1 581
2018-05-29 13:23:22,378 DEBUG    MQTT       Publishing: emon/emontx3/power2 570
2018-05-29 13:23:22,380 DEBUG    MQTT       Publishing: emon/emontx3/power3 3477
2018-05-29 13:23:22,382 DEBUG    MQTT       Publishing: emon/emontx3/power4 0
2018-05-29 13:23:22,384 DEBUG    MQTT       Publishing: emon/emontx3/vrms 262.56
2018-05-29 13:23:22,386 DEBUG    MQTT       Publishing: emon/emontx3/temp1 300
2018-05-29 13:23:22,388 DEBUG    MQTT       Publishing: emon/emontx3/temp2 300
2018-05-29 13:23:22,390 DEBUG    MQTT       Publishing: emon/emontx3/temp3 300
2018-05-29 13:23:22,392 DEBUG    MQTT       Publishing: emon/emontx3/temp4 300
2018-05-29 13:23:22,394 DEBUG    MQTT       Publishing: emon/emontx3/temp5 300
2018-05-29 13:23:22,397 DEBUG    MQTT       Publishing: emon/emontx3/temp6 300
2018-05-29 13:23:22,399 DEBUG    MQTT       Publishing: emon/emontx3/pulse 0
2018-05-29 13:23:22,401 DEBUG    MQTT       Publishing: emon/emontx3/rssi -80
2018-05-29 13:23:22,403 INFO     MQTT       Publishing: emonhub/rx/8/values 581,570,3477,0,262.56,300,300,300,300,300,300,0,-80
2018-05-29 13:23:32,070 DEBUG    RFM2Pi     13209 NEW FRAME : OK 8 71 2 54 2 135 13 0 0 202 102 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-81)
2018-05-29 13:23:32,074 DEBUG    RFM2Pi     13209 Timestamp : 1527600212.07
2018-05-29 13:23:32,075 DEBUG    RFM2Pi     13209 From Node : 8
2018-05-29 13:23:32,076 DEBUG    RFM2Pi     13209    Values : [583, 566, 3463, 0, 263.14, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:23:32,077 DEBUG    RFM2Pi     13209      RSSI : -81
2018-05-29 13:23:32,079 DEBUG    RFM2Pi     13209 Sent to channel(start)' : ToEmonCMS
2018-05-29 13:23:32,079 DEBUG    RFM2Pi     13209 Sent to channel(end)' : ToEmonCMS
2018-05-29 13:23:32,276 DEBUG    MQTT       Publishing: emon/emontx3/power1 583
2018-05-29 13:23:32,277 DEBUG    MQTT       Publishing: emon/emontx3/power2 566
2018-05-29 13:23:32,279 DEBUG    MQTT       Publishing: emon/emontx3/power3 3463
2018-05-29 13:23:32,280 DEBUG    MQTT       Publishing: emon/emontx3/power4 0
2018-05-29 13:23:32,282 DEBUG    MQTT       Publishing: emon/emontx3/vrms 263.14
2018-05-29 13:23:32,284 DEBUG    MQTT       Publishing: emon/emontx3/temp1 300
2018-05-29 13:23:32,286 DEBUG    MQTT       Publishing: emon/emontx3/temp2 300
2018-05-29 13:23:32,288 DEBUG    MQTT       Publishing: emon/emontx3/temp3 300
2018-05-29 13:23:32,290 DEBUG    MQTT       Publishing: emon/emontx3/temp4 300
2018-05-29 13:23:32,292 DEBUG    MQTT       Publishing: emon/emontx3/temp5 300
2018-05-29 13:23:32,293 DEBUG    MQTT       Publishing: emon/emontx3/temp6 300
2018-05-29 13:23:32,295 DEBUG    MQTT       Publishing: emon/emontx3/pulse 0
2018-05-29 13:23:32,297 DEBUG    MQTT       Publishing: emon/emontx3/rssi -81
2018-05-29 13:23:32,299 INFO     MQTT       Publishing: emonhub/rx/8/values 583,566,3463,0,263.14,300,300,300,300,300,300,0,-81
2018-05-29 13:23:42,007 DEBUG    RFM2Pi     13210 NEW FRAME : OK 8 69 2 50 2 133 13 0 0 161 102 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-81)
2018-05-29 13:23:42,010 DEBUG    RFM2Pi     13210 Timestamp : 1527600222.01
2018-05-29 13:23:42,012 DEBUG    RFM2Pi     13210 From Node : 8
2018-05-29 13:23:42,013 DEBUG    RFM2Pi     13210    Values : [581, 562, 3461, 0, 262.73, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:23:42,014 DEBUG    RFM2Pi     13210      RSSI : -81
2018-05-29 13:23:42,015 DEBUG    RFM2Pi     13210 Sent to channel(start)' : ToEmonCMS
2018-05-29 13:23:42,017 DEBUG    RFM2Pi     13210 Sent to channel(end)' : ToEmonCMS
2018-05-29 13:23:42,209 DEBUG    MQTT       Publishing: emon/emontx3/power1 581
2018-05-29 13:23:42,211 DEBUG    MQTT       Publishing: emon/emontx3/power2 562
2018-05-29 13:23:42,213 DEBUG    MQTT       Publishing: emon/emontx3/power3 3461
2018-05-29 13:23:42,216 DEBUG    MQTT       Publishing: emon/emontx3/power4 0
2018-05-29 13:23:42,218 DEBUG    MQTT       Publishing: emon/emontx3/vrms 262.73
2018-05-29 13:23:42,221 DEBUG    MQTT       Publishing: emon/emontx3/temp1 300
2018-05-29 13:23:42,224 DEBUG    MQTT       Publishing: emon/emontx3/temp2 300
2018-05-29 13:23:42,226 DEBUG    MQTT       Publishing: emon/emontx3/temp3 300
2018-05-29 13:23:42,228 DEBUG    MQTT       Publishing: emon/emontx3/temp4 300
2018-05-29 13:23:42,230 DEBUG    MQTT       Publishing: emon/emontx3/temp5 300
2018-05-29 13:23:42,233 DEBUG    MQTT       Publishing: emon/emontx3/temp6 300
2018-05-29 13:23:42,235 DEBUG    MQTT       Publishing: emon/emontx3/pulse 0
2018-05-29 13:23:42,238 DEBUG    MQTT       Publishing: emon/emontx3/rssi -81
2018-05-29 13:23:42,240 INFO     MQTT       Publishing: emonhub/rx/8/values 581,562,3461,0,262.73,300,300,300,300,300,300,0,-81
2018-05-29 13:23:45,042 DEBUG    emoncmsorg Buffer size: 3
2018-05-29 13:23:45,043 INFO     emoncmsorg sending: http://10.1.3.242/html/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1527600202.151188,8,581,570,3477,0,262.56,300,300,300,300,300,300,0,-80],[1527600212.070328,8,583,566,3463,0,263.14,300,300,300,300,300,300,0,-81],[1527600222.006624,8,581,562,3461,0,262.73,300,300,300,300,300,300,0,-81]]&sentat=1527600225
2018-05-29 13:23:45,086 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://10.1.3.242/html/emoncms
2018-05-29 13:23:51,852 DEBUG    RFM2Pi     13211 NEW FRAME : OK 8 69 2 47 2 142 13 0 0 166 102 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-80)
2018-05-29 13:23:51,855 DEBUG    RFM2Pi     13211 Timestamp : 1527600231.85
2018-05-29 13:23:51,857 DEBUG    RFM2Pi     13211 From Node : 8
2018-05-29 13:23:51,858 DEBUG    RFM2Pi     13211    Values : [581, 559, 3470, 0, 262.78000000000003, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:23:51,859 DEBUG    RFM2Pi     13211      RSSI : -80
2018-05-29 13:23:51,861 DEBUG    RFM2Pi     13211 Sent to channel(start)' : ToEmonCMS
2018-05-29 13:23:51,862 DEBUG    RFM2Pi     13211 Sent to channel(end)' : ToEmonCMS
2018-05-29 13:23:51,916 DEBUG    MQTT       Publishing: emon/emontx3/power1 581
2018-05-29 13:23:51,918 DEBUG    MQTT       Publishing: emon/emontx3/power2 559
2018-05-29 13:23:51,920 DEBUG    MQTT       Publishing: emon/emontx3/power3 3470
2018-05-29 13:23:51,922 DEBUG    MQTT       Publishing: emon/emontx3/power4 0
2018-05-29 13:23:51,924 DEBUG    MQTT       Publishing: emon/emontx3/vrms 262.78
2018-05-29 13:23:51,927 DEBUG    MQTT       Publishing: emon/emontx3/temp1 300
2018-05-29 13:23:51,929 DEBUG    MQTT       Publishing: emon/emontx3/temp2 300
2018-05-29 13:23:51,931 DEBUG    MQTT       Publishing: emon/emontx3/temp3 300
2018-05-29 13:23:51,933 DEBUG    MQTT       Publishing: emon/emontx3/temp4 300
2018-05-29 13:23:51,936 DEBUG    MQTT       Publishing: emon/emontx3/temp5 300
2018-05-29 13:23:51,938 DEBUG    MQTT       Publishing: emon/emontx3/temp6 300
2018-05-29 13:23:51,940 DEBUG    MQTT       Publishing: emon/emontx3/pulse 0
2018-05-29 13:23:51,942 DEBUG    MQTT       Publishing: emon/emontx3/rssi -80
2018-05-29 13:23:51,944 INFO     MQTT       Publishing: emonhub/rx/8/values 581,559,3470,0,262.78,300,300,300,300,300,300,0,-80
2018-05-29 13:24:01,750 DEBUG    RFM2Pi     13212 NEW FRAME : OK 8 67 2 47 2 152 13 0 0 190 102 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-82)
2018-05-29 13:24:01,753 DEBUG    RFM2Pi     13212 Timestamp : 1527600241.75
2018-05-29 13:24:01,754 DEBUG    RFM2Pi     13212 From Node : 8
2018-05-29 13:24:01,755 DEBUG    RFM2Pi     13212    Values : [579, 559, 3480, 0, 263.02, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:24:01,756 DEBUG    RFM2Pi     13212      RSSI : -82
2018-05-29 13:24:01,757 DEBUG    RFM2Pi     13212 Sent to channel(start)' : ToEmonCMS
2018-05-29 13:24:01,758 DEBUG    RFM2Pi     13212 Sent to channel(end)' : ToEmonCMS
2018-05-29 13:24:01,846 DEBUG    MQTT       Publishing: emon/emontx3/power1 579
2018-05-29 13:24:01,848 DEBUG    MQTT       Publishing: emon/emontx3/power2 559

pb66 · 29 May 2018 13:35

Each time a runtimesetting is changed it gets applied as the emonhub.conf gets saved, this should produce a log entry for the change. This will tell us if the emonhub.conf changes are being picked up or not.

spallared · 29 May 2018 13:51

I changed the interval to 15 seconds and saved the conf. This log was generated:

2018-05-29 13:50:20,160 DEBUG    emoncmsorg Buffer size: 3
2018-05-29 13:50:20,161 INFO     emoncmsorg sending: http://10.1.3.242/html/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1527601796.914646,8,467,588,3119,0,260.83,300,300,300,300,300,300,0,-80],[1527601806.8816,8,468,553,3109,0,260.84000000000003,300,300,300,300,300,300,0,-81],[1527601816.7615,8,467,550,3117,0,260.38,300,300,300,300,300,300,0,-80]]&sentat=1527601820
2018-05-29 13:50:20,205 DEBUG    emoncmsorg acknowledged receipt with 'ok' from http://10.1.3.242/html/emoncms
2018-05-29 13:50:23,058 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)
2018-05-29 13:50:23,160 DEBUG    RFM2Pi     acknowledged command: > 1p
2018-05-29 13:50:23,468 DEBUG    RFM2Pi     acknowledged command: <nn> i     - set node ID (standard node ids are 1..30)
2018-05-29 13:50:23,573 DEBUG    RFM2Pi     acknowledged command: <n> b      - set MHz band (4 = 433, 8 = 868, 9 = 915)
2018-05-29 13:50:23,679 DEBUG    RFM2Pi     acknowledged command: <nnnn> o   - change frequency offset within the band (default 1600)
2018-05-29 13:50:23,889 DEBUG    RFM2Pi     acknowledged command: <nnn> g    - set network group (RFM12 only allows 212, 0 = any)
2018-05-29 13:50:23,994 DEBUG    RFM2Pi     acknowledged command: <n> c      - set collect mode (advanced, normally 0)
2018-05-29 13:50:24,203 DEBUG    RFM2Pi     acknowledged command: ...,<nn> a - send data packet to node <nn>, request ack
2018-05-29 13:50:24,308 DEBUG    RFM2Pi     acknowledged command: ...,<nn> s - send data packet to node <nn>, no ack
2018-05-29 13:50:24,414 DEBUG    RFM2Pi     acknowledged command: <n> q      - set quiet mode (1 = don't report bad packets)
2018-05-29 13:50:24,520 DEBUG    RFM2Pi     acknowledged command: <n> x      - set reporting format (0: decimal, 1: hex, 2: hex+ascii)
2018-05-29 13:50:24,626 DEBUG    RFM2Pi     acknowledged command: <nnn> y    - enable signal strength trace mode, default:0 (disabled)
2018-05-29 13:50:24,733 DEBUG    RFM2Pi     acknowledged command: sample interval <nnn> secs/100 (0.01s-2.5s) eg 10y=0.1s
2018-05-29 13:50:25,045 DEBUG    RFM2Pi     acknowledged command: <hchi>,<hclo>,<addr>,<cmd> f     - FS20 command (868 MHz)
2018-05-29 13:50:25,150 DEBUG    RFM2Pi     acknowledged command: <addr>,<dev>,<on> k              - KAKU command (433 MHz)
2018-05-29 13:50:25,356 DEBUG    RFM2Pi     device settings updated: E i5 g210 @ 433 MHz q1
2018-05-29 13:50:26,667 DEBUG    RFM2Pi     13376 NEW FRAME : OK 8 209 1 45 4 40 12 0 0 61 101 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-80)
2018-05-29 13:50:26,670 DEBUG    RFM2Pi     13376 Timestamp : 1527601826.67
2018-05-29 13:50:26,671 DEBUG    RFM2Pi     13376 From Node : 8
2018-05-29 13:50:26,672 DEBUG    RFM2Pi     13376    Values : [465, 1069, 3112, 0, 259.17, 300, 300, 300, 300, 300, 300, 0]
2018-05-29 13:50:26,672 DEBUG    RFM2Pi     13376      RSSI : -80
2018-05-29 13:50:26,673 DEBUG    RFM2Pi     13376 Sent to channel(start)' : ToEmonCMS

pb66 · 29 May 2018 14:17

I’m guessing that log is incomplete since even an incorrect setting should log an error if the settings are applied and the settings must be getting applied to trigger the

2018-05-29 13:50:23,058 INFO     MainThread Setting RFM2Pi calibration: 230V (1p)

line (which is in turn triggering a response containing a commands menu since it is not recognised by what I think is an RFM?Pi rather than an emonPi), but that’s unrelated to your issue.

Ok so looking at the code for the emonpi variant of emonhub it seems the setting is not “sendinterval” but just “interval” despite the default conf saying otherwise.

Try

         interval = 10

although you should understand that this is the “minimum time between sends” to limit the traffic when you have many nodes. Your 10s data could theoretically still be delayed for up to 10s. If you have multiple 10s nodes they would be grouped and sent every 10s. Setting 'interval = 0` removes any limitations and will just send your data in realtime as it is processed, only if there is network or server issues will the data be buffered.

spallared · 29 May 2018 14:27

Thanks!
“interval” did the trick, i renamed it and set it to 0 and now everything is ok.

Thanks again
Luca

pb66 · 29 May 2018 14:31

Thanks for confirming that fixed it.

Looks like this was changed in January this year (Revert Emoncms HTTP Interfacer to use emonhub buffer approach develop… · openenergymonitor/emonhub@65278b2 · GitHub) but the conf wasn’t updated.

PR submitted

glyn.hudson · 7 December 2018 21:24

Fix merged.

Thank a lot for your help. Very sorry for the delay.

It should be noted that merging this fix will not update users emonhub.conf even if they update emonhub. It will only update the default emonhub.conf. If emonpi users have not made any custom changes to their emonhub.conf they can pull in this fix and to overwrite their current emonhub.conf by doing:

cd emonhub
git pull
cp emonhub.conf ~/data/emonhub.conf

Or just manually rename sendinterval= 30 to interval=30 in the [[emoncmsorg]] section of emonhub.conf.