Obsolete Values

Emoncms
61 
#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    #Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet debugging, show packets which fail crc
        # interval =  0                         # Interval to transmit time to emonGLCD (seconds)

[[ArduinoOnUSB0]]
        Type = EmonHubSerialInterfacer
        [[[init_settings]]]
            com_baud = 9600
            com_port = /dev/ttyUSB0
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12, 
        

[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = xxxxxxxxxxxxxxxxxxxxxxxxx
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-pi/conf/emonhub.conf

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

[[6]]
    nodename = ArduinoOnUSB0
    [[[rx]]]
       names = power1, power2, power3, power4
       datacode = 0
       scales = 1,1,1,1
       units =W,W,W,W

It’s configured like that

Thanks for answering

62

Can you post your latest emonHub log?

63

Here is my actual flux of emonHub log :

2019-07-09 07:46:19,068 DEBUG    ArduinoOnUSB0 152 Sent to channel(start)' : ToEmonCMS
2019-07-09 07:46:19,068 DEBUG    ArduinoOnUSB0 152 Sent to channel(end)' : ToEmonCMS
2019-07-09 07:46:19,083 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power1 4
2019-07-09 07:46:19,084 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power2 4
2019-07-09 07:46:19,085 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power3 18
2019-07-09 07:46:19,086 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power4 6
2019-07-09 07:46:19,087 INFO     MQTT       Publishing: emonhub/rx/6/values 4,4,18,6
2019-07-09 07:46:19,170 DEBUG    ArduinoOnUSB0 153 NEW FRAME : 6 4 4 17 7 
2019-07-09 07:46:19,172 DEBUG    ArduinoOnUSB0 153 Timestamp : 1562658379.17
2019-07-09 07:46:19,173 DEBUG    ArduinoOnUSB0 153 From Node : 6
2019-07-09 07:46:19,173 DEBUG    ArduinoOnUSB0 153    Values : [4, 4, 17, 7]
2019-07-09 07:46:19,174 DEBUG    ArduinoOnUSB0 153 Sent to channel(start)' : ToEmonCMS
2019-07-09 07:46:19,175 DEBUG    ArduinoOnUSB0 153 Sent to channel(end)' : ToEmonCMS
2019-07-09 07:46:19,277 DEBUG    ArduinoOnUSB0 154 NEW FRAME : 6 4 5 18 5 
2019-07-09 07:46:19,279 DEBUG    ArduinoOnUSB0 154 Timestamp : 1562658379.28
2019-07-09 07:46:19,280 DEBUG    ArduinoOnUSB0 154 From Node : 6
2019-07-09 07:46:19,280 DEBUG    ArduinoOnUSB0 154    Values : [4, 5, 18, 5]
2019-07-09 07:46:19,281 DEBUG    ArduinoOnUSB0 154 Sent to channel(start)' : ToEmonCMS
2019-07-09 07:46:19,282 DEBUG    ArduinoOnUSB0 154 Sent to channel(end)' : ToEmonCMS
2019-07-09 07:46:19,294 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power1 4
2019-07-09 07:46:19,296 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power2 4
2019-07-09 07:46:19,297 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power3 19
2019-07-09 07:46:19,299 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power4 5
2019-07-09 07:46:19,300 INFO     MQTT       Publishing: emonhub/rx/6/values 4,4,19,5
2019-07-09 07:46:19,384 DEBUG    ArduinoOnUSB0 155 NEW FRAME : 6 4 4 18 5 
2019-07-09 07:46:19,386 DEBUG    ArduinoOnUSB0 155 Timestamp : 1562658379.38
2019-07-09 07:46:19,387 DEBUG    ArduinoOnUSB0 155 From Node : 6
2019-07-09 07:46:19,388 DEBUG    ArduinoOnUSB0 155    Values : [4, 4, 18, 5]
2019-07-09 07:46:19,388 DEBUG    ArduinoOnUSB0 155 Sent to channel(start)' : ToEmonCMS
2019-07-09 07:46:19,389 DEBUG    ArduinoOnUSB0 155 Sent to channel(end)' : ToEmonCMS

LOG LEVEL: DEBUG

EDIT1: I’m also wondering how can i manage to have emonhub always refreshing itself when i’m varying the DC and the voltage because i always need to restart Emonhub to see a changement on my values (which are coherent from what i’m doing) . Does a way to don’t have to restart emonhub exist ?

64

That’s quite a small sample log to work with but if you look at these 2 blocks

2019-07-09 07:46:19,083 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power1 4
2019-07-09 07:46:19,084 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power2 4
2019-07-09 07:46:19,085 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power3 18
2019-07-09 07:46:19,086 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power4 6
2019-07-09 07:46:19,087 INFO     MQTT       Publishing: emonhub/rx/6/values 4,4,18,6

 . . . 

2019-07-09 07:46:19,294 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power1 4
2019-07-09 07:46:19,296 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power2 4
2019-07-09 07:46:19,297 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power3 19
2019-07-09 07:46:19,299 DEBUG    MQTT       Publishing: emon/ArduinoOnUSB0/power4 5
2019-07-09 07:46:19,300 INFO     MQTT       Publishing: emonhub/rx/6/values 4,4,19,5

they are the values being published by emonHub to emoncms via MQTT and as you can see the third and fourth values have changed.

The problem could well be the speed that your sketch is producing new values. Those block above are just 211 milliseconds apart.

If you look at the raw data coming into emonhub

2019-07-09 07:46:19,170 DEBUG    ArduinoOnUSB0 153 NEW FRAME : 6 4 4 17 7 
 . . . 
2019-07-09 07:46:19,277 DEBUG    ArduinoOnUSB0 154 NEW FRAME : 6 4 5 18 5 
 . . . 
2019-07-09 07:46:19,384 DEBUG    ArduinoOnUSB0 155 NEW FRAME : 6 4 4 18 5

you can see the new data frames are only 107 milliseconds apart and the actual sample values do not match the out going sample values from the same log period, I would suspect there is a backlog queued due to the incoming samples being too fast for emonhub to publish to so many different topics and keep up.

If you used (for example) a http interfacer to send data to emoncms instead of MQTT it might stand more chance of keeping up due to the bulk upload ability.

However! Since emoncms has a maximum resolution of 1sec, there is no point in sending any faster than 5 or 10 sec intervals since emoncms will round the timestamps and you will get very inaccurate results. The norm is to use 10s updates.

There is no reason to restart emonhub, the data is changing, just not as you expect it to. You need to slow down your sketch, either let it loop rapidly gathering data and then calculate averages every 10s to print, or add a sleep/timer function to only collate the data once every 5-10s. (or somewhere inbetween? loop every 1s and send every 5s?)

65

I think i’m understanding what you said @pb66 but it seems that the changement that i want is very different from what it appears : the LOG shows that the value is changing from 18 to 19 ( i was puting a certain voltage and a certain current at this moment but when i set my current and my voltage to 0 , no matter the time i’m waiting , the value doesn’t decrease , it decreases to the good value only when i restart emonhub.

EDIT1 : In emoncms.org , the inputs don’t update themselves and i always need to restart emonhub.

66

you are clearing the queue by restarting emonhub.

It’s not easy to tell from the very small snippet of log, but i would suspect that since the outgoing packet interval is around twice that of the incoming packets, you would need to wait 2x the uptime, for example if your sketch and emonhub have been running 10mins, then it will take 20mins to see any change you make published to mqtt.

Then, if you are only viewing the data in emoncms inputs you are only seeing part of the data, the inputs will be updated every 0.2s, but the page will not refresh that fast.

Look at the full on-disk /var/log/emonhub/emonhub.log file in more detail. You will see the raw data values will be updating. Please understand that the “NEW FRAME :” lines in the log are the raw serial data, what that says is what is coming from your device. If those values are not changing as you expect, I would put good money on the values being printed wrong by your sketch. Assuming of course that the right device is being connected to etc.

Remember what I said in your other thread about the USB devices not being tied to the addresses. /dev/ttyUSB0 and /dev/ttyUSB1 can swap, so is node 8 definitely the device you are changing the voltage to?

emoncms.org has input limiting and will discard anything that is less than 10s since the last update for any given node.

67

I’m only using the /dev/ttyUSB0 at the moment and it’s the device that i’m changing the voltage.

I was using the Arduino monitor to test printing my values and it works very well and it vary almost perfectly when i change the voltage and the current.

EDIT1 :

I’m not sure understanding that

Thanks for responding

68

In fact. SSH into your emonPi and issue this command

tail -f /var/log/emonhub/emonhub.log

now watch that console window whilst varying the voltage to your device, pay particular attention to the “NEW FRAME :” lines. In fact, at first just use this command instead

tail -f /var/log/emonhub/emonhub.log | grep "NEW FRAME"

This should update rapidly and show any changes in values pretty quickly, although there may be some small delay due to the loading of emonhub.

69

I need to put this command on my RPI3 ? (using Putty ?)

70

Ok, that’s good!

In that case the “tail tests” above should show a similar result. I wasn’t saying that there definitely was a issue with your sketch, I was making the point that if the raw values in emonhub are wrong it must be you sketch. But I don’t think you are seeing the raw values, you are seeing old data that has been queuing or missing the data due to emoncms input refreshing/limiting.

The hosted emoncms.org service has traffic limits. When an input is updated (eg node 8) anything else that arrives for that same node within the next 10secs will be discarded without being processed or displayed.

71

yes that’s correct.

72

It seems that at the moment the values don’t vary regardless of the current and voltage i put .

EDIT1 : the command that you gave me stopped itself without touching anything ( but the values are all the same). So I’m wondering why when i was restarting emonhub , the “coherent value” appeared.

EDIT2: I tested the command but before , i’ve put a certain voltage , it tooks time but the value are changing(but in my emonhub , they are same as before). So now , another question why it does’nt work when i change the voltage during testing the command .

EDIT3 : I just want to read in real time my values ( even if there is a short time interval) . It seems that the values are refreshing but it’s too long for me , it’s very very long.

73

So what is the current status? Where are you seeing the values changing?

As Robert has already said to you, ignore emoncms for the moment.

Are the values updating with the tail command? If that is slow, try temporarily disabling the mqtt interfacer by adding a hash (#) to comment out the “Type=” line in emonhub.conf eg

[[MQTT]]

    #Type = EmonHubMqttInterfacer
    [[[init_settings]]]

    . . .

This will free up emonhub to just handle the inputs without being slowed down by so much publishing. The data will not reach emoncms, but it should now be faster with the tail command (as a test step).

To use emoncms or MQTT you will have no choice but to slow down your sketch, emonhub may keep up if you configure it carefully, but neither emonhub or emoncms will be able to process 0.1s intervals, you will need to write your own software or find an alternative.

[edit] Perhaps you should just look ar using node-red to read the serial port directly and then write to file? i used to have a emonhub interfacer that wrote csv direct to file, but that was for the original version of emonhub and wouldn’t work with the emonpi variant.

74

I don’t know how to slow down my sketch like you’ve said :

I don’t know how to do that :confused:

Thanks for taking your time to answer

75

first off just start with a test, add a “delay(4990)” line inside the loop, that will stop the loop for just under 5s (4990ms) each loop.

This should allow all the parts to play nicely together, ie you should see emonhub processing those 5s updates and emoncms displaying those same updates.

This proves the system is all ok when the updates are slower.

From there you can change your sketch to only sleep 1s, sub-totalling the values on each loop, then every 5th loop it can divide each value by 5 and send it (for example).

76

In fact, depending on your range of possible values, you could probably just send the sub-totals and then divide the values in emonhub using the scales= setting.

eg if you (for example) sleep for 1 sec and then send the sub-total every 5secs, set the emonhub “scales=” values to 0.2 and emonhub will divide by 5 for you, giving you a 5s average derived from the 5x 1s samples.

The time you spend asleep should accommodate the time it takes to loop, eg if you sleep 1s then the loop interval is 1 second plus the time it takes to execute the code in the loop.

But this is something you can refine once you have the basic operation working. The normal way to get accurate loops is to use a “if time now = last time + 1000” type arrangement, but if you start simple and test at every change, you have more control over what you are doing rather than piling in the changes and then your lost when it doesn’t work as expected first time.

77

I was thinking about doing an average of the values received during like 10sec but i have no idea how can i do that and what i have to change in my sketch.

78

As I’ve explained above, you will need to sub-total each value for each loop and then divide by the number of loops to get the average. You can either do that in the sketch, which adds complexity and potentially a time bottle-neck, depending on loop speed and how/when it’s done. Or you can just let emonhub derive the averages from the sub-totals using the “scales=” function, this makes the sketch simpler and off-loads some work to emonhub.

79

How can i do that ? Is it on emonhub config ?

Thanks for all your answers

80 
[[6]]
    nodename = ArduinoOnUSB0
    [[[rx]]]
       names = power1, power2, power3, power4
       datacode = 0
       scales = 0.2,0.2,0.2,0.2     # divide by 5 as received values are sub-totals for 5 samples
       units =W,W,W,W

The “scales=” values will be 1/number_of_values_in_subtotal, eg 1/5 = 0.2 if 5 samples are subtotalled or 1/10 = 0.1 if 10x samples are subtotalled etc