Reinstall and data import, incorrect readings

Emoncms
1

I had the emonpi working perfectly. Then had an extension built and had the device offline for 6 months. On switching it on, it stalled always at one point. Turned out to be a damaged SDCard. So I had to reinstall the device and then import the old data. Unfortunately, the feeds are reading odd values. This is at night when solar is zero.

use PHPFINA n/a -24248 W
use_kwh PHPFINA n/a 3021 kWh
solar PHPFINA n/a 3 W
solar_kwh PHPFINA n/a -9.17 kWh
import PHPFINA n/a 0 W
import_kwh PHPFINA n/a 7430 kWh

I donā€™t get why the ā€˜useā€™ is now -24248 W

My house is certainly not using -24KW

Any thoughts as to what is wrong would be appreciated.

2

Hi, welcome,

Were the CTs removed during the build. Looks like they are on the wrong way round.

3

I did check this, by following the Solar Type 2 setup. Anyway, Iā€™ll switch them tonight and report back. Thank you.

1 Like
4

24kW does seem rather excessive use though (if the wrong way round)!

Have you shutdown completely, removed the power (both the 5V and the 9V ā€˜sensorā€™), unplug the CTs and reinsert, then power back up?

What would you expect the readings to be?

Have a look at the Inputs page - do they look right? It could be something to do with the long gap. If the Inputs look right, create a new Feed. You can just add a new ā€˜Log to Feedā€™ entry in the Input Processing (next to the old ā€˜Log To Feedā€™).

5

Without looking at the code, I think it will draw a straight line between the last power reading of the last batch and the first reading of the present batch, and integrate the energy (kWh) using that ā€˜straight-lineā€™ power.

The code used to be here in your emonBase/emonPi. I donā€™t know if this is still correct:: /var/www/emoncms/Modules/process/process_processlist.php

Hereā€™s something that looks relevant:

        // only update if last datapoint was less than 2 hour old
        // this is to reduce the effect of monitor down time on creating
        // often large kwh readings.
        $time_elapsed = ($time_now - $last_time);   
        if ($time_elapsed>0 && $time_elapsed<7200) { // 2hrs
            // kWh calculation
            $kwh_inc = ($time_elapsed * $value) / 3600000.0;
            $new_kwh = $last_kwh + $kwh_inc;
        } else {
            // in the event that redis is flushed the last time will
            // likely be > 7200s ago and so kwh inc is not calculated
            // rather than enter 0 we enter the last value
            $new_kwh = $last_kwh;
        }

I think this is saying you might have filled the gap with the final value before the shut-down.

6

ok. I think I am going to start from scratch and rebuild it. It all went wrong after the supplied sdcard corrupted. Then I rebuilt it and imported the data. After that feeds and such went n/a. Let me format and rebuild from the latest ISO.

If I can get this fully operational on a fresh install and then make a copy and import to that, hopefully we can get back to a working system.

Iā€™ll let you know.

Thank you,

Neil.

7

Iā€™ve reinstalled and then re-imported. It looks better but itā€™s still not right.

I can see that power1andpower2 is showing actual realtime usage. Everything else does not appear to be right.

The readings are as follows, at night.

use PHPFINA 20MB 19844 W
use_kwh PHPFINA 20MB 2764 kWh
solar PHPFINA 20MB 0 W
solar_kwh PHPFINA n/a 0 kWh
import PHPFINA 20MB 19844 W 8s
import_kwh PHPFINA 20MB 2099 kWh

Use just keeps adding up and never goes down. Use and Import seen to be the same value. Solar has stayed at zero for a few days now.

power1 19867
power2 0
power1pluspower2 432
vrms 0.12
t1 44.4
t2 2503

Any thoughts?

8

@TrystanLea

9

From what I can determine, it appears that the inputs are mixed up and getting the wrong data. I.e. VRMS is not showing the voltage but some other figure.

10

Hello @Neil_Hack, could you share a screenshot of your emonPi inputs? and if you go to Setup > Emonhub > Edit config, the entry that you can see in the nodes section for the emonPi?

Do you just have an emonPi there or do you have other nodes as well that the emonPi is receiving via radio?

Are you now running the latest emonSD image?

11

Hi Trystan,

Here is everything.

emonpiRelease - emonSD-21Jul21 (Latest version does the same except the demandshaper is not installed)

Response from emonhub
2023-06-24 11:41:47,985 DEBUG RFM2Pi 98 NEW FRAME : OK 5 101 0 0 0 119 254 133 3 252 1 41 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 132 219 253 255 167 87 11 0 (-0)
2023-06-24 11:41:47,988 DEBUG RFM2Pi 98 Timestamp : 1687603307.985073
2023-06-24 11:41:47,988 DEBUG RFM2Pi 98 From Node : 5
2023-06-24 11:41:47,989 DEBUG RFM2Pi 98 Values : [101, 0, -393, 9.01, 50.800000000000004, 2487.3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -9340, -3, 22439, 11]

image.png
image.png888Ɨ415 16.6 KB

image.png
image.png870Ɨ433 14.4 KB

Actual Solar right now, while writing this response is 3.47KW

It almost feels like, to me, that the inputs CT1 CT2 are feeding incorrectly or in a non-standard way.

Thank you,

Neil.

12

Not quite. I think Trystan will also need sight of the contents of emonhub.conf, because as you say, the numbers for power1, power2 and power1pluspower2 should add up and they donā€™t. Also, 51.2 and 2490 for t1 & t2 donā€™t look sensible. It looks to me as if the raw values (` OK 5 101 0 0 0 119 254 133 3 ā€¦ etc) are out of step with the decoding instructions in emonhub.conf.

13

I appreciate the help. Do you need the while thing or just the emonpi section?

14

Itā€™s safer to send the whole file so we have the whole picture, just in case thereā€™s a duplicate or something silly like that.

15

Full File.

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 115200                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

        group = 210
        frequency = 433
        baseid = 5                              # emonPi / emonBase nodeID
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        quiet = true                            # Disable quite mode (default enabled) to enable RF packet 'debugging, show packets which fail crc
        # interval =  300                         # Interval to transmit time to emonGLCD (seconds)


[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/

        # Single JSON payload published  - use with Emoncms MQTT
        node_JSON_enable = 0
        node_JSON_basetopic = emon/

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        url = https://emoncms.org
        apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        senddata = 1                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > 'https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: https://github.com/openenergymonitor/emonhub/blob/emon-'pi/conf/emonhub.conf

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
##        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units = W,W,W,W,V

[[7]]
   nodename = emontx4
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, 'pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
    nodename = emontx3
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, 'pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[9]]
   nodename = emontx2
   [[[rx]]]
      names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, 'pulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units = W,W,W,W,V,C,C,C,C,C,C,p

[[10]]
    nodename = emontx1
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, 'pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[11]]
    nodename = 3phase
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5,temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[12]]
    nodename = 3phase2
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[13]]
    nodename = 3phase3
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[14]]
    nodename = 3phase4
    [[[rx]]]
       names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.01,0.01,0.01,0.01,0.01,0.01,1
       units = W,W,W,W,V,C,C,C,C,C,C,p

[[15]]
    nodename = emontx3cm15
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[16]]
    nodename = emontx3cm16
    [[[rx]]]
       names = MSG, Vrms, P1, P2, P3, P4, E1, E2, E3, E4, T1, T2, T3, pulse
       datacodes = L,h,h,h,h,h,l,l,l,l,h,h,h,L
       scales = 1,0.01,1,1,1,1,1,1,1,1,0.01,0.01,0.01,1
       units = n,V,W,W,W,W,Wh,Wh,Wh,Wh,C,C,C,p
       whitening = 1

[[19]]
   nodename = emonth1
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[20]]
   nodename = emonth2
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[21]]
   nodename = emonth3
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[22]]
   nodename = emonth4
   [[[rx]]]
      names = temperature, external temperature, humidity, battery
      datacode = h
      scales = 0.1,0.1,0.1,0.1
      units = C,C,%,V

[[23]]
    nodename = emonth5
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[24]]
    nodename = emonth6
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[25]]
    nodename = emonth7
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[[26]]
    nodename = emonth8
    [[[rx]]]
       names = temperature, external temperature, humidity, battery, pulsecount
       datacodes = h,h,h,h,L
       scales = 0.1,0.1,0.1,0.1,1
       units = C,C,%,V,p

[edited by mod - please use CTL-E to paste Code etc.]

16

The single quote in front of ā€œpulseā€ (many instances) is not right, but I donā€™t think it will be interfering with Node 5. What is wrong there is the ā€˜datacodesā€™ line is commented out, meaning that everything is treated as a signed integer (h); besides which, the count of bytes of data doesnā€™t fit the names. This is no doubt your problem.

Comment out the whole of your Node 5, and substitute this:

[[5]]
     nodename = emonpi
     [[[rx]]]
         names = Msg, power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount,pulsecount2,E1,E2
         datacodes = L, h, h, h, h, h, h, h, h, h, h, L, L, l, l
         scales = 1, 1,1,1, 0.01, 0.01,0.01,0.01,0.01,0.01,0.01, 1, 1, 1,1
         units = n,W,W,W, V, C,C,C,C,C,C, p, p, Wh,Wh

ā€œMsgā€ is a serial number for the radio message - allows you to know when one (or many) goes missing. There are two pulse counts, you will only see the first. E1 and E2 are the accumulated energies from power1 & power2.

With luck, youā€™ll get sensible numbers. E1 & E2 should track the kWh values that emonCMSā€™s processing generates (but in Wh).

18

Luck is good. So things look normal now. I have numbers 12 to 20 in the inputs where the timer is just going up, so suspect I can simply delete those.

I am happy itā€™s working. Donā€™t get why a fresh installation simply does not work. For your question, I commented out the datacodes because it was causing an error in the emonhub. All the inputs and feeds were showing N/A. When I commented it out, it was able to read data, albeit incorrectly. I read that in the community somewhere.

At least now I can work on the energy use and the solar generation and fine tune it so that it matches the inverter, or as close as can be.

Iā€™m really grateful it is working now.

19

I think there is a bug in the way the emonhub config is imported.

20

That was telling you the data coming in wasnā€™t what the config file expected.

because the default decoding didnā€™t fit the data.

Because it inherited your old emonhub.conf. Although the emonPi itself has changed the format of the data it was sending, everything else coming in from outside needed your old emonhub.conf
It would need some fairly serious code to automatically edit emonhub.conf to replace the Node 5 section (or whichever node your emonPi is).

21

Thanks for all your help. Everything seems fine now.