Rename emonTX node name

toml · 2017-11-25T16:52:53.547Z

Hi,

My emonPI had the following default config file for an emonTX:

[[7]]
    nodename = emontx4
    [[[rx]]]
       names = power1, power2, power3, power4, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse
       datacodes = h,h,h,h,h,h,h,h,h,h,h,L
       scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
       units =W,W,W,W,V,C,C,C,C,C,C,p

I wanted to rename the nodename and names, so I updated the emonPI config to the following:

[[7]]
   nodename = emonElectricity1
   [[[rx]]]
names = sockets, lights, cooker, shower, vrms, temp1, temp2, temp3, temp4, temp5, temp6, pulse

Which results in this being posted to the CMS:

Any ideas?

Obviously I would have expected the node name to be ‘emonElectricity1’ with the field names as defined in names, but for some reason I only get the above.

My log is saying it is publishing the correct field names so not sure why this is happening, the emonCMS I am viewing on is the local one to emonPI.

Thanks

Robert.Wall · 2017-11-25T17:38:33.843Z

This has got to be one for Paul. In the meantime, is it the length of “emonElectricity1” that is causing a problem?

toml · 2017-11-25T18:01:19.381Z

Not sure it is the name as I tried simply naming it ‘emontxa’ rather than ‘emontx4’

My use case gets worse… I need groups of nodes, and some way to identify which group they are in, the groups are per floor so I extended the base topic like this emon/thefloor

I managed to get a workaround working for now using node red, but this obviously does not fix the local logging issue

Here is the config to give a better idea what im doing, imagine this same file on multiple emonPI’s on different floors posting to the same API’s and CMS’s:

#######################################################################
#######################      emonhub.conf     #########################
#######################################################################
### emonHub configuration file, for info see documentation:
### http://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md
#######################################################################
#######################    emonHub  settings    #######################
#######################################################################

[hub]
### loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
loglevel = DEBUG
### Uncomment this to also send to syslog
# use_syslog = yes
#######################################################################
#######################       Interfacers       #######################
#######################################################################

[interfacers]
### This interfacer manages the RFM12Pi/RFM69Pi/emonPi module
[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyAMA0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12,

        group = 210
        frequency = 433
        baseid = 1                             # emonPi / emonBase nodeID
        quiet = true                            # Report incomplete RF packets (no implemented on emonPi)
        calibration = 230V                      # (UK/EU: 230V, US: 110V)
        # interval =  0                         # Interval to transmit time to emonGLCD (seconds)

[[MQTT]]

    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = DELETED

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format
        # Use with emoncms Nodes module
        node_format_enable = 1
        node_format_basetopic = emonhub/ground/

        # emon/emontx/power1 format - use with Emoncms MQTT input
        # http://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md
        nodevar_format_enable = 1
        nodevar_format_basetopic = emon/ground/

[[emoncmsorg]]
    DELETED SECTION FOR FORUM POST

#######################################################################
#######################          Nodes          #######################
#######################################################################

[nodes]

## See config user guide: http://github.com/openenergymonitor/emonhub/blob/master/configuration.md

#######################          Base Node        #######################
[[1]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

#######################          Power Node 5-9       #######################

[[5]]
   nodename = powerNode1
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[6]]
   nodename = powerNode2
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[7]]
   nodename = powerNode3
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[8]]
   nodename = powerNode4
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[9]]
   nodename = powerNode5
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p


#######################          Boiler Node 10-14       #######################
[[10]]
   nodename = boilerNode1
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[11]]
   nodename = boilerNode2
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[12]]
   nodename = boilerNode3
   [[[rx]]]
      names = powerOverall, powerCooker , power3, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[13]]
   nodename = boilerNode4
   [[[rx]]]
      names = powerOverall, powerCooker , power3, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[14]]
   nodename = boilerNode5
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

#######################          Gas Node 15-24      #######################

[[15]]
   nodename = gasNode1
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[16]]
   nodename = gasNode2
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[17]]
   nodename = gasNode3
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[18]]
   nodename = gasNode4
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[19]]
   nodename = gasNode5
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[20]]
   nodename = gasNode6
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[21]]
   nodename = gasNode7
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[22]]
   nodename = gasNode8
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[23]]
   nodename = gasNode9
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

[[24]]
   nodename = gasNode10
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

pb66 · 2017-11-25T18:34:16.161Z

Very briefly (I can come back to it later, but this is a flying visit right now) when you originally renamed the input in emonhub.conf, emoncms will have seen this as a new input. There is a known issue with the mqtt input on emoncms that doesn’t see newly created inputs correctly.

Restart the mqtt_input service with

sudo service mqtt_input restart

if you can get to the command line via ssh and your newly named devices should pop up. Alternatively you can just reboot the emonpi instead.

I’m unsure how it is that you have this configured, since you have the emoncms.org config “deleted for forum” so that suggests you are posting remotely by http and locally via mqtt, where do all these converge? I’m wondering if it even matters if the local inputs are not seen if you are also posting to a single server via http. Are you really going to run an emoncms server instance for each floor?

The mqtt link between emonhub and emoncms isn’t very flexible, I think you are going to struggle to split the floors into different branched topics and get emoncms to subscribe to them as well. You can publish to emon/ground/ but then you need to get emoncms to subscribe to emon/ground/ and it will then miss all the other floors topics.

toml · 2017-11-25T18:43:57.672Z

Ah ok ill try what you say about restarting mqtt and see how that goes, here is the part I deleted from the above config file, i have removed the URL and API key though

[[emoncmsorg]]
    Type = EmonHubEmoncmsHTTPInterfacer
    [[[init_settings]]]
    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        senddata = 0                    # Enable sending data to Emoncms.org
        sendstatus = 1                  # Enable sending WAN IP to Emoncms.org MyIP > https://emoncms.org/myip/list
        sendinterval= 30                # Bulk send interval to Emoncms.org in seconds

To answer your question about floors, the reason it is by floor is for two reasons, the first is range/building structure, anything on one floor cant be transmitted to another floor because it simply does not reach.

The other reason is node limit, we are deploying just over 30 nodes so we are going to have to split them up anyway, I thought it was a good idea to just split them per floor as well, thats all that can be reached anyway.

In terms of logging, all floors will get posted to a remote CMS, combined if you like. currently, on my test setup, I am prepending the floor name onto the node name in node red.

The local logging is not working though, neither is remote logging (using .config) so I have that off, but id like to use it so I can log in locally and view whats happening a little easier.

Please feel free to share any ideas you have around how I have this setup if you think you have a better idea, this will be my first deployment of OOM, so I am sure it could be improved.

Thanks

toml · 2017-11-25T19:10:20.390Z

Paul, your advice fixed it, thanks for your help.

pb66 · 2017-11-26T11:21:29.999Z

toml:

Paul, your advice fixed it, thanks for your help.

No problem.

toml:

here is the part I deleted from the above config file, i have removed the URL and API key though

Sorry my hasty reply wasn’t very clear. I wasn’t actually asking for the details, I was merely stating I had taken the fact you had removed sensitive info to indicate you were using that route, when I was trying to visualise your setup.

For future info, when getting support, you are better off fudging the details rather than removing because you may also delete a formatting issue or something, eg change an domain name to “myserver.com” for example and replace your apikey with “xxxxxx”, “abc123” or “MYEMONCMSWRITEAPIKEY” etc, this is especially true with url request formation issues etc.

As for your proposed set up still a little foggy on some detail.

Splitting the 30’ish nodes up to separate emonBases is a wise idea, depending on the makeup of the building, by floor may be ok if the floors are concrete or the building is very tall (high ceilings) with a small area per floor, otherwise you might find that the signal strength between devices might be stronger in the room above or below a device, than it is at the other end of the same floor, especially if there are several solid walls on that floor. I think you would be better off experimenting with a couple of devices and laying out the system based on performance rather than a logical area map, unless you have specific reasons to divide it a cetain way, eg different tenants on each floor not wanting to share info.

Usually there is a desire to have all this info in one place, it would be easier to set up and maintain if that were true, but if you need separate accounts that can be done either way.

I’d be happy to help you formulate a plan of attack, although I would need more info to do so. There were a few points made in the “Setup multiple emontx's” thread too. Did you give any thought to my question about using some serially connected devices?

Is this a commercial building or domestic? Do you occupy all the floors or are they sublet? What is the approximate area and are there many walls to negotiate? what type of walls are they?

Normally, emonTx’s are located at the dist board or sub-boards, is that the case here? The fact you need so many emonT’s and they are spread over a large area suggests this might be a commercial install, in which case, are there any machines or other sources of interference to hamper RF signals?

Can you loosely describe the power layout of the building? eg is there a dist board for each floor? are they in plant rooms that are directly above one another? Is there a common “single supply” or are there meters on each floor? Is there 3phase or is it all single phase (or indeed are the floors each single phase, but spanning the full 3 phases?)

Are you just using the data for energy monitoring? Is there any plan for expansion into the control side? (just trying to determine if the data is actually needed local to the emonPi eg mqtt)

Are you planning to add emonTH’s or other nodes at any point?

Are you wanting to sum the energy data across floors? Does anyone else need access?

Lots of questions I know, but they do not all necessarily need a specific answer, they are more to prompt you as to what type of info is useful.

toml · 2017-11-26T12:31:31.812Z

Hi Paul,

Good tip on fudging sensitive info, ill do that in future and thanks for offering to look at my setup, I hope the following gives you a good enough overview:

The setup is in domestic flats, 11 flats in total will be monitored over 4 floors, with one floor hosting all of the gas meters for the entire block on a single room.

Per flat, I am monitoring within each flat its electrical consumption using multiple clamps attached to one emonTX, and heating/hot water pipe temperatures using two temp probes on another emonTX. External to the flat, I am monitoring gas consumption using an emonTX and an optical sensor.

On each floor, we have a secure room which we can locate a router, an emonPI and have access to power.

Within each flat and the room where the gas meters are, we have no direct access to internet or power, so we need to transmit data from the emonTX’s on a floor, to an emonPI on the same floor. We cant connect using serial due to distance.

We conducted a range test, we had been unable to send a signal between floors.

Setup per floor will look like this:

Floor G: 4 flats (so 8 emon tx’s), 11 gas meters so 19 emon tx’s total, and one emonPI/Router in secure room
Floor 1: 3 flats (so 6 emonTX’s) and one emonPI/Router in secure room
Floor 2: 2 flats (so 4 emon tx’s) and one emonPI/Router in secure room
Floor 3: 2 Flats (so 2 emon tx’s) and one emonPI/Router in secure room

My idea was to have my base node ID for the PI increment by one for each floor, so they will have nodeID’s 1 to 4, each floor will be on a different subnet just in case a signal crosses a floor.

I plan to identify each emonTX in relation to a floor using the flooring naming convention

MONITORINGTOPIC-FLOOR-ID

MONITORINGTOPIC will be something like GasMonitoring, PowerMonitoring etc.
FLOOR will be 0-3, matching the floor it is on
ID is a unique identifier for the node that ill use to identify where a node is.

Here is an example decoder:

[[5]]
   nodename = powerNode-0-1
   [[[rx]]]
      names = powerOverall, powerCooker , powerLight1, powerLight2, vrms, tempBoiler, tempHotWater, temp3, temp4, temp5, temp6, gasPulse
      datacodes = h,h,h,h,h,h,h,h,h,h,h,L
      scales = 1,1,1,1,0.01,0.1,0.1, 0.1,0.1,0.1,0.1,1
      units =W,W,W,W,V,C,C,C,C,C,C,p

in the above example of a power node, we are only interested in the power readings, the rest is redundant however the very same decoder format will be used for let’s say gas, in which case we will only be interested in the gas datapoint, the node name and ID will of course change to something like gasNode-0-1 if it’s for the same flat.

I plan to post all data to a custom API, in addition to an open energy monitor account.

Look forward to your thoughts on this setup, I understand I may get collisions, expanding the network is not a requirement.

pb66 · 2017-11-26T18:43:44.896Z

toml:

We conducted a range test, we had been unable to send a signal between floors.

That’s good, it will give you some segregation and improve your chances, but to be honest, I do not think the results will come anywhere close to justifying the expense and effort with the route you are on, (sorry!)

toml:

11 gas meters so 19 emon tx’s total, and one emonPI/Router in secure room

That is unlikely to work very well at all, 19 nodes on one RF network/group is going to be extremely flaky at best.

toml:

with one floor hosting all of the gas meters for the entire block on a single room.

Well that’s 11 emonTx’s you could connect serially freeing up a huge amount of airspace. However, personally I would have used a single Pi to just connect to all the gas meter optical pulsecounters.

toml:

Per flat, I am monitoring within each flat its electrical consumption using multiple clamps attached to one emonTX, and heating/hot water pipe temperatures using two temp probes on another emonTX.

Why 2 emonTx’s? You can connect up to 6 temp sensors (without changing/editing the firmware) on cables up to 10m in length on the same emonTx as the energy monitoring. all you need is a single Ethernet cable or a bit of BT phone wire between the emonTx and the place you were planning on siting the 2nd emontx’s. This single change will half the RF traffic on each floor (excl gas meters).

I also don’t understand why you are using emonPi’s rather than emonBases, the prominent differences are the LCD display which is going to be locked away in a secure cupboard and the on-board 2 channel emonTx which is not going to be used. The emonBase with case, is well under half the price of the emonPi.

My initial draft would be something like

Floor G: 4 flats (so 4 emontx’s each with CT’s and 2 temp sensors) and one emonBase in secure room.
Plus one more Pi in a case connected to the 11 gas meter optical probes.
Floor 1: 3 flats (so 3 emontx’s each with CT’s and 2 temp sensors) and one emonBase in secure room.
Floor 2: 2 flats (so 2 emontx’s each with CT’s and 2 temp sensors) and one emonBase in secure room.
Floor 3: 2 Flats (so 2 emontx’s each with CT’s and 2 temp sensors) and one emonBase in secure room.

With a third of the traffic (and costs) of using so many emonTx’s, this would increase reliability beyond significantly. The emonBases will give you exactly the same software and performance as the emonPi’s, and the 5 of them would cost around half the cost of the emonPi’s

If you are using a single emoncms server instance to log the data, mqtt will not play a part, that is for local data, you will have 4/5 different ip addresses and no way to sum or compare the data if you use 4/5 separate instances.

Are the tenants getting access to this data? with either public, read-only or fully writable (admin) access? or is it just for building management staff?

toml:

in the above example of a power node, we are only interested in the power readings, the rest is redundant however the very same decoder format will be used for let’s say gas, in which case we will only be interested in the gas datapoint, the node name and ID will of course change to something like gasNode-0-1 if it’s for the same flat.

To have any chance of this working with the 33 tx’s you will need to edit all the sketches so that a minimal payload is sent and as infrequently as possible eg temps and gas only every 60s. The redundant payloads entries will just block other nodes. You might find it’s ok’ish on the top floor, but certainly the ground floor is gonna be trouble.

Is this for info only or will it be the basis for electricity charges? (you don’t say if there are elec meters)

Anyways, my bellies rumbling so I’m going off to cook something, I will wait to see your reply before delving any deeper in case I’m on the wrong path.

toml · 2017-11-26T19:42:50.926Z

Oh my bad, I am using emon bases not emonPI!

Well the reason I am using two emonTX’s a flat is because of the distance between the consumer unit and boiler, we cant use one as the sensors simply will not reach, it was the original idea we had but we just couldn’t make it work.

The optical sensors to one PI does sound like a good idea, not sure we can get all of them onto one emon base as the meters are spread out a bit, but even a few would help, that is a good suggestion.

We do not need to sum the data, so being spread over IP’s will be ok, however, I have a custom platform I made which takes all the data in, so im ok with how that’s working, and it seems I can post to a remote emon cms also providing the bases node id’s are different.

For the gas nodes, what I can do is only send the gas data by filtering out the rest in node-red, it will save having to do anything with the sketches, won’t reduce that i am transmitting though, I could also reduce the timing like you pointed out.

Currently, I have 20 emonTX’s all transmitting to a simple base at home as a little test, so far they seem ok to be fair (as in I am getting data from every node at least every 60secs) but I need to look up how the optical sensors work, i.e if I do not get an optical read for a min, will the that one be the most upto date situation, i.e the total count since the TX was last reset as if so, then its properly going to be ok as this frequency I am getting so far is more than fine, providing no actual information for any totals is lost (i.e gas pulses), that being said, any increase would always be good.

The data is info only, not for charging anyone.

I suppose I could do the following:

Reduce transmission times on the boiler nodes, as the pipe temps won’t change that rapidly, and a 60sec lag will be fine, filter out unrequired datapoints on these nodes (ideally in tx firmware)
Try and wire up the gas ones directly, but if not then reduce times and filter out all data other than the optical reading data.

toml · 2017-11-26T23:41:10.479Z

Took your advice and did a gas pulse only firmware, much better! ill do the same for the rest of the nodes, that will cut the traffic down quite a bit.

pb66 · 2017-11-27T10:59:24.783Z

toml:

Well the reason I am using two emonTX’s a flat is because of the distance between the consumer unit and boiler, we cant use one as the sensors simply will not reach,

The 1-wire temp sensors can be extended by up to 10 meters (or more?) of cable, don’t be put off by the fitted plug, you can even use a Ethernet patch lead and a in-line F-F RJ45, although personally I like the continuous cable without connectors, the sensors can even be purchased with 15meters of cable.

Even the emonbase could be used for the temp sensors, are they not within 10m of either the CT emonTx or the emonBase?

Alternatively a per flat emonTH would be half the price and already configured for 60s updates, they can run of a cheap 5v dc power supply (or batteries lasting up to 2yrs, how are you powering the temp tx’s?) Plus they would add ambient temp and humidity sensing if the device was suitably located, temp adds meaning to the heating temp reading and humidity is never a bad thing to monitor in rented flats.

toml:

The optical sensors to one PI does sound like a good idea, not sure we can get all of them onto one emon base as the meters are spread out a bit, but even a few would help, that is a good suggestion.

Again the wiring can be extended, you could put a rj45 socket next to each meter and wire these all back to a central Pi, then the sensors are just plugged in next to the meters.

toml:

Currently, I have 20 emonTX’s all transmitting to a simple base at home as a little test, so far they seem ok to be fair (as in I am getting data from every node at least every 60secs)

I would have hoped that most of the time all of the packets would make it, the real issue is that all the devices send intervals are almost exactly the same, there will be minor differences and because of that, the intervals will gradually change position relative to the others, there will come a time when ech device will block another device for a significant time, think of 2 HGVs passing on a motorway when both are doing ~55mph. with 20 devices, that’s 19 other devices each device will regularly clash with at some point.

I have a client that called me in at the 11th hour to help install some OEM deices they had bought, this site is 3ph and there are just 3 emonTx’s in a project box with an emonBase, >80% of the time we have 100% of the packets accounted for, but when the devices 10s intervals clash, a device will be blocked for >20mins, since they are all 10s, they tend to go offline one after the other around the same time, so monitoring can be disrupted for a couple of hours, every few days (I cannot remember how often off hand, it’s been a couple of years since).

toml:

but I need to look up how the optical sensors work, i.e if I do not get an optical read for a min, will the that one be the most upto date situation, i.e the total count since the TX was last reset as if so, then its properly going to be ok

The pulse count is ever increment, only reseting with the device being reset or at rollover of the max value. Both of these can be handled in emoncms. So as long as the emonTx isn’t reset after a failed delivery and before a successful delivery, you shouldn’t loss any pulse counts.

Again, emonTh’s can work for pulse counting if you cannot, go the single Pi route. The update frequency of the pulses will not really impact the pulse readings directly, but reducing the interval (or removing them from the RF network altogether) will reduce the lost payloads from the ground floor flats.

toml:

Took your advice and did a gas pulse only firmware, much better! ill do the same for the rest of the nodes, that will cut the traffic down quite a bit.

That will work well for the temps and gas, but not so well for the power readings so it will be a case of finding a happy medium and accepting the power monitoring is very approximate and when larger samples are use there is a greater potential margin of error, loads like inductive hobs could drop off the radar entirely some days.

pb66 · 2017-11-27T14:00:57.535Z

Actually, the penny’s just dropped, if you have 20 emonTx’s on test, that means you have already purchased all the equipment so emonTH’s instead of emonTx’s is no longer a real option. The only real options are whether to fit all the emonTx’s (with shorter, slower updates) or only fit some of them to improve the reliability and accuracy.

toml · 2017-11-29T10:56:40.128Z

They are good ideas but unfortunately we cant use one tx per flat as whilst they are within 10m, we are not able to drill or surface mount anything as they are new builds.

The gas idea is also a good one but we do not have a plug for the emonBase to make serial connection work.

All this being said, I have now been running 20 tx’s and a further 70 RF433 temp sensors at home all running together. I am getting data quite frequently (not seeing anything > 60 secs) which for our use case will be fine).

I have experienced the 10-15 min blackout you described, I thought it was blocking related but last night I was quite lucky as I was on my laptop when I got one, I was quickly able to log into the emonBase, it was having some issues with MQTT and it performed a couple of self-resets and sorted itself out, I grabbed a log file, so I will see if I can have a dig through as this might be what your client is getting?

I am going to redo the sketches on all sensors though like you mentioned, that idea has greatly tidied up the database and I’m sure will only help in terms of blocking to some degree.

If you like, I can let you know how this goes when we actually deploy them (in a week or two). Its a feasibility study so if does not go quite as well as hoped, we can simply turn a few things off and find a balance if required.

Thanks for your advice

pb66 · 2017-11-29T12:40:32.697Z

toml:

I have experienced the 10-15 min blackout you described, I thought it was blocking related but last night I was quite lucky as I was on my laptop when I got one, I was quickly able to log into the emonBase, it was having some issues with MQTT and it performed a couple of self-resets and sorted itself out, I grabbed a log file, so I will see if I can have a dig through as this might be what your client is getting?

This will be unrelated to the blocking. The blocking occurs in the RF portion of the journey and when it happens, it is as if the packets were never sent. There will be no traces of them in the logs, at most if you are lucky to only corrupt rather than totally block a packet and you have “quiet mode” disabled in emonhub, you may occasionally see those failed packets, but without doubt the majority will disappear without trace, because they were never seen by the receiver.

My clients set up did not have any of the software we are discussing here, mqtt was not used and it was based on original emonhub not the emonpi variant which has been stable (dare I say flawless) for a couple of years or so.

The “blackouts” are purely caused by RF blocking, when an emonTx (using JeeLib) goes to transmit it will listen for a gap in the RF traffic, it will wait and try again if it doesn’t see a gap. beyond that the packet is lost. I have not dug deep enough to know whether the emonTx gives up trying or whether it just “sends anyway” and the payload is lost in the traffic, but I have never seen a packet delayed more than about 6 secs and back then I did a substantial amount of testing.

Similarly if an emonTx goes to transmit and cannot see/hear an emontx at the opposite extreme edge of the network, it will go about transmitting as it believes the way is clear, In actual fact the receiver could be tied up receiving a packet from that other emonTx that first cannot see (assuming the receiver is in the middle) and that packet will be lost entirely or perhaps the tail end of it received, but that will fail CRC and the packet will not be repeated as it was “successfully sent”.

You could try testing with ACK’s (not normally used in OEM firmwares) but this could swing either way, increase reliability because packets that are not acknowledged are re sent, or as I think might be the case here, cause further issues because of the increased traffic from both the repeated packets and from all the new acknowledgement transmissions.

Another point worth mentioning is if you are in anyway debugging or studying the packet loss and reliability, do not use the phpfina feeds or any emoncms graphing as these will add to the issue. They are fine for displaying known good data, but the datapoints they return will not reliably reflect the consistency of the incoming data due to rounding, averaging and sampling etc.