Direct connection emonTx V3 3-phase to RaspberryPi

johnbanks · 2017-04-16T15:24:51.798Z

I got it from here …

https://github.com/openenergymonitor/emontx-3phase

Robert.Wall · 2017-04-16T17:47:03.518Z

I, as the originator of the sketch, worked from my master copy. That has obviously been edited by someone else.

johnbanks · 2017-04-16T23:51:49.100Z

Robert - here’s where I’m currently at …

Using the .ino file you sent but with SERIALOUT substituted in line 165

Just one CT connected plus AC adapter plus pulse sensor

The RFM2Pi interfacer deleted from the emonhub config

This interfacer used instead …

[[SerialTx3e]]
Type = EmonHubTx3eInterfacer
[[[init_settings]]]
com_port= /dev/ttyAMA0
com_baud = 115200
[[[runtimesettings]]]
pubchannels = ToEmonCMS,

With Node 11 following yr suggestion (except datacodes rather than datacode) …

[[11]]
nodename = 3phase
[[[rx]]]
names = powerL1, powerL2, powerL3, power4, Vrms, pulse
datacodes = h,h,h,h,h,L
scales = 1,1,1,1,0.01,1
units =W,W,W,W,V,p

The result in the log is …

2017-04-16 22:02:15,837 INFO MainThread EmonHub emonHub ‘emon-pi’ variant v1.2
2017-04-16 22:02:15,838 INFO MainThread Opening hub…
2017-04-16 22:02:15,838 INFO MainThread Logging level set to DEBUG
2017-04-16 22:02:15,839 INFO MainThread Creating EmonHubTx3eInterfacer ‘SerialTx3e’
2017-04-16 22:02:15,842 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 115200 bits/s
2017-04-16 22:02:15,843 DEBUG MainThread Setting SerialTx3e pubchannels: [‘ToEmonCMS’]
2017-04-16 22:02:15,844 DEBUG MainThread Interfacer: Subscribed to channel’ : ch2
2017-04-16 22:02:15,846 INFO MainThread Creating EmonHubMqttInterfacer ‘MQTT’
2017-04-16 22:02:15,847 INFO MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2017-04-16 22:02:15,850 DEBUG MainThread MQTT Subscribed to channel’ : ToEmonCMS
2017-04-16 22:02:15,851 INFO MainThread Creating EmonHubEmoncmsHTTPInterfacer ‘emoncmsorg’
2017-04-16 22:02:15,853 DEBUG MainThread emoncmsorg Subscribed to channel’ : ToEmonCMS
2017-04-16 22:02:15,952 INFO MQTT Connecting to MQTT Server
2017-04-16 22:02:15,955 INFO MQTT connection status: Connection successful
2017-04-16 22:02:15,957 DEBUG MQTT CONACK => Return code: 0
2017-04-16 22:02:16,058 INFO MQTT on_subscribe
2017-04-16 22:02:22,497 WARNING MainThread SerialTx3e thread is dead
2017-04-16 22:02:22,699 WARNING MainThread SerialTx3e thread is dead

However, minicom reports …

Welcome to minicom 2.7

OPTIONS: I18n
Compiled on Jan 12 2014, 05:42:53.
Port /dev/ttyAMA0, 17:41:08

Press CTRL-A Z for help on special keys

11 589 0 0 0 25744
11 619 0 0 0 25800
11 582 0 0 0 25893

Current & voltage sensors are reporting but the pulse sensor is not 

Now if I shutdown and just substitute a temp sensor for the pulse sensor and change Node 11 accordingly to …

[[11]]
nodename = 3phase
[[[rx]]]
names = powerL1, powerL2, powerL3, power4, Vrms, temp1
datacodes = h,h,h,h,h,h
scales = 1,1,1,1,0.01,0.1
units =W,W,W,W,V,C

SerialTx3e thread is still dead however Minicom now reports …

Welcome to minicom 2.7

OPTIONS: I18n
Compiled on Jan 12 2014, 05:42:53.
Port /dev/ttyAMA0, 22:18:55

Press CTRL-A Z for help on special keys

11 602 0 0 0 25891 217
11 615 0 0 0 25886 217
11 599 0 0 0 25923 217
11 607 0 0 0 25921 217

Current, voltage & temp sensors are all reporting 

In summary …

The pulse sensor is not reporting.

And the format of reports in general seems unacceptable to emonhub.

What is the significance of the leading 11 in the reports? Could the body be buried there?

Thx for yr support

pb66 · 2017-04-17T08:42:55.903Z

johnbanks:

Current & voltage sensors are reporting but the pulse sensor is not

Looking at the new sketch file the pulse count has not been added to the print outputs, so that would make sense.

johnbanks:

Current, voltage & temp sensors are all reporting

The temp sensor values are added to the serial prints dynamically based on how many temp sensors are found at startup, so this would be as expected too.

johnbanks:

The pulse sensor is not reporting

Just needs adding to the serial print blocks in the sketch.

johnbanks:

And the format of reports in general seems unacceptable to emonhub.

AFAIK this interfacer takes full human readable values rather than byte values that require decoding so the use of datacodes shouldn’t be needed. Try using “datacode = 0” in the node definition. I have no experience of this interfacer and have not studied the code so I cannot say if the names and scales are used, I would guess not.

johnbanks:

What is the significance of the leading 11 in the reports?

That is the node id, it informs emonhub and emoncms were it originated.

johnbanks:

2017-04-16 22:02:22,497 WARNING MainThread SerialTx3e thread is dead

Unfortunately the “thread is dead” message is the result of an unhandled fatal error, that message should never be seen and it only exists in the code to ensure threads do not disappear silently, it offers no insight into what caused it, further debugging will probably involve adding some log “checkpoint” entries to the code unless something is found in the settings.

Try datacode = 0 in the node definition.

pb66 · 2017-04-17T08:47:30.700Z

Robert.Wall:

I, as the originator of the sketch, worked from my master copy. That has obviously been edited by someone else.

Following this thread

3-Phase Serial Connection - emonhub query Installation

Hi, I am brand new to emoncms so bear with me. I am running into some issues with the configuration of emonhub. My general setup is as follows: emonTx 3.4 with 3xCT’s connected to 3 phases (240v, AU) 9V AC power pack for the emonTx direct serial connection to RPi3 running emonPi SD image emonTx using 3-phase approximation sketch example with config changed to Serial (I think) I have been able to get the emonTx to transmit over serial to the Pi and can confirm with Minicom that data is bein…

there was a small change made to a couple of line endings back in Feb

github.com/openenergymonitor/emontx-3phase

Changed newline line endings for newline + carriage return line endings.

committed 10:29AM - 13 Feb 17 UTC

pb66

+3 -3

See ["3-Phase Serial Connection - emonhub query"](https://community.openenergymo…nitor.org/t/3-phase-serial-connection-emonhub-query/3099?u=pb66)

johnbanks · 2017-04-17T14:22:11.906Z

Robert/Paul …

Using datacode = 0 did not solve the thread is dead problem.

Then for the umpteenth time I read this post …

Serial-Direct connection between Pi and emonTx shield Hardware

[ moved from Does the emonTx Shield need an actual Arduino? ] Hi, I’m new to this forum but this topic seems to me beeing read by a lot of experts. I still have some issues with my monitoring which is built with a TX shield without RF module, 2 CTs - one for the PV and the other for network income - sticked on an Arduino Uno. Communication via serial to a Raspberry PI running mysql, emoncms emonhub, etc. as described in the articles https://openenergymonitor.org/emon/node/3884 and emoncms/d…

… and found the solution … Interfacer Type needs to be EmonHubSerialInterfacer.

I now have Inputs to emoncms!

So to recap my set up …

Substitute SERIALOUT in line 165 of the .ino file and flash to the emonTx unit.

In my case, I connected just one CT plus AC adapter plus temp sensor.

My direct serial connection is like so …

https://wiki.openenergymonitor.org/images/thumb/Emontx_to_pi_serial.JPG/764px-Emontx_to_pi_serial.JPG

Delete the RFM2Pi interfacer from the emonhub config and add …

[[SerialTx]]
Type = EmonHubSerialInterfacer
[[[init_settings]]]
com_port= /dev/ttyAMA0
com_baud = 115200
[[[runtimesettings]]]
pubchannels = ToEmonCMS,

Modify Node 11 in the emonhub config like so …
[[11]]
nodename = 3phase
[[[rx]]]
names = powerL1, powerL2, powerL3, power4, Vrms, temp1, temp2, temp3, temp4, temp5, temp6
datacode = 0
scales = 1,1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1
units =W,W,W,W,V,C,C,C,C,C,C

Note – pulse to be added later – once the .ino file has been amended to add the pulse count to the print outputs.

Here’s a log file extract …

2017-04-17 12:29:22,315 INFO MainThread EmonHub emonHub ‘emon-pi’ variant v1.2
2017-04-17 12:29:22,316 INFO MainThread Opening hub…
2017-04-17 12:29:22,316 INFO MainThread Logging level set to DEBUG
2017-04-17 12:29:22,317 INFO MainThread Creating EmonHubSerialInterfacer ‘SerialTx’
2017-04-17 12:29:22,320 DEBUG MainThread Opening serial port: /dev/ttyAMA0 @ 115200 bits/s
2017-04-17 12:29:22,321 DEBUG MainThread Setting SerialTx pubchannels: [‘ToEmonCMS’]
2017-04-17 12:29:22,322 DEBUG MainThread Interfacer: Subscribed to channel’ : ch2
2017-04-17 12:29:22,323 INFO MainThread Creating EmonHubMqttInterfacer ‘MQTT’
2017-04-17 12:29:22,325 INFO MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2017-04-17 12:29:22,328 DEBUG MainThread MQTT Subscribed to channel’ : ToEmonCMS
2017-04-17 12:29:22,329 INFO MainThread Creating EmonHubEmoncmsHTTPInterfacer ‘emoncmsorg’
2017-04-17 12:29:22,331 DEBUG MainThread emoncmsorg Subscribed to channel’ : ToEmonCMS
2017-04-17 12:29:22,429 INFO MQTT Connecting to MQTT Server
2017-04-17 12:29:22,432 INFO MQTT connection status: Connection successful
2017-04-17 12:29:22,434 DEBUG MQTT CONACK => Return code: 0
2017-04-17 12:29:22,535 INFO MQTT on_subscribe
2017-04-17 12:29:22,828 DEBUG SerialTx 1 NEW FRAME : 11 994 0 0 0 25366 223
2017-04-17 12:29:22,830 DEBUG SerialTx 1 Timestamp : 1492432162.83
2017-04-17 12:29:22,830 DEBUG SerialTx 1 From Node : 11
2017-04-17 12:29:22,831 DEBUG SerialTx 1 Values : [994, 0, 0, 0, 253.66, 22.3]
2017-04-17 12:29:22,831 DEBUG SerialTx 1 Sent to channel(start)’ : ToEmonCMS
2017-04-17 12:29:22,833 INFO SerialTx Publishing: emon/3phase/powerL1 994
2017-04-17 12:29:22,834 INFO SerialTx Publishing: emon/3phase/powerL2 0
2017-04-17 12:29:22,836 INFO SerialTx Publishing: emon/3phase/powerL3 0
2017-04-17 12:29:22,849 INFO SerialTx Publishing: emon/3phase/power4 0
2017-04-17 12:29:22,851 INFO SerialTx Publishing: emon/3phase/Vrms 253.66
2017-04-17 12:29:22,854 INFO SerialTx Publishing: emon/3phase/temp1 22.3
2017-04-17 12:29:22,856 INFO SerialTx Publishing: emon/3phase/rssi 0
2017-04-17 12:29:22,858 INFO SerialTx Publishing: emonhub/rx/11/values 994,0,0,0,253.66,22.3
2017-04-17 12:29:22,860 INFO SerialTx Publishing: emonhub/rx/11/rssi 0
2017-04-17 12:29:22,862 DEBUG SerialTx 1 adding frame to buffer => [1492432162.828043, 11, 994, 0, 0, 0, 253.66, 22.3]
2017-04-17 12:29:22,862 DEBUG SerialTx 1 Sent to channel(end)’ : ToEmonCMS

Can I assume that you guys will fix the .ino file for pulse print output? Also, line 121 needs to reflect datacode = 0

Thx again for yr support

pb66 · 2017-04-17T14:41:39.457Z

johnbanks:

… and found the solution … Interfacer Type needs to be EmonHubSerialInterfacer.

That would mean you are using a slightly different approach rather than sorting out the existing approach. Personally I prefer the way you are doing it now, but the “& is ESP module compliant” in your opening post and Glyn’s link to the seriialtx3e configuration led me to believe you wanted to use the emonESP format not good old fashion, tried and tested CSV.

The penny should have dropped when I saw your minicom output, that is CSV not ESP format.

Glad you got it going though.

Robert.Wall · 2017-04-17T14:44:14.114Z

I thought I’d put those in. But as I’ve got sore eyes, a runny nose, a sore throat and a thick head, anything’s possible. You’re on your own for a few days.

johnbanks · 2017-04-17T20:56:16.512Z

Paul …
I had no idea that I was using a different approach - having much more persistence than detailed knowledge.
Just love yr … “good old fashion, tried and tested …”.
My initial ref to ESP compliant was just because it was mentioned in the 3 phase sketch.
(By way of background - I struggled with getting 3 ESP Huzzah modules to connect for a couple of weeks. Succeeded with one but the other two are now not functioning after attemping to clean by flashing with blank files - got tme outs and can now flash nothing.)

Robert …
Hope you soon get well.

PS - If I’m not in a small minority attracted by the simplicity/reliability of 3 jumper wires as a connection, should this approach be more fully documented in the technical part of the website?

Robert.Wall · 2017-04-18T15:42:05.192Z

johnbanks:

If I’m not in a small minority

I think actually you are - it’s rare that anyone asks about a serial connection.

johnbanks:

should this approach be more fully documented in the technical part of the website?

I think it is: emonTx3 User Guide — OpenEnergyMonitor 0.0.1 documentation

The 3-phase sketch uses compiler directives to generate the required output data format, rather than a separate sketch.

Or is that not enough? Bear in mind that it’s quite hard for us to understand the difficulties faced by a newcomer, so details of where (and better still, why) you got stuck are very helpful.

BTW, I’ve discovered why my copy of GitHub hadn’t been updated - stuff had been moved about, nobody told my script so it didn’t find it to update it.

I thought the sketch I posted had the pulse outputs. And if somebody tells me the “correct” emonESP format, I can amend that too. But I’m in no state to write code. It’ll get looked at later.

johnbanks · 2017-04-18T23:45:31.668Z

Robert …

Yes, I guess I’m in a minority (in UK) in needing 3 phase capability to monitor a 30kWp PV array approx 100 metres from the house/grid supply meter plus a pool Air Source Heat Pump approx 60 metres from the house.

I have an emonTx for each of the 3 locations and, at an early stage, concluded that Radio connection would not be suitable.

I did see the link you provided and wanted to try simply connecting with 3 jumper wires as in the picture but …

The additional links were all for single phase and the examples all related to using a USB to UART adapter … they did not relate to the picture.

For this reason, I just started trial & error experimenting and stumbled on a connection that worked. It was not a conscious decision to go for a serial connection. And that it necessitated deleting the RFM2Pi interface was of no consequence.

I realise I’ll end up with a local emoncms at 3 locations but envision a 4th RPi that ‘mounts’ the 3 x local emoncms data – how TBD – SAMBA or autofs & sshfs ??

A lot of RPi’s? – yes - but the new RPi Zero W is comparably priced to the ESP Huzzah module and both require a good 5 volt PSU.

Just some thoughts.

pb66 · 2017-04-19T08:50:40.009Z

You shouldn’t need a full emoncms server at each location, you can just use emonhub on 2 or more of the Pi(Zero?)'s to post the data from each device to one or more common emoncms serverss hosted elsewhere (on the 3rd Pi?). I have some recollection of helping someone using 2 emonPi’s where one was a full server and the 2nd just forwarded data to the 1st, both also sent data to emoncms.org too.

It’s a shame the emonESP’s don’t work for (not that my experience with them is any better) as that might have given you more options. Perhaps you could still use that one Huzzah and point that at a Pi and emonTx “direct-serial” combo at another location that also serves as a emoncms server.

The direct-serial connection is by far the most reliable connection but is limited by distance, if you include the emonPi which is essentially a cut-down Tx permanently connected via “serial-direct” to a Pi, then actually a majority of users are using that method, just without the wires.

There are variations of the “serial-direct” method, for example using a USB-serial adapter, effectively a “usb-direct” I guess, which is limited to 5m by USB standards BUT you can get USB “active-extension” leads (quite expensive though) and “USB over Ethernet” adapters, recently I briefly tested a pair of these over a 50mtr Cat5e cable and it seemed to work (I intend to experiment further when I get a chance).

Although the devices are 60m and 100m from the house, how for from each other are they? could a 40-50m link between them work? Then just the one needs to be network connected, via WiFi maybe.

You would not be able to connect both emonTx’s via the same GPIO serial port, the serial uart is a one-to-one communication so even if distance wsn’t a factor at least 2 would need to be connected another way, those “remote devices” could be connected via extended USB (“Active extn” or “over Ethernet”) if within 50m or you could use WiFi, either a couple of Pi zero W’s or the Huzzah on one device.

Personally I would avoid trying to synchronize and maintain multiple servers.

Bill.Thomson · 2017-04-19T15:25:48.196Z

Another alternative is an RS-485 link.
USB to RS-485 adapters are inexpensive, the max link distance is 1200 Metres and RS-485 has multi-drop (bus) capability.

Examples here and here

pb66 · 2017-04-19T15:39:10.458Z

Hi Bill, can you expand a bit?

I get that a rs-485 signal can travel further and using a rs-485 to USB adapter could make the connection at the “Pi end” but how would one go about getting the emonTx’s compatible? What hardwire connections are required? would the existing serial output be used/converted or does the sketch need to be significantly different ?

Bill.Thomson · 2017-04-19T15:44:53.915Z

Sorry 'bout that. I should have specified the RS-485 to USB connection was for multiple emomPis.

To make it work with a Tx, an RS-485 to RS-232 converter would be needed.
(They cost less than the USB to RS-485 converters)

Examples here and here.

As long as the existing serial output is RS-232 compatible, the sketch should work as is.
Transmitted / received data, earth and power should be the only connections required.

Dave · 2017-04-19T22:06:24.020Z

Hi Bill
Does that mean that you can have multiple EmonTx’s (Equipped with RS232>RS485 adaptors) sitting on one RS485 network and then have a master USB > RS485 adaptor plugged onto the main EmonPi?
Wouldn’t you need a direct serial type sketch to include the node ID?

Regards
Dave

Robert.Wall · 2017-04-19T22:28:10.307Z

If you’re going to RS-485, would it be possible to do a multiple drop link via the emonTx’s straight to a single emonBase? The sketches would need to be developed to work on a master-slave basis, otherwise there is the probability of collisions on the serial link. First thoughts are one emonTx acts as the master and works normally, the second is modified to listen on the bus and do its measurement as soon as the ‘master’ completes its transmission, the third likewise after a delay. All will of course need to send the Node ID.

johnbanks · 2017-04-21T13:03:12.112Z

Paul …
Belated thx for yr comments & suggestions.

Two outdoor long range antennas have been installed so there is wifi coverage across the site. At the inverter (furthest from the house/supply meter) the signal strength is -55 dB and the inverter is successfully communicating with the manufacturer’s server.

I will follow yr suggestion of only having one local emoncms server and two Rpi’s just running emonhub.

For these two units – is it best to install standalone emonhub on top of Raspbian or to install the emonSD image and just not use the emoncms functionality (by only having an HTTP interfacer and not creating local feeds)?

The third unit (local emoncms) will run the emonSD image. Is it advisable also to do apt –get update/upgrade and the rpi-update on this unit?

Do you see a problem also installing a realVNC server? (The installation is an hour’s drive from my home and it’s difficult being ‘resident engineer’ when not actually resident).

Does Direct Serial have limitations – the large numbers associated with pulse counts or cum kWh’s for example?

Sorry for all the questions

pb66 · 2017-04-21T14:33:17.003Z

johnbanks:

For these two units – is it best to install standalone emonhub on top of Raspbian or to install the emonSD image and just not use the emoncms functionality (by only having an HTTP interfacer and not creating local feeds)?

you could go either route here, the “official” recommendation would probably be to use the emonSD as that is intended as a single solution. But personally, I would go for installing the original emonHub on Raspbian “Jessie” OS configured as read-only.

The “emonpi variant” of emonHub is ok for local emoncms but it does not buffer data or wait for confirmed receipt before deleting the sent data, basically it fires and forgets your data every 30s, so any network interruptions, router or server downtime all result in lost data which is not the case with original emonHub.

Plus there’s also a lot of other stuff going on on the emonSD that you simply do not need and that just adds vulnerabilities and complexity that has no upside.

If you use the full Jessie image (not Lite) it has a realVNC licence included and WiFi configuration is as easy as adding a wps_supplicant.conf file to the boot partition before plugging it into a Pi Zero W (no Ethernet) or any Pi for that matter.

Although personally I prefer to use xrdp on my remote devices if I need more than just SSH, that is easy to install and IMO much easier to use via Windows RDP than using VNC.

Since these devices will all be sharing a LAN you can SSH between them so another option might be to use Dataplicity, the first account is free and it negates any need for a fixed Ip (or dynamic DNS) and port-forwarding/exposed ports etc. It depends a lot on whether you need a desktop or just a command line, and how easy it is to get external access.

johnbanks:

Does Direct Serial have limitations – the large numbers associated with pulse counts or cum kWh’s for example?

Not at all (well it probably does, just not within the scope of what we’re doing) I have just installed some 3phase devices which are basically 1x Pi with 3x emonTx’s (converted to have a 5th CT) connected via USB-serial adapters, each emonTx send a payload of 20 values (21 inc node id) every 5secs so emonHub is receiving around 756 full values each min via the serial interfacer without breaking a sweat.

Using the serial-direct (via GPIO or USB) means there no need to decode the payload, so no datacode(s) to define, the default datacode = 0 will be used. Unfortunately original emonhub doesn’t use scales and names, but there again the emonPi variant doesn’t use the name over http either so not much is lost (I really should add scales to the original version - maybe I will take a look this weekend if I get time)

johnbanks:

Sorry for all the questions

Not at all, I enjoy discussing this stuff, when time permits me to, that is.

Robert.Wall · 2017-06-24T16:04:18.421Z

A post was split to a new topic: Connect a Raspberry Pi 3 via RS485 to a vending machine