Collect data from RPI3 to Emoncms

@Atelier21,

You can send/receive data between Emoncms and the RPi using node-RED and MQTT. You should read: MQTT, to familiarize yourself with MQTT protocol and requirements. You should also read MQTT Service Credentials and MQTT Installation Docs.

You will need to install node-RED on the RPi.

node-RED contains MQTT nodes with which you will be able send and receive data to/from Emoncms. There’s also a node-RED node called node-red-node-emoncms which you can install into node-RED and then use to send/receive data.

I will be glad to post a node-RED flow that shows how to send/receive using MQTT nodes and the Emoncms node if this is the method you decide to use after reading the docs.

Good luck!

So are you looking to store the data on emoncms locally and send it to emoncms.org?

It sounds very much like you are describing what emonhub does.

emonhub is designed to collect data from various sources and hardware to send on to one or more, remote or local emoncms’s.

Although I do not use the emonpi variant of emonhub, I do use the original emonhub with 3 usb-serial connected emonTx’s for 3ph monitoring. That data is then sent to multiple emoncms instances via http.

I want to store datas collected from my RPI3 in the emoncms server ( i don’t know how to link my local emoncms to the emoncms global website ( i used the API key from my account in the website and paste it to the emonhub config in the “xxxxxxx…” zone)).
I’ve connected my RPI3 with my 2 arduinos and launch my system . I just want to display my energy consumption in real time and i have some issues . I’ve taken some screenshoots :
The first screenshot is from the emoncms website and the second is from my local emoncms ( don’t know why i don’t have the same inputs)

Sorry for thoses messages , i’m a novice on energy monitoring

What messages are you seeing in the emonhub.log via emoncms?
Is your serial data being passed ok?

What format payload are the arduinos pushing out data to the serial port in? How are they connected? What baud? Have you set up emonhub for each serially connected device? You also need to edit the node definitions in emonhub.conf to suit your payloads.

Here is the logs


What do you mean ?

I can send you in PM my arduino’s program for my datalogging system if you want

USB port

What do you mean ?

Don’t know how to do that or if i have to ?

You are welcome to post your sketch on the public forum but but please don’t PM it to me. If you post publicly other people might be able to assist and/or other users might benefit from the discussion. Unfortunately I do not have the time available to offer one-to-one support.

Have you done anything on the pi to connect to the arduinos?

If you look at your emonhub.conf there is a section called [[rfm2pi]], this is an “interfacer” configuration for the emonpi or rfm2pi add-on boards attached to the pi’s gpio serial port. You can see that interfacer being created in your emonhub.log images. You will need similar configurations for each of the arduinos if you want emonhub to set up a usb/serial port and communicate with them.

There are 2 existing interfacer types to do this, one is specifically for the emonTx v3.4 firmware and the other is a more generic type.

You can find more info here on the forums by searching terms like “serial-direct”, “emonhubserialinterfacer” and “serialtx3e” etc

You should also take a look at the openenergymonitor/emonhub github repo directserial-serialtx3e and directserial examples.

The Does the emonTx Shield need an actual Arduino? thread might be useful too.

In that thread there is also info about creating udev rules so that the arduinos can be uniquely identified otherwise you may find they can swap addresses (/dev/ttyUSB0 and /dev/ttyUSB1 etc) but if you opt for the serial-direct (not tx3e) format the node id is passed within the payload to the addressing is not an issue.

The 2 serial formats are quite different, you can choose to use either as long as you use the correct interfacer type to decode the payload sent by your arduino’s.

I hope this points you in the right direction, feel free to ask questions and we can try to help you, but most if not all the info you need is here on the forums in abundance.

I’ve just linked my arduinos with my RPI3 using USB port

All right , this is the sketch i’m using for my arduinos

Thanks for responding , i’ll take a look on your advices

Best regards

It looks like you will need to change your Serial.prints in your sketch as it currently doesn’t match either format accepted by emonhub.

Can you cut and paste logs and code etc rather than posting pictures where possible please? It makes reading them easier and also (for example) offering edits etc can be done by rearranging your snippets rather than typing everything out from an image.


You can use 3 back-ticks (```) in the line before and the line after any code or log snippets eg

this is code 

(the image below is of the example above.)

From what I read, you just need to format your Serial.Print so that the EmonHub will recognize.
Then Configure EmonHub to Read it.
Following are snippets of my Arduino Serial Code, and the EmonHub Configuration that recognizes it.
Note: I am using the UART Serial.

Arduino Code in Proper format for EmonHub to recognize.

//dlp 0102018 change to match format of EmonCMS Original is following line.
         Serial.print(p[i].rate(time));  Serial.print(':'); Serial.print(p[i].count);  //Print values

EmonHub Configuration:

[[SerialTx3e]]
     Type = EmonHubTx3eInterfacer
      [[[init_settings]]]
           # Un-comment line below if using RS485 adapter
           #com_port = /dev/ttyRS485-0
           # default com port if using USB to UART adapter
           #com_port= /dev/tty/ACM0 1/1/2019 dlp use this id /dev/ttyUSB0 don't work
           #com_port= /dev/ttyUSB0
           #com_port= /dev/ttyS0
           com_port = /dev/ttyAMA0
           com_baud = 115200
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,
	   nodeoffset = 8

Edit - fixed formatting. BT, Moderator

It looks like you some difficulty finding the right address for yor device, using this command will simplify that as it lists all the devices with the known serial address prefixes. To be double sure you can run it before and after attaching a device so that the exact address is discovered by the change in output.

sudo ls -la /dev/{tty{ACM,AMA,S,USB},serial}*

In this instance @Atelier21 will most likely be better off using the more generic serial-direct format as that contains the node id and is therefore safer to use with multiple devices, unless the devices have unique hw serial numbers that can be access and used by a udev rule. The generic format is space separated values with a leading node id

nodeid val1 val2 val3 . . .

the EmonHubTx3eInterfacer method is a very specific tool designed to simply accommodate the emonTx v3.4 serial output intended for a emonESP without needing to change the firmware in the emonTx. If you are able to edit/write your sketch then the EmonHubSerialInterfacer is the better option.

Not only because it includes a node id, but also because it is more streamline, although the emonTx prints key:value pairs to the emonESP, the emonhub tx3e interfacer basically treats each key:value as a space separated value by stripping away and ignoring the key: in favour of the names defined in the [nodes] configuration of emonhub.

Hi , thanks for responding, this is the code like you wanted

  #include "EmonLib.h"
// Include Emon Library
EnergyMonitor emon1;
// Create an instance

EnergyMonitor emon2;
// Create an instance

EnergyMonitor emon3;
// Create an instance

EnergyMonitor emon4;
// Create an instance

//https://github.com/openenergymonitor/learn/blob/master/view/electricity-monitoring/ct-sensors/how-to-build-an-arduino-energy-monitor-measuring-current-only.md
void setup()
{
  Serial.begin(9600);

  emon1.current(A0, 111.1);             // Current: input pin, calibration.
  emon2.current(A1, 111.1);             // Current: input pin, calibration.
  emon3.current(A2, 111.1);             // Current: input pin, calibration.
  emon4.current(A3, 111.1);             // Current: input pin, calibration.
}

void loop()
{
  // calculer et afficher le courant 1
double Irms1 = emon1.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms1*24.0);         // Apparent power 
  Serial.print(" ");
  Serial.println(Irms1);          // Irms
  //FIN

   // calculer et afficher le courant 2
double Irms2 = emon2.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms2*24.0);         // Apparent power
  Serial.print(" ");
  Serial.println(Irms2);          // Irms
  //FIN

   // calculer et afficher le courant 3
double Irms3 = emon3.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms3*24.0);         // Apparent power
  Serial.print(" ");
  Serial.println(Irms3);          // Irms
  //FIN

   // calculer et afficher le courant 4
double Irms4 = emon4.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms4*24.0);         // Apparent power
  Serial.print(" ");
  Serial.println(Irms4);          // Irms
  //FIN
}

I’ll try your suggestions and returning to you if i have some questions

@pb66 i’m sorry but i don’t understand all of what you said :

EDIT1: I’m wondering how can i basculate to EmonhubSerialInterfacer , it seems that i’m using EmonHubJeeInterfacer according to my Emonhub configuration

Thanks for your help and advices

An interfacer is how emonhub interfaces with various hardware and sources etc, there are different “types” of interfacer for different jobs, eg the EmonHubJeeInterfacer is for communicating with JeeLib based transceivers (base stations) like RFM2Pi, emonPi and JeeLink etc. Basically it is a EmonHubSerialInterfacer with an additional feature set to send specific “JeeLib” configuration settings via serial as well as just reading the serial port. It is very common to OEM setups but if you are not using one of those devices you can disable/delete it from your emonhub.conf.

Since you are connecting via serial, you could actually still use that EmonHubJeeInterfacer but there is a lot of complexity that you simple do not need for simple serial comms. All you really need is a EmonHubSerialInterfacer instance for each of your usb connected arduino’s in the interfacers section of emonhub.conf eg

[interfacers]

    [[ArduinoOnUSB0]]
        Type = EmonHubSerialInterfacer
        [[[init_settings]]]
            com_baud = 9600
            com_port = /dev/ttyUSB0
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12, 

    [[ArduinoOnUSB1]]
        Type = EmonHubSerialInterfacer
        [[[init_settings]]]
            com_baud = 9600
            com_port = /dev/ttyUSB1
        [[[runtimesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12, 

Offhand I do not know what addresses your devices will be, they could be /dev/ttyUSBx if you’re using generic usb-serial adaptors or they could be /dev/ttyACMx if they are Arduino’s with on-board USB or even something else, that’s why i posted the command to run before and after connecting the device(s), it tests for all the “usual” serial addresses.

sudo ls -la /dev/{tty{ACM,AMA,S,USB},serial}*

As for the sketches and the output format, if you add a leading node id, comment out the text labels and add a trailing empty “print with newline” like so

void loop()
{
// Add a leading node id eg 28 ?
Serial.print("28"); Serial.print(" ");

  // calculer et afficher le courant 1
double Irms1 = emon1.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms1*24.0);         // Apparent power 
  Serial.print(" ");
  //Serial.println(Irms1);          // Irms
  //FIN

   // calculer et afficher le courant 2
double Irms2 = emon2.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms2*24.0);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms2);          // Irms
  //FIN

   // calculer et afficher le courant 3
double Irms3 = emon3.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms3*24.0);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms3);          // Irms
  //FIN

   // calculer et afficher le courant 4
double Irms4 = emon4.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms4*24.0);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms4);          // Irms
  //FIN

Serial.println()
}

you should get serial output like so 28 1 2 3 4 on each line (where 1,2,3&4 are your current values).

Each device needs a unique node id and a corresponding entry in the [nodes] section of emonhub.conf will allow you to define names etc eg

[nodes]
    [[28]]
        nodename = Arduino_A
            [[[rx]]]
                names = value1, value2, value3, value4

    [[29]]
        nodename = Arduino_B
            [[[rx]]]
                names = value1, value2, value3, value4

there are other things you can define here like scales and datacodes, but if you are sending “real human values” ie not scaled integers or byte encoded values etc you shouldn’t need to set anything else. The number of names should obviously match the number and the order of the values you are passing over serial.

I hope this helps a bit, apologies in advance if I’ve missed something.

Hi,

Thanks for responding ,

If i have understood , i need to add manually those lines in my emonhub.conf in my local emoncms website ?

I also have to create this in my emonhub.conf in the nodes part right ?

I’ve typed this command before and after unplugged the Arduinos to the RPI3 and this is the result , i don’t know how to interpretate that : with the command “lsusb” i can see that the arduinos are recognized .

emonpiUSB

For the sketch , i don’t understand what i have to add or change (I know it’s some Serial.print subtlety but i don’t figure it out)

Thanks again for your help , i begin to understand some things that i did’nt understand before

lines like those yes, I was giving you examples and explaining the content so you could make your own decisions.

You can edit emonhub.conf either via the emoncms gui or manually from the command-line, both methods edit the same file, there is just one emonhub.conf.

If you compare the output of the “before” and the “after” you can see there are 2 lines in the “before” that are not in the “after”.

Those 2 lines tell you what the addresses are for your 2 Arduino’s as they are only there when the Arduino’s are connected. (please cut and paste the command output rather than post pictures)

So, going back to the edits you need to make in emonhub.conf, the 2 “com_port =” lines need to use the addresses “/dev/ttyACM0” and “/dev/ttyACM1” as determioned from the test above.

The code block under where I outlined the changes needed, has been changed to demonstrate the required changes. Compare the code block you provided with the code block I replied to see the changes.

The nodenames, the nodeids and the names of the values (value1, value2 etc) are all down to what you prefer, I do not know what any of the values are, nor do I know what you would chose to call each device, that’s your decision. Rather than spell out exactly the changes you need for your setup, I’m trying to explain what and why so that you have the understanding to make those changes to suit your needs better.

2 Likes

I’ve found how to change my “com_port” :

[[RFM2Pi]]
    Type = EmonHubJeeInterfacer
    [[[init_settings]]]
        com_port = /dev/ttyACM0
        com_baud = 38400                        # 9600 for old RFM12Pi
    [[[runtimesettings]]]
        pubchannels = ToEmonCMS,
        subchannels = ToRFM12, 
[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

I have those kind of values and i’m wondering if i have to adapt “names” according to my sketch.

Thanks for responding , you allowed me to understand things that I did not assimilate before

I’m returning to you if i have some questions

No, not quite there!

That is the “jee” interacer configured for the emonPi or RFM2Pi boards. Re-read this thread as I have explained that interfacer is no longer required if you do not have and emonPi/rfm2pi. You need to create 2 new interfacers of the EmonHubSerialInterfacer type not re-use the 1 EmonHubJeeInterfacer.

You have quoted the node [[5]] configuration, yes you could indeed reuse that one node id it you need to, but there are ample other numbers you could use as an id! All the other stuff in that [[5]] definition is of no use to you. The reusable bits are

    nodename =
    [[[rx]]]
        names =
1 Like
[[ArduinoOnUSB0]]
        Type = EmonHubSerialInterfacer
        [[[init_settings]]]
            com_baud = 9600
            com_port = /dev/ttyACM0
        [[[runtinesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12, 

[[ArduinoOnUSB1]]
        Type = EmonHubSerialInterfacer
        [[[init_settings]]]
            com_baud = 9600
            com_port = /dev/ttyACM1
        [[[runtinesettings]]]
            pubchannels = ToEmonCMS,
            subchannels = ToRFM12, 

It’s like that if i’ve understood ?

And for my sketch i’ve done that :

void loop()
{
  //Add a leading node id eg 
  Serial.print("5");
  Serial.print(" ");
  
  // calculer et afficher le courant 1
double Irms1 = emon1.calcIrms(1480);  // Calculate Irms only
Serial.print(Irms1*24.0);         // Apparent power 
  Serial.print(" ");
  Serial.println(Irms1);          // Irms
  //FIN 

Thanks for your help

That should work ok!

that’s only the “leading node id” part of the changes

. . . all the changes

Think about what your sketch code is doing. Serial.print() just prints to serial, whereas Serial.println() prints to serial and then creates a new line. The output you are after is

nodeid val1 val2 val3 val4
nodeid val1 val2 val3 val4

1 Like

Hi,

I have now this for my sketch :

void loop()
{
  int power1;
  int power2;
  int power3;
  int power4;
  //Add a leading node id eg 
  Serial.print("6");
  Serial.print(" ");
  
  // calculer et afficher le courant 1
double Irms1 = emon1.calcIrms(1480);// Calculate Irms only
power1 = Irms1*24.0 ;
Serial.print(power1);         // Apparent power 
  Serial.print(" ");
  //Serial.println(Irms1);          // Irms
  //FIN

   // calculer et afficher le courant 2
double Irms2 = emon2.calcIrms(1480);  // Calculate Irms only
power2 = Irms2*24.0 ;
Serial.print(power2);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms2);          // Irms
  //FIN

   // calculer et afficher le courant 3
double Irms3 = emon3.calcIrms(1480);  // Calculate Irms only
power3 = Irms3*24.0 ;
Serial.print(power3);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms3);          // Irms
  //FIN

   // calculer et afficher le courant 4
double Irms4 = emon4.calcIrms(1480);  // Calculate Irms only
power4 = Irms4*24.0 ;
Serial.print(power4);         // Apparent power
  Serial.print(" ");
  //Serial.println(Irms4);          // Irms
  //FIN

Serial.println();    
} 

But it seems that it does not work either with the changes , i’ve this for the log:

For the nodeid i’ve choosen “6” cause it seems that the number of variables matches with the number of variables that i want :

[[5]]
    nodename = emonpi
    [[[rx]]]
        names = power1,power2,power1pluspower2,vrms,t1,t2,t3,t4,t5,t6,pulsecount
        datacodes = h, h, h, h, h, h, h, h, h, h, L
        scales = 1,1,1,0.01,0.1,0.1,0.1,0.1,0.1,0.1,1
        units = W,W,W,V,C,C,C,C,C,C,p

[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units =W,W,W,W,V

I’m not sure of what you’re meaning by this(i’m not at ease in English with technical things sorry) :

Thanks for responding,

In your sketch, you have:

  Serial.print("6");
  Serial.print(" ");
 ...
  Serial.print(power1);         // Apparent power 
  Serial.print(" ");
...
  Serial.print(power2);         // Apparent power
  Serial.print(" ");
...
  Serial.print(power3);         // Apparent power
  Serial.print(" ");
...
  Serial.print(power4);         // Apparent power
  Serial.print(" ");

  Serial.println();    

That will print, for example

6 123.45 234.56 345.67 456.78 :leftwards_arrow_with_hook:

So that emonhub correctly interprets the data, you must have exactly the same number of data as you have in your sketch. You have 4 items of data. The NodeID is not data.

Therefore, you must have:

[[6]]
    nodename = Arduino1
    [[[rx]]]
       names = power1, power2, power3, power4
       datacode = 0
       scales = 1,1,1,1
       units =W,W,W,W

You must have “datacode = 0” because the numbers are readable by a person - they are not byte data that must be decoded.

@pb66 - Am I correct here?