Arduino pull data from emoncms

Good morning!

I have a running emoncms server with raspberry pi and emonsd. I use it as database, it-s receiving multiple data of my home. Some arduino-s ar sending data to it via ethernet. Power, temperature humidity… It-s monitoring my heating system status, etc.

Now I am searching what can I do with this data, I would like to configure an independent arduino to access emoncms and get some data to do actions. Easy example would be, access emoncms and get temperature, if it is lower than x, turn on a led. Or get all data and show it on a lcd… etc…

How can I do somthing like that, I have been searching on the forum and found this working url http://192.168.1.30/emoncms/feed/value.json?apikey=022a67a6d8899ea1913xxxxxxx=33 it returns me the feed data, but how to store it in a variable of arduino.

I have node red on emonsd too, perhaps it could send data to arduinos, any recomendation or arduino sketch, project that I can see to start?

Thanks.

I am using the following code to get data from emoncms using arduino:

String sss = "";
while (client3.available()) {
    char c = client3.read();
    //Serial.print(c);
    sss +=c;    
} 

if (millis() - lastAttemptTime > requestInterval) {
  getTodays();
  int z = sss.lastIndexOf("."); // parses today's kwh out of web response
  String q = sss.substring(z-1,z+5);
  if (q != "" and z > 5){
      //Serial.println(q);
      lcd.setCursor(14,0);
      lcd.print(q);
  }  
}


void getTodays (){
  if (client3.connect(server2, 80)) {    
    //Serial.println("CL3 is called out");
    client3.print("GET /emoncms/feed/value.json?id=15&apikey=");
    client3.print(apikey2);
    client3.println(" HTTP/1.1");
    client3.println("Host:aa.com");
    client3.println("User-Agent: Arduino-ethernet");
    client3.println("Connection: close");
    client3.println();
    //return readPage(); //go and read the output
  }

Thanks! I will try!

Hi again recepta, I have been playing with your code, but I get the value 6.7" istead of 16.7, it’s a ºC temperature. I have been trying but I amb a little noob with arduino code and don’t understand string sss, int zm string q… I need to get 4 different values from emoncms. If i try the GET on the browser I get the 16.7 correct value.

How to get 4 different id’s values and store in 4 arduino variables?

That is getting only one character before the decimal point (‘.’). My guess is @recepta did not expect you to use it for °C. You might find changing it to
String q = sss.substring(z-2,z+5);
works, but I would be much happier looking for the character that separates the two fields, rather than looking for a ‘.’ inside a field.

“sss” is a string. The first block builds the string by reading characters one at a time and adding them until there are no more:
String sss = ""; while (client3.available()) { char c = client3.read(); //Serial.print(c); sss +=c; }

Next, he finds the last occurrence of ‘.’ It’s at position ‘z’.
int z = sss.lastIndexOf(".");

Then he copies out part of string “sss” and calls it “q”, from 1 character before the ‘.’ to 5 after:
String q = sss.substring(z-1,z+5);

That could be quite hard. Assuming it all comes as one big string, you need to step through the string, looking for the character that is used to separate each field, then use ‘substring’ to lift out each nodeID and the corresponding value in turn.

just a thought – not sure how important the min & max temp is for you -maybe for security reasoning maybe have the device take the reading execute the condition or have the device that takes the reading send the info directly to the secondary device executes the condition . just that the more layers you have the more chance of failure if it critical condition that you are worried about.

Thanks all,

stephen it’s very too important, I am only trying to show emoncms values for informative reasons, at this moment. I will try to visualize some values at home, like interior/exterior temperature and humidity, power, kwh…

Robert thanks for your tips and for explain, I got it showing 2 values for now it’s working, but I have to improve it a little more because with feed/fetch I get multiple feed list of values like this:

[“273.92”,“160.01134265234”,“234.28”,“18.5”,“25.2”]
Power, kwh, V, T, H

I locate the position of the last colon “.” of the strig, I said it p1, and in reference of it I can define Te, He. It’s showing well but the problem appears when a value of the list lost a digit, for example T is lower than 10ºC so it loses the 1 digit, all the sentence will change and all the first values will got broken. Perhaps can I force via emoncms feed list the digits of every value? Will try.

String sss = "";
while (client.available()) {
    char c = client.read();
//    Serial.print(c);
    sss +=c;    
} 

 //if(!client.connected() && (millis() - lastConnectionTime > getingInterval)) {
 if(millis() - lastConnectionTime > requestInterval) { 
  getmultiplefeeds();
  int p = sss.lastIndexOf("."); 
  String He = sss.substring(p-2,p+2);
  String Te = sss.substring(p-9,p-5);


  Serial.print(Te);
  Serial.print("ºC  "); 
  Serial.print(He);
  Serial.print("%  "); 
  Serial.println();


      lcd.setCursor(10,0);
      lcd.print(Te);
      lcd.setCursor(10,1);
      lcd.print(He);

}



void getmultiplefeeds (){
  if (client.connect(server, 80)) {    
    //Serial.println("CL3 is called out");
    client.print("GET /emoncms/feed/fetch.json?ids=33,34,35,39,40&apikey=");
    client.print(apikey_read);
    client.println(" HTTP/1.1");
    client.println("Host:aa.com");
    client.println("User-Agent: Arduino-ethernet");
    client.println("Connection: close");
    client.println();
    //return readPage(); //go and read the output
  }
   }

What you need to do there is search along the string, picking out each value. You cannot search for ‘.’ because you do not know how many characters that you want come before it, nor after it.
Study the Arduino documentation for “StringObject”. The English version is String() - Arduino Reference

Here is a very useful method: indexOf() - Arduino Reference
string.indexOf(val, from)

You will need to use that many times. Start with from = 0 and search for the quote ( " )
The first value starts one character after the number that indexOf returns.
Do it again, this time with from equal to one more than the start. That will find the end of the first value. Then do substring to get the first value.

This is not the most elegant way to do it, but it illustrates what you must do:
strings.ino (860 Bytes)

[Edit - didn’t get it quite right the first time, it did not handle an empty value correctly.]

Sorry, I was busy with the finals at the college. Do you still need help parsing the values?

It would be easy to parse this strings.
Does your response actualy contains " in the json response?

Don’t worry. That’s MUCH more important than this. (I did mine 40+ years ago, I know what it’s like.) We’ll see what Ernest says.

If it does not, then Ernest must search for ( , ), and set sstart and send (my demo sketch) relative to the position of the ( , ) before and after the section of sss that he wants.
The principle remains the same.

Somehow for me, it seems like it is never ending education. Once I finish something, I come up with bright ideas to study something else :slight_smile:

There will be a lot of ( , ) in the actual response from the web. Response will include user agent, encoding, and a bunch of other things. Hopefully ( [ ) is only used to encapsulate actual data. Before they “broke” emoncms, only actual data was encapsulated with ( " ) so it was very easy to parse it.

I don’t have a working emoncms, so all I can do is answer the question that was asked. If the string read contains much more, but the wanted data is enclosed by ( [ ) and ( ] ), then the obvious way is to find those, copy out the substring between and then process that.

Hi all,
I don’t have time to test this until weekend, but I think that understand your method Robert. Thanks for the documentation and for the sample sketch, I will study it on weekend.

This is the exactly JSON reply I get:
[“550.43”,“168.26346562226”,“230.43”,“16.1”,“47.9”]

Hi folks,

This weekend I have been trying with Robert.Wall code, and it’s working perfect. I got all the string varaibles on the receiver arduino. It’s just perfect! Thank you very much. :grinning:

1 Like

Hi all,

I am still working with the same project, Now I can get the arduino data with the string and get the individual values right. But I am trying to do some actions with this values and I get an strange effect that I don’t understand.

One of this values is pg status, it’s a door status, open or close, 0 or 1. I am trying to enable a green led with an arduino output when the door is closed, and enable antoher red led when the door is open.

When the door is closed, the led red is working well (value it’s not equal to 1 for arduino), green led is enabled and stable, but when the door is open, the variable is 1, and the red led is not stable, I get a flashing effect, red green red green… it’s like the 1 value it’s not stable.

How can I correct this?

Here is the part of the code mentioned:

 //GET ARDUINO DATA
String sss = "";
while (client.available()) {
    char c = client.read();
//    Serial.print(c);
    sss +=c;    
} 

 //if(!client.connected() && (millis() - lastConnectionTime > getingInterval)) {
 if(millis() - lastConnectionTime > requestInterval) { 
  getmultiplefeeds();

String potencia, kwh, v, te, he, pg;
int sstart = 0, send = 0;

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
potencia = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
kwh = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
v = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
te = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
he = sss.substring(sstart, send);

Serial.println(he);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
pg = sss.substring(sstart, send);

//LEDS OUTPUT
if(pg.toInt() == 1) {  
    digitalWrite(ledvermell, HIGH);        
    digitalWrite(ledverd, LOW);

  }
  else{                                   
    digitalWrite(ledverd, HIGH);
    digitalWrite(ledvermell, LOW);       
  }

Are the LEDs doing exactly as they should, and is the problem that “pg” is getting a wrong value?
How often is that section of code executed? If it is only every time emonCMS gets updated, that is good. If not, does emonCMS still have the old value? Does “sss” become an empty string, hence pg will be false? That will happen if your code gets executed and there’s no data available.

These are all things to look at. Not knowing how the data gets from emonCMS to the Arduino, I can’t help much more than point you in the general direction of things to check for.

Thanks!

I have more information. There is a first arduino at garage posting data to emoncms every 5s. Door status, temperature, power, etc…

The second arduino, at home, is posting some data every 5s and is pulling the string every 0,5s from the emoncms. Do you think this could be the problem? I suposed that the information on emoncms is stable until it’s updated every 5s. For example if the first arduino post an 1, it is stable on emoncms until there is another update at 5s with a 0. The second arduino is geting 1, 1, 1, 1… every 0,5. Perhaps the problem is here as you suggested. I could try to update emoncms every 0,5s?

Why there is no data available, 1 or 0 it’s not ever there?

Here is the complete code of pulling arduino:

//LLIBRERIES
  #include <Ethernet.h>
  #include <DHT.h>;
  #include <Wire.h> 
  #include <LiquidCrystal_I2C.h>

//LEDS
int ledverd = 6;
int ledvermell = 7;


//CONSTANTS DHT
  #define DHTPIN 5     // pin del sensor DHT
  #define DHTTYPE DHT22   // tipus de sensor DHT
  DHT dht(DHTPIN, DHTTYPE); //// Inicialitzar sensor DHT 16mhz Arduino

//VARIABLES DHT
  int chk;
  float hum;  //Enmagatzema valor humitat
  float temp; //Enmagatzema valor temperatura

//LIQUID CRYSTAL
#define BACKLIGHT_PIN     13

LiquidCrystal_I2C lcd(0x3F, 2, 1, 0, 4, 5, 6, 7, 3, POSITIVE); // Set the LCD I2C address

// Creat a set of new characters
const uint8_t charBitmap[][8] = {
   { 0xc, 0x12, 0x12, 0xc, 0, 0, 0, 0 },
   { 0x6, 0x9, 0x9, 0x6, 0, 0, 0, 0 },
   { 0x0, 0x6, 0x9, 0x9, 0x6, 0, 0, 0x0 },
   { 0x0, 0xc, 0x12, 0x12, 0xc, 0, 0, 0x0 },
   { 0x0, 0x0, 0xc, 0x12, 0x12, 0xc, 0, 0x0 },
   { 0x0, 0x0, 0x6, 0x9, 0x9, 0x6, 0, 0x0 },
   { 0x0, 0x0, 0x0, 0x6, 0x9, 0x9, 0x6, 0x0 },
   { 0x0, 0x0, 0x0, 0xc, 0x12, 0x12, 0xc, 0x0 }
   };
  

//ETHERNET
  byte mac[] = {xxxxx};

 
  IPAddress ip(xxxx);   // fill in an available IP address on your network here, if dhcp auto config fail 
  IPAddress subnet(xxxx);
  IPAddress DNS(xxxx);
  IPAddress gw(xxxx);
  EthernetClient client;  // initialize the library instance:


//CONFIGURACIÓ EMONCMS
   //char server[] = "emoncms.org";      // name address for emoncms.org
  // O
  IPAddress server(xxxxxxx);  // numeric IP for emoncms.org (no DNS)

  String apikey = "1dddddddd";  //api write key
  String apikey_read = "0xxxxxxx";  //api read key
  int node = 2; //if 0, not used

  unsigned long lastConnectionTime = 0;          // last time you connected to the server, in milliseconds
  boolean lastConnected = false;                 // state of the connection last time through the main loop
  const unsigned long postingInterval = 5000;  // delay between updates, in milliseconds
  const unsigned long requestInterval = 500;  // delay between updates, in milliseconds




void setup() {
  
 Serial.begin(9600);
 Serial.println("ARRENCANT ARDUINO CONTROL MENJADOR");

//LEDS

  pinMode(ledverd, OUTPUT);
  pinMode(ledvermell, OUTPUT); 
 

//CONNECTAR ETHERNET
Ethernet.begin(mac, ip, dns, gw, subnet);

//LIQUIDCRYSTAL
 lcd.begin(20, 4);              // arrenca la llibreria

//ARRENCAR SENSOR DHT
  dht.begin();


  imprimirestat(); //imprimir estat per port serie

}
// fi de void setup



void loop() {


 //ETHERNET 
  // if there's incoming data from the net connection.
  // send it out the serial port.  This is for debugging
  // purposes only:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if there's no net connection, but there was one last time
  // through the loop, then stop the client:
  if (!client.connected() && lastConnected) {
    Serial.println();
    Serial.println("Disconnecting...");
    client.stop();
  }


//LLEGIR VARIABLES HUM I TEMP DHT
// Wait a few seconds between measurements.
  //delay(2000);
  // Reading temperature or humidity takes about 250 milliseconds!
  // Sensor readings may also be up to 2 seconds 'old' (its a very slow sensor)
  hum = dht.readHumidity();
  temp= dht.readTemperature();  


  // if you're not connected, and at least <postingInterval> milliseconds have
  // passed sinceyour last connection, then connect again and
  // send data:
  if(!client.connected() && (millis() - lastConnectionTime > postingInterval)) {


// IMPRIMIR DADES PORT SERIE 
  Serial.print(temp);
  Serial.print("ºC  "); 
  Serial.print(hum);
  Serial.print("%  "); 
  Serial.println();
  

  enviardades();
    }
    
  // store the state of the connection for next time through the loop:
  lastConnected = client.connected();



 //REBRE DADES EMONCMS
String sss = "";
while (client.available()) {
    char c = client.read();
//    Serial.print(c);
    sss +=c;    
} 

 //if(!client.connected() && (millis() - lastConnectionTime > getingInterval)) {
 if(millis() - lastConnectionTime > requestInterval) { 
  getmultiplefeeds();

String potencia, kwh, v, te, he, pg;
int sstart = 0, send = 0;

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
potencia = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
kwh = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
v = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
te = sss.substring(sstart, send);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
he = sss.substring(sstart, send);

Serial.println(he);

sstart = sss.indexOf('"', send+1) + 1;
send = sss.indexOf('"', sstart);
pg = sss.substring(sstart, send);

//LEDS
if(pg.toInt() == 1) {  
    digitalWrite(ledvermell, HIGH);        
    digitalWrite(ledverd, LOW);

  }
  else{                                   
    digitalWrite(ledverd, HIGH);
    digitalWrite(ledvermell, LOW);       
  }



// IMPRIMIR DADES PORT SERIE 
  Serial.print(potencia);
  Serial.print("W  ");
  Serial.print(kwh);
  Serial.print("kWh  "); 
  Serial.print(v);
  Serial.print("V  "); 
  Serial.print(te);
  Serial.print("ºC  ");  
  Serial.print(he);
  Serial.print("%  ");
  Serial.print(pg);
  Serial.println();

// IMPRIMIR DADES LIQUIDCRYSTAL
  
  
if(potencia < 190) {  
  lcd.noBacklight();
  }

  else{                                   
  lcd.backlight();    
  }

  
//fila 0
  lcd.setCursor(0,0);
  lcd.print("EXT");
  lcd.setCursor(4,0);
  lcd.print(te);
  lcd.setCursor(8,0);
  lcd.print("C");
  lcd.setCursor(11,0);
  lcd.print(he);
  lcd.setCursor(15,0);
  lcd.print("%");

//fila 1
  lcd.setCursor(0,1);
  lcd.print("INT");
  lcd.setCursor(4,1);
  lcd.print(temp);
  lcd.setCursor(8,1);
  lcd.print("C");
  lcd.setCursor(11,1);
  lcd.print(hum);
  lcd.setCursor(15,1);
  lcd.print("%");

//fila 3
  lcd.setCursor(2,2);
  lcd.print(potencia);
  lcd.setCursor(9,2);
  lcd.print("W");
  lcd.setCursor(11,2);
  lcd.print(v);
  lcd.setCursor(17,2);
  lcd.print("V");

//fila 4
  lcd.setCursor(0,3);
  lcd.print("GARATGE");

if(pg.toInt() == 0) {  
  lcd.setCursor(8,3);
  lcd.print("TANCAT");
  }
  else{                                   
  lcd.setCursor(8,3);
  lcd.print("OBERT");     
  }



      

}
  
}
//fi de void loop



//ENVIAR DADES EMONCMS
void enviardades() {
    // if there's a successful connection:
if (client.connect(server, 80)) {
    Serial.println("Connectant...");
    // send the HTTP GET request:
    client.print("GET /emoncms/input/post?apikey=");
    client.print(apikey);
    if (node > 0) {
      client.print("&node=");
      client.print(node);
    }
    client.print("&json={");
    client.print("Temperatura_menjador:");
    client.print(temp);
    client.print(",Humitat_menjador:");
    client.print(hum);  
    client.println();
    
    // note the time that the connection was made:
    lastConnectionTime = millis();
  } 
  else {
    // if you couldn't make a connection:
    Serial.println("Conexió fallada");
    Serial.println("Desconnectant...");
    client.stop();
  }
}



//IMPRIMIR ESTAT PER PORT SÈRIE
void imprimirestat() {
    // print your local IP address:
    Serial.print("IP address: ");
    for (byte thisByte = 0; thisByte < 4; thisByte++) {
      // print the value of each byte of the IP address:
      Serial.print(Ethernet.localIP()[thisByte], DEC);
      Serial.print("."); 
    }  
  Serial.println();
}

//REBRE DADES EMONCMS
void getmultiplefeeds (){
  if (client.connect(server, 80)) {    
    //Serial.println("CL3 is called out");
    client.print("GET /emoncms/feed/fetch.json?ids=33,34,35,39,40,51&apikey=");
    client.print(apikey_read);
    client.println(" HTTP/1.1");
    client.println("Host:aa.com");
    client.println("User-Agent: Arduino-ethernet");
    client.println("Connection: close");
    client.println();
    //return readPage(); //go and read the output
  }
   }




void getsinglefeed (){
  if (client.connect(server, 80)) {    
    //Serial.println("CL3 is called out");
    client.print("GET /emoncms/feed/value.json?id=39&apikey=");
    client.print(apikey_read);
    client.println(" HTTP/1.1");
    client.println("Host:aa.com");
    client.println("User-Agent: Arduino-ethernet");
    client.println("Connection: close");
    client.println();
    //return readPage(); //go and read the output
  }
   }