Sending data from Arduino to RasberryPi using Serial

Hi,

I want to send my Data Logged by an Arduino Mega 2560 using the USB to UART Cable to a RaspberryPi3 with an emoncms installation (latest Version) an log it ehre remotely. I figured this should be pretty easy using the serialprint() function.
I allready found this documentation on how to do this. Sadly it is refering to the deprecated gatewayconfig of Jerome.
Then I found within the examples of emonHUB (sadly documentation links on git are dead ends) this examples for a directserial connection:

Emonhub can read from a serial device directly e.g. /dev/ttyUSB0

Data Format

Data should be printed to serial (integer only) with space sperators in the format:

NODEID VAR1 VAR2 VAR3 …and so on

Here is an example of printing data from an Arduino sketch:

Serial.print(nodeID);     Serial.print(' ');
Serial.print((int)(realPower1)); Serial.print(' ');   // These for compatibility, but whatever you need if emonHub is configured to suit.
Serial.print((int)(realPower2)); Serial.print(' ');
Serial.print((int)(realPower3)); Serial.print(' ');
Serial.print((int)(realPower4)); Serial.print(' ');
Serial.print((int)(Vrms*100));

Example emonhub Config

In the [interfacers] section:

[[SerialDirect]]
     Type = EmonHubSerialInterfacer
      [[[init_settings]]]
           com_port = /dev/ttyUSB0
           com_baud = 9600
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,

Debugging

  • Ensure the data received on the serial port is 100% numerical characters with no rogue spaces. Non-numerical characters could result in Thread is dead error.

My RaspberryPi3 didn’t recognice the Mega on the dev/ttyUSB*. After some research I found that they sometimes appear on dev/ttyACM0. On this threat franknfurther seemed to have a similar experience. But this thread is also reffering to the deprecated gatewayconfig.
The emonhublog told me that the connection was succesfull with this path. Another good indicator seems to be the error message “Thread is dead” when I unplug the Arduino. Here is the testsketch running on the Arduino, based on the code I found in the example above (I also tried the version with the (int) within the serialfunction; same results):

const int nodeID = 6;

void setup() {
Serial.begin(9600);
}

void loop() {

int a = 1;
int b = 2;
int c = 3;
int d = 4;
int e = 107;

Serial.print(nodeID); Serial.print(’ ');
Serial.print(a); Serial.print(" “); // These for compatibilit$
Serial.print(b); Serial.print(” “);
Serial.print(c); Serial.print(” “);
Serial.print(d); Serial.print(” ");
Serial.print(e);

delay(1000);

}

This triggers nothing at all within emoncms. No emonHUB log, no inputs created. I also checked via ssh directly on the pi with the command tail -f /dev/ttyACM0 , which should print whats actually received on this path. Nothing.
The LED on the Mega shows that its actually sending. Connected to the IDE I also get the expected result when I check the Serial Monitor.

Whats wrong here? What am I missing?
Assuming emoncms would receive Data, would i need to make further changes for logging remotely?

Thanks for your help.

Regards,
kaulquappe

Start looking around this post down to the end of the thread: Direct connection emonTx V3 3-phase to RaspberryPi - #12 by johnbanks

That is correct, most of the serial-direct implementations on these forums relate to connecting an emonTx (or similar) to the Pi via serial, usually using the Pi’s GPIO. When connecting the serial ports directly the port will usually be ttyAMA0 and when connecting via a USB-serial adapter it will usually be ttyUSB0 etc. Since you are connecting an Arduino device via it’s own USB port it is recognised as a “ACM” (abstract control model), it will work the same way but you just need to use the right address.

Space separated values with a leading node ID is correct, but they do not need to be ints, they can be any datatype or indeed just be normal human readable numbers.

6 123 456.89 0.123 45.6 7.89 0.123456789 1234567890
``
is a totally valid payload.

[quote="kaulquappe, post:1, topic:3822"]
The emonhublog told me that the connection was succesfull with this path. Another good indicator seems to be the error message "Thread is dead" when I unplug the Arduino.
[/quote]

If you have the log level set to "DEBUG" in emonhub.conf you should also get messages showing any received payload progressing through emonhub, including if it fails. If there are no signs of recieved data the probability is it isn't recieving the data. 

The serial connection must be exclusive to emonhub, if you have arduino IDE open on that serial port or minicom etc it will be anyones guess who will get the data, but it will only be one SW so ensure the other SW's are not active when testing emonhub.

Your example sketch should also work fine so I would expect there to be much more info available in emonhub.log.

Where is emonCMS? If you are using the emonSD image there is a known issue with the MQTT implementation in emonCMS that prevents new nodes showing up, this can usually be gotten around by rebooting the Pi. This will only work if you are seeing the values being published to emonCMS in the emonhub.log.

EDIT - Also make sure if there is an entry in the [nodes] section of emonhub for the node you are using (ie 6) that it's definition matches the payload you are sending. If sending un-coded values (human-readable) then no decoding is required and `datacode = 0` can be used to specify no decoding (ie zero decoding). 

If the definition doesn't fit the data it will discard the data but that will be logged as an error in emonhub.log, if sending data remotely a "suspect" definition can just be removed (commented out) to test but If using the emonSD defining the node is mandatory as emoncms will not work well with numeric node ids or value labels/indexes via MQTT. If using original emonhub the definition is not mandatory.

Thanks for the response.

The first one, is not quite what I need I guess. Read through the threat but found the topics discussed there just partially related to mine.

Second:

log level is DEBUG. No response though.

Serial connection is exclusive to emonHUB, using the Arduino IDE for checking the actual Serial Output was not done simultaniusly. What do you mean by SW’s.

emonCMS is running on a USB-Flashdrive, to prevent the SD-Card from beeing killed. The Usage of the whole System will not create much Data, so USB-Flashdrive will work. Tried rebooting quit often.

I took nodID 6 on purpose, because it expects a Payload of like NodeID val1 val2 val3 val3 val4 val5. 6 is the second of the nodes in the config file. As you pointed out, there should at least be dropped a error message on the log. So i still assume, there is no data coming in at all.

Struggling with your last sentence, because of language and lack of knowledge. From my point of View, without changing the nodees in the config file, the ID 6 should work the way I arranged my datatransmission. Right?

If you are refering to this config

[[6]]
    nodename = emontxshield
    [[[rx]]]
       names = power1, power2, power3, power4, vrms
       datacode = h
       scales = 1,1,1,1,0.01
       units =W,W,W,W,V

No that will not work with your payload (but there should be log messages to that effect) datacode = h says the payload will consist of an unspecified but even number of byte values between 0 and 255, and that each pair of byte values represent a 16bit signed integer. So 5 signed ints would be sent in a payload of 10 byte values.

You are sending full human readable values so no decoding is required. use datacode = 0 for no decoding.

Am I right in assuming you are using the correct address in the conf file as well as the Arduino ID ie

[[SerialDirect]]
     Type = EmonHubSerialInterfacer
      [[[init_settings]]]
           com_port = /dev/ttyACM0
           com_baud = 9600
      [[[runtimesettings]]]
           pubchannels = ToEmonCMS,

All right. Changed it to 0. But as you already expected that didn’t change anything, because no Data is coming in, in the first place.

My Config file looks like that, yes. To be sure I also out commented the [RFM2Pi] section. And yes it is the same adress as shown in my IDE.

Can you attach some emonhub.log starting from a restart of emonhub

Ps - “SW” is just shorthand for software

Yes. This is my Log after a successful connection.

2017-04-25 17:24:22,486 INFO     MainThread EmonHub emonHub 'emon-pi' variant v1.2
2017-04-25 17:24:22,486 INFO     MainThread Opening hub...
2017-04-25 17:24:22,487 INFO     MainThread Logging level set to DEBUG
2017-04-25 17:24:22,487 INFO     MainThread Creating EmonHubSerialInterfacer 'SerialDirect' 
2017-04-25 17:24:22,489 DEBUG    MainThread Opening serial port: /dev/ttyACM0 @ 9600 bits/s
2017-04-25 17:24:22,490 DEBUG    MainThread Setting SerialDirect pubchannels: ['ToEmonCMS']
2017-04-25 17:24:22,490 DEBUG    MainThread Interfacer: Subscribed to channel' : ch2
2017-04-25 17:24:22,491 INFO     MainThread Creating EmonHubMqttInterfacer 'MQTT' 
2017-04-25 17:24:22,492 INFO     MainThread MQTT Init mqtt_host=127.0.0.1 mqtt_port=1883 mqtt_user=emonpi
2017-04-25 17:24:22,494 DEBUG    MainThread MQTT Subscribed to channel' : ToEmonCMS
2017-04-25 17:24:22,495 INFO     MainThread Creating EmonHubEmoncmsHTTPInterfacer 'emoncmsorg' 
2017-04-25 17:24:22,496 DEBUG    MainThread emoncmsorg Subscribed to channel' : ToEmonCMS
2017-04-25 17:24:22,596 INFO     MQTT       Connecting to MQTT Server
2017-04-25 17:24:22,599 INFO     MQTT       connection status: Connection successful
2017-04-25 17:24:22,600 DEBUG    MQTT       CONACK => Return code: 0
2017-04-25 17:24:22,702 INFO     MQTT       on_subscribe

Arduinos TX-LED is blinking as it is actually sending something. It’s not doing so if there was a failure while connecting, I figured so far.

What does this command return?

sudo ls -la /dev/{tty{ACM,AMA,S,USB},serial}*

is the Arduino IDE on the same device? Or were you testing on another machine?

Do you get any output if you try using minicom (stopping emonhub while you do of course)

Does the log change at any point after the clip you posted?

That clip is less than 0.25 of a second so not conclusive in itself that there is no data.

This is what is returned to your command:

ls: cannot access /dev/ttyUSB*: No such file or directory
crw-rw---- 1 root dialout 166,  0 Apr 25 17:39 /dev/ttyACM0
crw-rw---- 1 root dialout 204, 64 Apr 25 17:39 /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Apr 25 17:39 /dev/ttyS0

/dev/serial:
total 0
drwxr-xr-x  4 root root   80 Apr 25 17:39 .
drwxr-xr-x 17 root root 3520 Apr 25 17:52 ..
drwxr-xr-x  2 root root   60 Apr 25 17:39 by-id
drwxr-xr-x  2 root root   60 Apr 25 17:39 by-path

Arduino IDE runs on my computer, which also talks to the Pi via SSH.
Tried minicom. But not with stopped emonHUB. How can I stop it?

Log does not change at all. Ever. What do you mean by:

I’m sorry for not understanding.

so I tried minicom after rebooting the whole system, but before logging into emonCMS. I assume emonHUB is not up then. minicom is not displaying anything.

BUT the TX-LED is not blinking, as it first starts when emonHUB actually connected to the device. So I assume there is no Data sent.

As I am powering the Arduino with USB and thereby over the Pi, I was actually thinking if the power drain could be a problem. But as far as the System on the Pi runs stable, I didn’t think to deep into that. What are your thoughts on that?

Sorry for disappearing last night, something unexpected cropped up and i had to shoot out.

sudo service emonhub stop

and to start it again afterwards

sudo service emonhub start

I was just saying the emonhub.log you posted was less than 0.25 seconds in length, the fact we see no serial data in the first 0.25 seconds doesn’t prove that the serial isn’t working as even your example only send data every 1 second. That is why I asked if there are any other log messages after a time has passed to confirm.

No that isn’t the case. emonhub starts at boot up and runs independently of emoncms so you would need to stop emonhub and start it again afterwards.

Personally I would not expect a problem powering via the Pi’s USB but I cannot be sure. I would only expect a problem if the Pi’s PSU wasn’t up to powering both the Pi and the Arduino, but in that instance I would also expect the Pi to be struggling too. Do you have another PSU to try?

Do you know how much power the Arduino requires? Upto 500mA should be no problem via a Pi’s USB and there is also a max_usb_current=1 in the /boot/config.txt at the end for allowing upto 1000mA to be drawn from USB if the Pi’s PSU can supply that demand.

It’s a long shot but you could also try using address /dev/ttyS0 just to rule it out.

I don’t know exactly, but I know it won’t be 1000 mA, more around 500mA. PSU of the Pi is capable of 2.1 A. So there shouldn’t be a problem.

Changed it in the config to ttyS0, but then emonHUB doesn’t want to connect. I think we are using the right Port with ACM0. We are just not reading it out.

Before using minicom, I just tried reading out the Port with:

cat /dev/ttyS0

But console stayed empty. (emonHUB was stopped)

I never used minicom before, so I am not sure if I am using it the right way. I was simply doing:

sudo minicom /dev/ttyACM0 9600

But I think this just opens a menu. So I started looking in how to use minicom using this Userguide.According to it I started with dmesg | grep tty what gave me this

[    0.000000] Kernel command line: 8250.nr_uarts=1 dma.dmachans=0x7f35 bcm2708_fb.fbwidth=720 bcm2708_fb.fbheight=480 bcm2709.boardrev=0xa02082 bcm2709.serial=0xbc32f20e smsc95xx.macaddr=B8:27:EB:32:F2:0E bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 vc_mem.mem_base=0x3ea00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=tty1 elevator=noop root=PARTUUID=57C7E94B-091C-4F42-9533-79DC7FFE8407 rootdelay=5 rootfstype=ext4 fsck.repair=yes rootwait
[    0.001364] console [tty1] enabled
[    1.737846] 3f201000.uart: ttyAMA0 at MMIO 0x3f201000 (irq = 87, base_baud = 0) is a PL011 rev2
[   10.298688] cdc_acm 1-1.5:1.0: ttyACM0: USB ACM device

Next step would be going into the setup of the Port and change settings there. I wasn’t sure if I should do that, so i didn’t.

So do I, I just thought we could rule it out as I’m not experienced with Arduino’s with on-board USB ie ACM’s.

I’m no minicom expert either as I do not use it often. Ithink the correct syntax might be

sudo minicom -D /dev/ttyACM0 -b 9600

You shouldn’t need to, but until we know what the issue is we cannot be sure.But I wouldn’t start changing anything yet as the emonPi image should be all set to go. What model Pi are you using? and can you confirm if your SDcard is a pre-built purchase, downloaded image or self built image ? Have you updated or changed anything outside what we have discussed here?

sudo minicom -D /dev/ttyACM0 -b 9600 gave me:
Device /dev/ttyACM0 is locked.

It’s a Pi3 modelB. It is a downloaded image of the latest version, which I flashed myself. As I want to use it locally, I installed the midori browser and wrote a simple startup script to automatically start the browser with the emoncms GUI:

#!/bin/sh
xset -dpms # disable DPMS (Energy Star) features.
xset s off # disable screen saver
xset s noblank # don't blank the video device
matchbox-window-manager &
midori -e Fullscreen -a http:localhost

I did this before, using the radiomodule for datatransmission, what worked fine. I think at some point I needed to install something like Xorg. Don’t recall it that good, because it was an easy apt-gett install. Then I moved emoncms to the flashdrive.

Could using the other Serialport, that the radiomodule would use, be an option?

Did you stop emonhub? do you have any other softwares trying to access /dev/ttyACM0 ? When did you last reboot?

It sounds like something else is using the port or there is a orphaned lock file from a previous failed or crashed attempt at using the port.

Try to see what (if anything) is using the serial port

sudo fuser /dev/ttyACM0

if that returns a PID (process number) and you are unaware of anything using the port, kill the process with

sudo kill -9 12345

where “12345” is the PID returned by fuser.

If no PID was returned see what is listed at

ls -la /var/lock/

or

ls -la /var/lock/lockdev

or maybe even

ls -la /tmp

is there anything related to /dev/ttyACM0?

if there is a stray lock file delete it eg

sudo rm ls -la /var/lock/lockdev/LCK..ttyACM0

and try Minicom again (with emonhub stopped)

I just restarted the whole system and tried minicom again. now I actually received:

`2 3 4 1076 1 2 3 4 1076 1 2 3 4 1076 1 2 3 4 1076 1 2 3 4 1076 1 2 3 4 1076 1 27…

I also saw its an ongoing flow. So we actually are receiving the expected Data.

BTW: you can use minicom with emonHUB up I just found out.

Yes it can work but since the serial port is for one-to-one comm it is not wise to allow conflicts to “sort themselves out” especially when debugging. It may have been such a conflict that caused a stray lock file.

I can now see a formatting issue that should look like this

6 1 2 3 4 107
6 1 2 3 4 107
6 1 2 3 4 107
6 1 2 3 4 107

the line ending is missing. Change your test sketch to

const int nodeID = 6;

void setup() {
Serial.begin(9600);
}

void loop() {

int a = 1;
int b = 2;
int c = 3;
int d = 4;
int e = 107;

Serial.print(nodeID); Serial.print(' ');
Serial.print(a); Serial.print(" "); // These for compatibilit$
Serial.print(b); Serial.print(" ");
Serial.print(c); Serial.print(" ");
Serial.print(d); Serial.print(" ");
Serial.print(e); Serial.println();

delay(1000);

}

the println will add the correct line ending and that will also help emonhub as it is waiting for that line ending to signify the end of the payload.

OK, wow this is ridiculous. This was one of the first things I changed in the Code right in the beginning. But as it didn’t solve the problem in the beginning (I think due to the datacode = 0 issue), i thought better moving back to the code in the example file. But the question remains why emonhub didn’t tell something about a wrong format. Right?

Anyway, THANKS A LOT.

PS: Maybe it would be good to change the directserial example in the examplefolder and the documentation of emonHUB. Also it could be pointed out there, that datacode needs to be set to 0. I think it is just talking about choosing the right NodeID.