Help Required on Getting Homebrew EmonPi Running

Hi Paul,
after a long delay due to family, work and a lot of travel (plus some frustration with lack of progress causing me to give up for a while), I have been working on the “connecting… please wait” issue and this is my progress to date:
You recall that I had boards made using the gerbers provided on Github, which I assume are the same as your commercial boards (same as Amir?). I have been over the boards as much as I can and haven’t found any obvious issue - the boards were made by a company that has produced other boards for me with no issues.
Could there be a small difference in the gerbers between the boards I found on Github and yours?
How can I check? - I realise that you don’t want to do a comparison, but I would be happy to do a comparison if you were able to supply your gerbers. I accept that you may not want to, and this is OK with me.

I have also built a second Emonpi board with just the basic Arduino components and there is no difference in result.
I have also obtained a USBASP and have programmed both boards with the test sketches I have found on the Raspi software and the results are as follows:
Blink_Led - works OK and led blinks 5 secs on/5 secs off

Pi_shutdown_LCD - short press and I get the LCD message Raspberry Pi Booting
Long press and I get the count down 5 to 0 and the LCD shows SHUTDOWN.

DS18B20 Test Sketch - doesn’t work (nothing on LCD).

Pi Shutdown_button_test - doesn’t work (nothing on LCD)

So, from these tests it appears that the EmonPi is able to load software and function, so I don’t now believe it is an EmonPi s/w load issue.

I have also loaded the latest software emonpi Build - EmonSD-13Jun18 and no change in behavior
(didn’t think it would, but tried anyway).
I have also ran the Emonpi update with no obvious issues.
(BTW, printout of the emoncms below - Seems to be connecting to MQTT OK)

View last entries on the logfile:/var/log/emoncms.log

Auto refresh Download Log Copy to clipboard
2018-08-05 05:18:04.246|WARN|phpmqtt_input.php|Not connected, retrying connection
2018-08-05 05:18:04.321|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0

So, I really don’t know where to go from here, and there is a common issue between myself and Amir’s, and this is we have both had boards made from your Github-published gerbers.
Do you have any other diagnostic software I could try, or what else should we try to overcome the issues? - happy to work with you online or offline (I can Skype at a mutually convenient time if you want to).

Alternatively if there is another Australian reading this post (and preferably lives in Sydney) I could communicate with them, and try to resolve the issues.
Either way, it would be good to close out this issue if others have similar problems down the track.

Paul, in your response to Amir, you said "the fix is already there for you - can you elaborate on what you mean?

As a side issue, I found that the red led on the RPi3 was either flickering or stayed on, and on researching this, I see it relates to a low voltage issue and changing the USB cable for a heavier cable seems to have cured this problem - I don’t think it was the cause but worth including anyway.

Regards,
Dave

After 18 days wait, I have received no response from Paul so I assume he must be away or otherwise distracted.
I have come to the conclusion that the problem is a lack of serial communication between the EmonPi board and the RPI3 as I can see pulsing on the TxD lead from the ATMega328 and nothing on the RxD.
I am aware of the need to change the Uart over from UART1 to UART0 and have done the requisite steps except when I enter the last step:

sudo systemctl disable hciuart
I get an error message - see below

looking at the config.txt file I see:

Disable Pi3 BT (Not used EMONSD Nov 2017) (Note there is no hashtag at the start of the line??)

#dtoverlay=Pi3-disable-bt

# Switch Pi3 Bluetooth function to use the mini-UART (ttys0) and restore UART0/ttyAMA0 over GPIO 14 & 15.
Note This may reduce the maximum useable baudrate.

dtoverlay = pi3-miniuart-bt

After exiting and trying to stop BT modem trying to use UART by entering the following command

sudo systemctl disable hciuart

I get: failed to disable unit: No such file or directory.

I think I am getting closer to resolving the issues (caused mainly due to my lack of experience), so once again, I would like some assistance and guidance to resolve this issue. (Note I have loaded emonpi Build - EmonSD-13Jun18)

Paul, Glyn, Trystan and anyone else, can you offer any guidance please?

Dave

If you put an ‘@’ in front of somebody’s user name, they will generally get an email to notify them of the mention. Without that, they might not notice your post.

Sorry, I did see your post but over the 127 days between my last reply and your next response, I forgot a fair amount of detail and did not have the time to review the whole thread again, I did intend to do that but I have been very busy and unable to get back to this (I still have not reread all of it), most of us only do this on a voluntary basis in our spare time and have our own jobs and lives too.

From that last reply in March…

Did you do this? Is the emonPi serial communications as expected and fully functional when tested standalone without the Pi?

Normal running the emonpi only transmits serial, emonhub will configure the emonpi board at start up using serial and at changes of settings only (unless you have set up the emonGLCD time transmissions). Just seeing a pulsing line doesn’t tell us if that is good data or not.

When we know exactly what is coming out of the emonpi and how it reacts to commands issued via the serial terminal (Arduino IDE) we will know better whether it might work with the Pi.

Going back to the “homebrew” discussion, I explained the fact that a “shop bought” unit would be tested to a certain degree before shipping, those tests will most likely be done via that 6pin FTDI header without attaching to a PI. That is the main reason why identifying a “homebrew” unit is important. We need to know that the emonpi board works in it’s own right before plugging it on to a Pi.

Please post some serial output from your emonpi board.

Are you thinking the RPi tx line is damaged?

You should not need to make any changes to the emonSD image for it to work with an emonpi board, however, if your emonpi IS outputing valid serial data and you still cannot get it working, please use the tried and tested image NOT the beta June2018 image as that too adds too many possibilities for locating issues as it is not tried and tested. In fact since G&T are currently working on a new image it’s quite possible the June2018 image will never get finished/released.

Once you know your board is good and it is working with the known good (older) emonSD image, you will then be in a better position to try other images etc.

@pb66
Paul,
after much investigation, I have determined the following:
The EmonPi board is outputting serial data at 38400 bps on power-up as follows:

EmonPi V2.90
OpenEnergyMonitor.org
Startup…

The RPi V3 is responding with what I believe is serial data (repeating several times), but I can’t find the correct baudrate that it is transmitting at the best result I get is at 38400 bd and the characters are:
v4b210g0q5ilp (repeated several times)
This occurs several times and I have tried every baud rate on Putty that I could find and even tried odd and even parity (which I didn’t expect to make any difference, but tried in desperation any way).
I have made the appropriate changes in the config.txt as follows:
dtoverlay=pi3-disable-bt (comment says it is not required in EmonSD Nov 2017 which I had reloaded at your suggestion)

dtoverlay=pi3-miniuart-bt

I have also set the core clock in the RPi3 to 250MHz (core_freq=250) as also suggested with no effect.

BTW, I have also tried minicom with the following command with the Rx and RX looped and nothing printed on the screen : sudo minicom -D /dev/ttyAMA0 -b 38400.

So, along my travels, I have certainly learnt a lot about Arduino and RPi which has been good, but extremely frustrating at times, and I think I have proven the issue down to a serial data issue with the RPI, but I have run out of ideas on where to go from here. Anyone have any ideas on what the issue is with the RPi and how I can prove the issue and fix it?

The obvious solution is to try another RPI, but I don’t have one (apart from a Pi Zero W that I purchased, but never used in anger)

In summary (for readers of this lengthy series of posts), the issue is that I only get as far as the LCD displaying Connecting… Please Wait.

I don’t want to abandon this project as I have put considerable effort so far, as Paul has provided considerable assistance in helping me identify the problem, but I admit I am close to giving up.

Regards,
Dave

Is that all it outputs?

How are you testing this? Are you checking the emonpi away from the RPi using the Arduino IDE and a serial programmer on the 6 pin header as I suggested?

Assuming you are testing as above and that the serial output does just stop after the “Startup…” line, then it looks like there might be a problem with the rfm side of your board. IIRC you have fitted an RFM69CW, is it definitely a “69CW”? Is it orientated the right way? Are there any bad connections or solder bridges etc on close inspection?

Looking at these few lines in the emonpi firmware

they call 4 functions (“emonPi_startup()”,“RF_Setup()”,“check_for_DS18B20()” and “i2c_lcd_detect()”), the first function is confirmed as working by the serial output you have provided, the middle 2 functions do not produce serial prints and the last one should print a message on success or fail

So it suggests the emonpi firmware is executing the first of those 4 functions ok but not reaching the 4th one. From experience I would look at the RFM stuff before even considering the temp sensor code, as a quick test you maybe able to just change this one line in the emonpi firmware and recompile/reload if you have the Arduino IDE and serial USB programmer connected.

by setting that to 0 it should bypass any RFM checks and go straight onto the temp sensor checks, although the confirmation of successfully detecting temp sensors doesn’t get printed to serial until the “serial_print_startup()” function is called in L208 (see here). So the line you are looking to see is either “LCD not found” or begins with “LCD found”.

Ultimately you want all the setup info to finish printing and then a new line every 5s starting with “OK” and containing all numbers, the first number on each line will be 5 and the last will be “(0)”.

In addition to the above, for your information.

The baud should indeed be 38400, those chars represent the following

v is requesting the version and status info
4b is setting the freq band to 433MHz
210g is setting the group to 210
0q is setting quiet mode 0 (Boolean false = off)
5i is setting the node/base id to 5
1p is the voltage setting for when there is no AC adapter

These are not really responses to the emonpi board, they are commands sent by emonhub to configure the emonpi when it (emonhub) start up, they are sent multiple times due to an error built in to the emonpi variant of emonhub.

I’m confused as to how you are using putty and emonhub together, the serial connection is monogamous to a single device/software at each end (ie emonhub on the RPi end and the FW on the emonpi board end) any attempt to use minicom or any other serial software on the same serial port whilst emonhub is running will skew the results as it cannot “snoop” it will break in to and interfere with the serial transmissions.

You should not need to change anything on the emonSD card to get this working, these changes must be undone or preferably, reflash the sdcard again so we know you definitely have a “stock” sdcard.

This will not work properly with emonhub running for the reasons above, and it will only echo back what you write (using minicom) however since you have been altering the overlays, I do not know if ttyAMA0 is the GPIO or the Bluetooth serial port, you can use this check to test the physical serial port on the RPi if you stop emonhub and then start minicom, then anything you type should be echoed back with the rx and tx pins shorted. But this test is not needed until you know the emonpi board is outputting as discussed in the last post.

Lets save the Zero for a last resort test once you have the emonpi board working and IF we still can’t get the RPi3 going. You will need to set up wifi on the RPi3 before moving the SDcard to the Zero as there is no way (on the emonSD image) of setting up the WiFi (via ethernet!) on the Zero directly.

@Pau,
Paul,
Thanks for all the perseverance you are giving me to help resolving this issue.
I will respond to your questions tomorrow in detail, but in regards to the RFM module, I have removed it as I couldn’t see the board designation as it was on the soldered side, and it is an RFM69 C - 433 mHz (no mention of CW although, that is what I ordered).
I will try changing the Boolean RF_Status tomorrow and recompiling, so will let you know the result.
BTW, I didn’t think the RFM module was essential if it wasn’t going to be used, and I was expecting to use wifi for data transfer.

Regards,
Dave

Cool, just so you are aware going forward. the emonpi update routine will try to overwrite your firmware every time it is run, so having the rfm (and therefore a stock FW sketch) may have it’s benifits, but you can get around this by only ever using the “update emonbase” button (rather than the “update emonpi” button) so it is not essential for you to have the rfm if you are not going to use it (or it’s problematic).

Additionally if you start with a fresh emonSD image, you must remove the emonpi board until the “firstboot” update has completed (as that is an “update emonpi” run) or just accept that the FW will get overwritten and reinstall your custom FW again afterwards.

First things first though, lets get it running then you can decide on the rfm etc. Offhand I have no idea if a “C” is different to a “CW” I’m afraid, but with it removed you can be sure it’s not causing any issues and there is no concern about hardware or library incompatibilities.

@Paul

Paul,
The saga continues…
I understand your comments about the emonpi update routine overwriting, and I appreciate that this is just an interim solution to get over the current problem around the RFM issue.

I am having problems compiling the sketch in the IDE; I have set the RF_Status to 0, in the sketch, and when I try compiling the sketch it can’t find a number of xx.h inclusions. I have downloaded them from the relevant Github sources that have been indicated in the comments but I don’t know which folder to store them in. I have tried storing them in various folders but I keep getting an error message stating eg " avr/Jeelib.h: no such file or directory"
Obviously I am storing them in the wrong folder, but I don’t know where to store them.
The files in question are:
Jeelib.h
onewire.h
pgmspace.h
emonlib.h

plus some others.

Can you tell me where I store the sketch inclusions please?

regards,
Dave

Look in Learn at Electricity Monitoring → Using the Arduino IDE. In there, there’s information about the library structure and your setup generally.

Perfect! Thanks Robert - I didn’t know that library explanation existed…

Dave

@Robert (and Paul)
think I am getting close. I have set up the libraries as per your guide and overcome several other issues along the way. I now have some issues relating to Boolean declarations that I can’t find the answers on how to fix them.
There is also a serial_print_startup issue that may be caused by a missing function that I couldn’t find on Gihub (not sure about this though)

Any advice on how to resolve these issues would be appreciated.
See below for error messages:

  Arduino: 1.8.6 (Windows 7), Board: "Arduino/Genuino Uno"

src:78:35: error: 'TRUE' was not declared in this scope

 boolean debug =                   TRUE;

                                   ^

src:93:35: error: 'FALSE' was not declared in this scope

 boolean USA=                      FALSE;

                                   ^

C:\Users\Fred\Desktop\EmonPi\Software\src\src.ino: In function 'void setup()':

src:186:12: error: 'TRUE' was not declared in this scope

   if (USA==TRUE)

            ^

C:\Users\Fred\Desktop\EmonPi\Software\src\src.ino: In function 'void loop()':

src:235:12: error: 'TRUE' was not declared in this scope

   if (USA==TRUE)

            ^

C:\Users\Fred\Desktop\EmonPi\Software\src\lcd_serial.ino: In function 'void serial_print_startup(int)':

lcd_serial:104:25: error: 'showString' was not declared in this scope

     showString(helpText1);

                         ^

exit status 1
'TRUE' was not declared in this scope

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

Regards,
Dave

Strictly speaking the “boolean” datatype is not (no longer?) supported by Arduino (or in C++), it should be a “bool” datatype and regardless of which is used, it should be lowercase true and false.

I think the Arduino IDE has toughened up on some loose ends recently, see the “Arduino IDE problem - showString( )” thread for info on the showstring() error.

I had hoped it was a simple case of changing that one variable when I suggested this route (silly me!), I’m sorry it’s not worked out that way but the emonPi FW is not recompiled very often and when it is (unfortunately) it’s usually only ever done on a Linux PC using PlatformIO and a set of Libraries that are stuck in time rather than the latest versions, it’s only when users such as yourself run into difficulties we become aware of any issues.

If you are up for trying to iron out these wrinkles we can try and help you through them.

First you could replace all “boolean” declarations with “bool” whilst also editing the declared values to lowercase as well. [edit - see the bolded last line of this post first]

Where the bools are used can be edited from if (USA==TRUE) to just if (USA) so that it is a logic test on the value itself and not a test to see if the value equals something eg TRUE.

some further reading on the bool vs boolean thing

The firming up of the position on using booleans has happened since the emonpi FW was written, it could be argued that it’s an Arduino IDE issue/bug but the “bool” datatype is the way forward even if backward compatibility is supported.

Looking at the changelog there was a change back in version “ARDUINO 1.6.0rc2 - 2015.01.20” of the Arduino core that states “Arduino “boolean” type is now mapped to “bool” instead of “uint8_t””, perhaps the ability to use uppercase “TRUE” and “FALSE” was lost in that move, you could try changing the case of all the boolean values to lower as a first step rather than changing to bools, it might be just the uppercase it objects to (worth a try?).

Yes, I’ve never understood why it has been done that way. Neither have I understood why Arduino C is different in these respects to K&R C.

A quick fix to check the ‘case’ conjecture:
#define TRUE true
etc.

3 Likes

@pb66 that did the trick - I changed all appearances of Boolean to bool, and made all uppercase TRUE and FALSE to lowercase and the IDE compiler does not throw up errors now.
The (hopefully) last error I have now is the one I referred to in my last post:

‘void serial_print_startup(int)’:

lcd_serial:104:25: error: ‘showString’ was not declared in this scope

 showString(helpText1);

Looks like there is something missing in the lcd_serial .ino file, but not sure what it should be.
Can you advise please?

regards,
Dave

Could be a problem encountered in another recent thread: the
new Arduino IDE requires a declaration of showString( ) in the main sketch.

[Note: the formatting has disappeared, please look up the original post as quoted correctly below (it’s Discourse, not you, Gwil) - RW]

@pb66, disregard the above as Robert has discovered an issue with the IDE V 1.8.6 is what I am using) and I fixed this issue as per Robert Wall’s post under the heading ""Arduino IDE problem - showString( )
see below:

That has solved all compilation problems but I was unable to upload the sketch using the IDE as it failed each time, BUT I UPLOADED THE SKETCH USING AVRDUDE AND IT NOW WORKS!!!
Just wondering why I couldn’t upload using the IDE - any ideas?
I am using the following IDE settings:
Board:Arduino/Genuine Uno
Port: Com 3
Programmer: USBasp

To sum it up, I can’t thank you enough for all the help you have given me along this very frustrating but ultimately satisfying path. I now have an understanding of both Arduino and RPI that I didn’t have before, so I have learnt a lot which is a good result in the end.
There is still some investigations required by me as to why the RFM chip caused the problem, but I think it highlighted a deficiciency in the firmware that could have had an error message as part of the setup that indicated an issue with RFM communication. If there had have been an error message pointing towards an issue with the RFM chip, it would have prevented a lot of angst early on. Also, the update to the IDE causing issues with both the Boolean function and the TRUE/FALSE statements needs to be reflected in the source files down the track.
I am sure that this won’t be the last question I have for this forum, so I hope everyone is as patient with me as has been the case with this looooong post.

Regards,
Dave

Glad you got there in the end. I had included a to the showstring discussion but it was possibly lost in the bool discussion (thanks for highlighting again @Gwil).

What programmer are you using with the IDE? If you are using a parallel programmer via the 2x3 pin ICP header you need a difference firmware with the bootloader included, what avrdude command was successful?

You’re absolutely right, as frustrating as it is at the time, you can learn so much more over coming obstacles like these.

For the record, I’ve pointed out in a PM to Gwil that many sketches will be affected by the changes made to the Arduino IDE, hopefully these will be made compliant with the new standard soon.