EmonCMS, EmonHub and Bluetooth

Hi
OK - got very confused with what can, and can’t, be done with Bluetooth in OEM

I have (amongst others) a RPi 3b with Emon board in the garage, connected to 2CTs, and hardwire pulse counter, running the latest SD card. Nearby, there is a nearby SunnyBoy inverter, which I would like to interface by buetooth to my OEM setup. The garage EmonPi is connected by wifi to an EmonBase RPi 3b, also running the latest SD card, which gives me access to a local EmonCMS on that Pi…

Having read various posts about ports shared between the BlueTooth and serial, I am confused as to whether I can expect to set up Bluetooth on the EmonBase, the EmonPi or neither (given my use of wifi). Both devices are within reach of SunnyBoy, though the EmonPi is much nearer. I’m not using RFM at all

I could connect directly to the generation meter next to the SunnyBoy, but that would give me KwH pulses rather than Kw flow, and in any case as there is an interfacer already available it seems a shame not to try it out!

I’ve tried several of the ‘turn the bluetooth on’ solutions on the web, and usually end up breaking the Emon setup! So I’m not sure if it’s incompetence, or if it’s not supposed to be possible…

Many thanks
Damien

Is this the EmonPi Shield? https://shop.openenergymonitor.com/emonpi-shield-kit-no-enclosure/

Do you have anything else reporting (an EmonTH for instance).

If not, you do not really need the EmonBase, it could all be done on the one Pi.

BT is disabled because of the UART port sharing on the EmonPi. This is something I profess to not fully understand - @pb66 is better placed to do that.

Have you looked at the other threads regarding this inverter?

Searching here gives 2 results that may help

The simplest way may be to use a USB Bluetooth dongle.

Hi Brian
Yes, it’s a shield. Onto the EmonBase I also have a Tx3.4, one of the Heatpump boards (when built!), and my Tesla batteries report to it as well.

Yes - and got more confused. I don’t understand the role of the serial port in the emonPi, nor if I’m actually using it in the base either. I think I do use it in my emon ESP8266 Tx, but that’s not the unit I’m trying to get bluetooth onto.

Yes - I gathered that, but it would be sooo frustrating given the Pi has bluetooth on board. Hence my question as to whether I am actually trying to make an impossible combo happen!

All the gear and no idea :joy: is my problem!

The emonPi’s analogue board/shield uses the serial connection to get its data into the Pi. So if you’re using the Pi+Shield for analogue measurements or as a radio receiver, then it’s using the serial port on pins 8 & 10.
And the “or as a radio receiver” part applies to the emonBase Pi too - if it had the RFM69Pi piggy-back radio module, that too is using the same pins to send its part of the same data.

Likewise, but using the same pins as your programmer, the emonTx talks serially to the ESP8266.

Thanks @Robert and @borpin.

So, in plain English - as the EmonPi is sensing from CTs (using the ADCs and the serial port), I can’t/shouldn’t enable the bluetooth, whereas on the EmonBase, where all my comms are Wifi (and not RFM), I should be able to disable the RFM69B functions (if enabled at install in the firmware) and enable the bluetooth…?

D

I think that’s correct, but I was careful not to say so because I’m not a Pi expert, Brian is.

If you’re not receiving anything by radio (i.e. from an emonTx without the ESP or an emonTH) on the emonBase - you can remove the RFM69Pi board and then it’s strictly speaking not an emonBase.

Well you can but you have to swap the UARTs around. Did you read those threads?

The UART is in use because that is how data is passed from either (EmonPi Shield or EmonBase) plug in boards.

If you have an RFM card on that take it off then the UART is not being used for data transfer.

I’m still not sure why you have 2 Pis running Emoncms.

Yes I did a couple of days ago. Fiddling with the emonBase sounds safer!

I don’t, Brian. The Pi with shield is used for its inputs (2xCT + pulse + T1), which then get posted via the emonHub MQTT settings (on the Pi + shield) to the emonBase on which I am running EmonCMS. (I could have used an EmonTx instead of a Pi with shield, but I was attracted by the inbuilt wifi. My other EmonTxs use ESP8266s)

Thanks both for your help. Massively appreciated as always

Just use the emoncms on the Pi with the Shield is my suggestion.

Add a Bluetooth dongle to that for the Inverter and free up a Pi (and reduce unecessary power consumption) :slight_smile:.

Ah, I did ask earlier if you had other Emon kit. My suggestion still stands.

Can someone run this command on a emonSD that has not had it’s serial ports altered? I don’t have anything running emonSD currently.

ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null

the result from a newly downloaded raspiOS installed to a Pi4 is

lrwxrwxrwx 1 root root          5 Jan  4 21:17  /dev/serial0 -> ttyS0
lrwxrwxrwx 1 root root          7 Jan  4 21:17  /dev/serial1 -> ttyAMA0
crw-rw---- 1 root dialout 204, 64 Jan  4 21:17  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Jan  4 21:17  /dev/ttyS0

As you can see there are 4 lines, the first 2 lines are hyperlinks created by the OS at boot up and the second 2 lines are the 2 serial ports accessible on all Bluetooth&Wifi enabled Pi’s (3, 4 and ZeroW).

Only one port “/dev/ttyAMA0” is available on other Pi’s (A, B, 2 and Zero) so on those models there would only be 2 lines returned from the command above, one hyperlink and one actual serial port.

“/dev/ttyAMA0” is the MCU’s higher quality serial port made available (accessible) on all Pi’s

“/dev/tty S0” is the MCU’s lower quality serial port made available (accessible) on Pi’s with onboard bluetooth due to that onboard bluetooth using the higher quality “/dev/ttyAMA0” serial port on those devices by default.

“/dev/serial0” is always the physical GPIO port on all Pi’s and

“/dev/serial1” is only available on the BT Pi’s as it is always the onboard Bluetooth transceiver.

By default non-BT enabled Pi’s have the high quality “/dev/ttyAMA0” serial port mapped to the physical GPIO UART “/dev/serial0” and that is it, no options or variations etc.

Whereas by default, BT enabled Pi’s have that higher quality serial port “/dev/ttyAMA0” mapped to the BT transceiver “/dev/serial1” to provide good BT performance whilst the lower quality “/dev/ttyS0” port mapped to the physical GPIO UART “/dev/serial0”.

The BT enabled Pi’s with 2 accessible serial ports can have those ports mapped to various configurations at boot time by using overlays in the config.txt, those overlays are used at boot to map the serial ports, which creates those hyperlinks mentioned above and shown in the command output.

lrwxrwxrwx 1 root root          5 Jan  4 21:17  /dev/serial0 -> ttyS0

shows us that “ttyS0” is linked to the GPIO “/dev/serial0” and

lrwxrwxrwx 1 root root          7 Jan  4 21:17  /dev/serial1 -> ttyAMA0

shows us that “ttyAMA0” is linked to the BT “/dev/serial1”.

This leads to a common misconception that “ttyAMA0” and “/dev/serial1” are the same thing, that they address the same port, they do in this configuration, because they are linked together, but under other configurations they will not address the same port. The “ttyAMA0” addresses the higher quality serial port (in the MCU) and “/dev/serial1” addresses the BT, the same thing when they are connected (mapped) together.

I do not know what the current status is on the emonSD but it used to map the higher quality port to the GPIO for the emonPi/RFM2Pi boards as BT isn’t often used. Rather than swap the ports, they just dropped BT and mapped the GPIO to the better port, so for BT to work you would need to first map that connection by using a different overlay.

If the original mapping is/was left alone configured by default, both the GPIO UART and BT would function just fine **, I had thought (hoped?) this had been sorted in the latest images. Alternatively I think someone documented how to get BT working on a emonSD and put it in the emonhub repo (IIRC) but this was some time ago and may not still be relevant to the current images.

(** when I say “BT would function just fine” I mean with some small edits. We should not be addressing the GPIO UART via “ttyAMA0” as that is not always correct, we should be using “/dev/serial0” as that is always correct, hence why it is now used in cmdline.txt and not “ttyAMA0” as was true in Pi’s pre-BT days)

With the output from the above command we can see how the emonSD is currently mapped and determine what is needed, and @dlongson, if you can also post the output from your setup we can see if you’ve upset anything along the way and get your BT running (fingers crossed :grin:)

Sorry for the long post, but having explained this before here on the forum, apparently not very well, I thought I would try another way. Sorry if it’s excessively dumbed down or assumes too much.

[edit] I can’t find the info on using BT in the emonhub repo, must be elsewhere, I will post a link if I find it.

Last login: Tue Dec 29 15:40:51 2020 from 192.168.1.80
pi@emonpi:~ $ ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null
lrwxrwxrwx 1 root root          7 Dec 31 20:26  /dev/serial0 -> ttyAMA0
lrwxrwxrwx 1 root root          5 Dec 31 20:26  /dev/serial1 -> ttyS0
crw-rw---- 1 root dialout 204, 64 Dec 31 20:27  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Dec 31 20:26  /dev/ttyS0

It self-updated a week ago when I plugged it in to test something else for the forum. The versions are:

Version: low-write 10.2.6 (24 Jul 20)

Modules:
Administration | App v2.2.3 | Backup v2.2.6 | EmonHub Config v2.0.5 | Dashboard v2.0.9 | DemandShaper v2.1.2 | Device v2.0.7 | EventProcesses | Feed | Graph v2.0.9 | Input | Postprocess v2.1.4 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.3

Git URL:
GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
Branch: * stable
Describe: 10.2.5-3-gf781bed0

It’s on a Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)

1 Like

The GPIO UART is mapped to the higher quality “/dev/ttyAMA0” serial port

and BT is mapped to the lower quality “/dev/ttyS0” serial port

So the BT serial port should be accessible on both “/dev/ttyS0” and “/dev/serial1” whilst configured this way and the emonpi/rfm2pi are addressed at “/dev/ttyAMA0” and “/dev/serial0” whilst configured this way.

Best to always use “/dev/serial1” for BT and “/dev/serial0” for GPIO IMO but the current OEM practice is to always use “/dev/ttyAMA0” for the GPIO rather than edit the emonhub.conf.

It would appear the current config uses the “swap BT and GPIO” overlay now (I’ll look up the proper name in a min).

BT should work, but it maybe let down by the lower quality (slower) port, either port would suffice for the relatively low requirements of the emonpi/rfm2pi boards so I’m not sure why they get swapped. Personally I assume that the RPF know a thing or 2 about Pi’s and swap the high quality port to the BT for a reason?

1 Like

Is there a line

dtoverlay=pi3-miniuart-bt

or

dtoverlay=miniuart-bt

in /boot/config.txt?

I have just been looking through the emonscripts repo and can’t find where the ports are remapped. In fact I can’t find any active ref to the config.txt (there’s a bit commented out about arm_freq), I’ve searched the repo for “miniuart-bt” and “dtoverlay” with zero results. I have no idea how the emonSD image is not as per the standard raspIO image in this respect, maybe it’s an undocumanted edit during production?

Can you post your /boot/config.txt content please.

Top-tip!

If (as I do) you run this command line just once

printf 'alias lsser="ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null"\n' >> ~/.bash_aliases && source ~/.bashrc

it will simplify keeping an eye on the serial ports as you can then use just lsser (like ls or lsusb but for serial info)

pi@Pi4:~ $ lsser
lrwxrwxrwx 1 root root          5 Jan  4 21:17  /dev/serial0 -> ttyS0
lrwxrwxrwx 1 root root          7 Jan  4 21:17  /dev/serial1 -> ttyAMA0
crw-rw---- 1 root dialout 204, 64 Jan  4 21:17  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Jan  4 21:17  /dev/ttyS0

very handy when swapping serial ports around or when plugging in arduino’s or programmers via USB etc

1 Like

:angry: I’d powered it down, now restarting it, it’s decided to update itself again. It’s a good job it’s past midnight and I’ve got free Internet, because it’s a bit naughty downloading and updating without asking. And overwriting my CM front end again.
(How do you get a WiFi signal strength of 140%?)

Here’s the file - bt looks clobbered right at the end - noting I’m not running a Pi 3 or 4:

pi@emonpi:~ $ cat /boot/config.txt
# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
#disable_overscan=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
#overscan_left=16
#overscan_right=16
#overscan_top=16
#overscan_bottom=16

# uncomment to force a console size. By default it will be display's size minus
# overscan.
#framebuffer_width=1280
#framebuffer_height=720

# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

#uncomment to overclock the arm. 700 MHz is the default.
#arm_freq=800

# Uncomment some or all of these to enable the optional hardware interfaces
dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Uncomment this to enable infrared communication.
#dtoverlay=gpio-ir,gpio_pin=17
#dtoverlay=gpio-ir-tx,gpio_pin=18

# Additional overlays and parameters are documented /boot/overlays/README

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
dtoverlay=vc4-fkms-v3d
max_framebuffers=2

[all]
#dtoverlay=vc4-fkms-v3d
dtoverlay=pi3-disable-bt

And given that it’s just done an update, here’s:

pi@emonpi:~ $ ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null
lrwxrwxrwx 1 root root          7 Jan  4 23:54  /dev/serial0 -> ttyAMA0
lrwxrwxrwx 1 root root          5 Jan  4 23:54  /dev/serial1 -> ttyS0
crw-rw---- 1 root dialout 204, 64 Jan  5 00:34  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Jan  4 23:54  /dev/ttyS0

[unchanged]

My Top Tip is install a pair of key files, then the machine you’re connecting to the Pi gets access, and if you didn’t set a password on the key file, you don’t need the SSH password again (until you access it from a different machine, or delete the private key).

Yeah I was just about to hit the hay too, sorry.

So what are you running? It looks like it has 2 serial ports.

Yep, that isn’t “stock” the at last line has been added without a comment or even a spacing line. I can’t see that in the emonscripts anywhere.

Oops - yes I am, I was thinking I’d got the Pi 2B, and I haven’t. :sleepy:

Not my doing.

1 Like

What is updating - emoncms or Rasbian/Raspberry OS?

I am still on the previous SDImage - emonSD-17Oct19

pi@emonpi:/opt/openenergymonitor/emonhub/service $ ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null
lrwxrwxrwx 1 root root          7 Dec 12 11:17  /dev/serial0 -> ttyAMA0
lrwxrwxrwx 1 root root          5 Dec 12 11:17  /dev/serial1 -> ttyS0
crw-rw---- 1 root dialout 204, 64 Dec 12 12:14  /dev/ttyAMA0
crw-rw---- 1 root dialout   4, 64 Dec 12 11:17  /dev/ttyS0

Changes to the config.txt are now done in the emonhub install (for obvious reasons)

This means all the bits about freq changes etc have been dropped as it just modifies the installed file.

Note some of the comments - what is done might not be correct or for the correct reasons based on other posts.

Hi @pb66.
One of my My RPi 3B (set up as an emonBase 24July20 version) has come up with:

lrwxrwxrwx 1 root root 7 Dec 21 23:17 /dev/serial1 -> ttyAMA0
crw-rw---- 1 root dialout 204, 64 Dec 22 18:06 /dev/ttyAMA0

and the other RPi3B (set up as an RPi with shield, 24July20 version) comes up with

pi@emonpi:~ $ ls -la /dev/{tty{ACM,AMA,S,USB},serial}* 2>/dev/null
lrwxrwxrwx 1 root root 7 Jan 5 10:17 /dev/serial0 -> ttyAMA0
lrwxrwxrwx 1 root root 5 Jan 5 10:17 /dev/serial1 -> ttyS0
crw-rw---- 1 root dialout 204, 64 Jan 5 11:09 /dev/ttyAMA0
crw-rw---- 1 root dialout 4, 64 Jan 5 10:17 /dev/ttyS0

I have followed @borpin suggestion of just adding a bluetooth dongle to my Pi with Shield. That has worked in so far as I was able to follow all the instructions in the section on emonHub interfacers for the SMA (EmonHub Interfacers - Guide | OpenEnergyMonitor) and it reported correctly to the local copy of emonCMS (result!). However, any attempt to restart the hub after adjusting settings crashed the pi, and indeed it crashed anyway about 30 mins after I left it alone last night. So I’m progressing. Slowly

[mod - edited code quotes for readability, please use 3 backticks on the lines before and after code blocks and avoid trying to use bold and/or itallics within code blocks]

[mod - edit 2, still doesn’t look right as the tabs have been lost along the way. It is so much better for all if you just copy and paste code/logs etc and add the triple ticks for and aft, thank you]

Ok, so shall we leave pursuing the onboard BT? I don’t want to confuse things if your happy with that approach. I would have been nice to sort this once and for all but I have no idea if there is any appetite for changes in the OEM approach.

Regards the crashing you would probably need to look at the emonhub logs for info.