New user with questions about emonpi

[Thomas_Adeyinka (bigbrovar)]

Hi guys. Thanks for a wonderful project. I have been living offgrid now going 2 years due to the unstable grid power in my country. I have a 1.8kw offgrid solar setup that pretty much serves my needs. I a nag for logging and although I use to export csv of my PV Generation to pvoutput I wanted a better real-time logging solutions. After some searches I decided to settle for emonpi. I would have loved to get the hardware but for budget considerations.

I tried to Bury myself in as many docs, guides and forum post about emonpi emoncms as I can but I have some questions which I have been unable to find answer for.

Here are my solar equipments:

2 Victron bluesolar mppt charger
1 victron battery monitor bmv 700
1 Axpert 24v 3kva Inverter

My objectives:

Monitor PV generation real time from the 2 victron mppt and keep an historical log

Monitor battery state of charge and keep an historical log of that too

Monitor consumption from the inverter using the serial to USB port.

I also want a system that logs locally but can also log online anytime I am connected to the Internet.

I got a RaspberryPi with a 4.3 480x272 lcd and the idea is to have the local emoncms always on display to view real-time situation report from all the devices connected to the Pi.

I have a RaspberryPi with the above mentioned lcd. I also intend to use the raspberry with a connected hd so as to avoid sd card corruption.

Are my objectives realistic? Would it be possible to have a Pi do monitoring, and log locally and retroactively post online once connected to the Internet. And lastly would it be possible to use emonpi with a RaspberryPi lcd where dashboard of the working system can be glanced at anytime am in the solar inverter room.

Apologies for my long questions and language.

[Thomas_Adeyinka (bigbrovar)]

Please let me know if my question is ambiguous or was posted in the wrong section.

[Thomas_Adeyinka (bigbrovar)]

So I found this post EmonPI connection to television as monitor - #5 by glyn.hudson which seem to address the part about attaching an lcd to the RaspberryPi with emonpi on it.

[pb66 (Paul)]

@Thomas_Adeyinka

Would it be possible to have a Pi do monitoring, and log locally and retroactively post online once connected to the Internet.

The emonPi image has emonhub installed (the “emonpi” variant) and that can log locally and to a remote server simultaneously, the data for the remote server(s) is buffered before being passed as a http request, the idea is if the network is down it will “catch up” when the network comes up. This data is held in RAM though, so a local power outage could lose the data before it is posted to the remote and you are limited by the amount of RAM you have available for buffering.

I use the original emonhub and produce my own SDcard image that doesn’t run so many services. But I have ran tests with no network connection for a month and suffered no data loss or running issues from lack of RAM on a Pi 3 with 3 emonTx’s posting large payloads at 5s intervals. One day I would like to add a way of persisting the buffered data “just in case” but it’s not high on the agenda right now.

[Thomas_Adeyinka (bigbrovar)]

Thank you so much for your response. It really did help in answering the questions I have. How can I have the OG emonhub? Can it be installed on a Pi? I intend to run my Pi on a hard drive to bypass the space limitation and SD card life span issue.

[pb66 (Paul)]

@Thomas_Adeyinka

How can I have the OG emonhub?

The original emonhub is the one linked from the emoncms install guide, although you should be aware that the original version is quite basic in functionality compared to the emonpi variant which was based on the unreleased “experimental” branch of the original emonhub. Most comments on this forum and the OEM guides relate to the emonPi variant NOT the original version. I and a few others use the original version over the emonpi variant as it is more stable and reliable, there have been no changes (or problems with it) for around 4 years now. There have been talks of re-uniting the 2 variants but that isn’t a small task.

@Thomas_Adeyinka

Can it be installed on a Pi?

Yup!

@Thomas_Adeyinka

I intend to run my Pi on a hard drive to bypass the space limitation and SD card life span issue.

The HDD will not help with any potential RAM “space limitation” unless there is some swap space in play, but the hdd will be better for emoncms and also for log files, the read-only SD image loses the log files on reboots making debugging a problem, with plenty of disc space, your emonhub.log files will be very valuable when debugging.

The RAM on a Pi 3 is 1Gb which is plenty. The original emonhub has a setting for capping the max size of the buffered data (in frame counts not bytes) where as the emonpi version doesn’t, so it is possible if the network is down indefinitely, it could build the buffer up until it chokes the Pi. The original emonhub default setting is a very reserved 1000 frames (1 device @ 10s data = ~2.75hrs), I currently run between 70000 and 200000 on some of my remote devices as schools can close for weeks without there being someone available on-site to check why the network is down. You will need to find your own “sweet-spot” setting that provides enough cover without the risk of choking other services.

[edit] can you clarify if you are using the emonSD image on a hdd or if you intend to create your own image from scratch. The RAM available is largely dependent on how many services you have running and the emonPi image does have loads going on. Plus replacing the emonpi variant of emonhub with the original version on the emonSD image is rather more involved than just using the standard version on a new build.

[Thomas_Adeyinka (bigbrovar)]

@pb66

although you should be aware that the original version is quite basic in functionality compared to the emonpi variant which was based on the unreleased “experimental”

Thanks. Its clearer to me know I use to hear a lot about OG emonhub now I understand for the clearer. Is there a feature matrix between the OG and the latest version? Or what is the major difference feature wise.
My needs are pretty basic. Monitor output from 2 victron mppt charge controller, a victron battery monitor and Axpert inverter. I will be attaching a 3G USB modem to it to ensure Internet availability which might be patchy sometimes.

@pb66

The RAM on a Pi 3 is 1Gb which is plenty. The original emonhub has a setting for capping the max size of the buffered data (in frame counts not bytes) where as the emonpi version doesn’t, so it is possible if the network is down indefinitely

I intend to run a GUi on the Pi an idea which I might have to drop based on your reveal. I wanted to have an lcd 4.3 attached to the Pi permanently to provide some form of dashboard with pie chart of status reports.

@pb66

can you clarify if you are using the emonSD image on a hdd or if you intend to create your own image from scratch. The RAM available is largely dependent on how many services

I would be using the emonsd image for now to get a feel and then eventually I would like to spin my own custom image based on my use case.

Does the low wite mode of the emonpi really help with SD card life or just a marginal incremental.

[pb66 (Paul)]

@Thomas_Adeyinka

My needs are pretty basic. Monitor output from 2 victron mppt charge controller, a victron battery monitor and Axpert inverter. I will be attaching a 3G USB modem to it to ensure Internet availability which might be patchy sometimes.

We may have different idea’s on what “basic” is

@Thomas_Adeyinka

I intend to run a GUi on the Pi an idea which I might have to drop based on your reveal.

I’m not sure that’s the case, the data held in RAM by emonhub uses very little space, however the size of the device payloads, the number of devices you have, the frequency at which they post and of course how long you are without a network will all determine how much room you need whilst how many things you have going on on the Pi will determine how much RAM is available, as long as you are mindful of this you should be able to have what you want.

For example the emonPi image is running emonPiLCD, mqtt_input, nodered, openhab, lightwaverf and mosquitto whether you want them or not, you must have Apache2 to run emoncms, so by simply using http to post to emoncms you can shutdown all of those services (if you don’t need the emonPi display) which should give you plenty of scope to run a desktop, especially if you only need a browser console.

(sidenote - Are you aware that you cannot access the HDMI socket on an emonPi due to the case?)

Can you clarify what HW you are using? You have said “I decided to settle for emonpi.” and "I got a RaspberryPi with a 4.3 480x272 lcd " so I’m unsure if you have the emonPi or have another Pi + LCD or both, if both, are you planning on having just one or both set up?

@Thomas_Adeyinka

Does the low wite mode of the emonpi really help with SD card life or just a marginal incremental.

It works very well, I haven’t heard of any failed SD’s for a significant time now, but it has it’s limitations, personally I like emonhub on a SDcard with a read-only fs, but always install emoncms to a Pi with a hdd or to a cloud VM, but that’s just my preference, I guess many more users use the emonSD for emoncms than use hdd.

The main difference in emonhub versions is that the original isn’t geared up for mqtt, the way mqtt is set up in the emonPi variant is VERY emonpi orientated and not so flexible, so you either do MQTT the way it’s done on the emonPi emonSD image or you don’t do mqtt, there is no generic mqtt options in either emonhub yet. The emonPi variant also uses named inputs rather than just node ids, but you only need to name your inputs once, so that’s no real big issue. However there are some bespoke interfacers that will only work with the emonpi variant to connect to modbus etc, so depending on how you are planning to connect to your devices may impact your options.

It’s pretty difficult to adapt the emonSD to your own requirements, so you either need to just go with it and accept what to have, or avoid it and do your own thing. You can chop and change it about, but whether those changes would survive an “emonpiupdate” or even cause compatibility issues is something you will need to deal with as it happens.

[Thomas_Adeyinka (bigbrovar)]

@pb66

(sidenote - Are you aware that you cannot access the HDMI socket on an emonPi due to the case?)

Can you clarify what HW you are using? You have said “I decided to settle for emonpi.” and "I got a RaspberryPi with a 4.3 480x272 lcd " so I’m unsure if you have the emonPi or have another Pi + LCD or both, if both, are you planning on having just one or both set up?

Sorry I did not make myself more clearer. I meant I would settle for the emonPi image not the hardware. I settled for burning the image to an SD to get me started. Once I have gotten a feel of how it all works, Then I can see about building one from scratch.

@pb66

However there are some bespoke interfacers that will only work with the emonpi variant to connect to modbus etc, so depending on how you are planning to connect to your devices may impact your options.

It pretty much works with all the interface I require beside the inverter, all my solar components are Vedirect compatible i.e battery monitor, and 2 solar charge controllers. The inverter has some issue with the usb driver which is not compatible with Linux so I might have to find a way to get the current sensor, most like I might be using this addon https://www.amazon.co.uk/Aviosys-USB-e-Meter-8870-White/dp/B00N0SRRK0 It is a current clam sensor to usb and works well on Linux. I might have to find a way to interface it with the emonpi.

Currently I have the setup working the way I want without making much change to the way it was designed to work. In fact the only way I have differ from officially supported install is in the network stack. I do not have a Huawei 3G modem, I use some Generic 3G modem (which also supports sms and ussd) so I had to setup a PPP0 interface and get it to work with HOSTAP… so in essence the Rpi works in AP mode and I can connect to the wireless (and browse the web from there) and it can also send data using the 3G dongle. Works pretty well (after some tweaking of cause

The other modification is to have chromium in kiosk mode to display a dashboard designed for a 480 x 272 LCD screen. The whole system is powered by a DC DC buck converter which converts 9v - 35v to 5v 5A with 4 usb ports as output. That way the Rpi is powered directly by the battery bank and is not affected even when the inverter is turned off (like most mornings when I decide to give a battery a 2 hours rest - or when am carrying out routine maintenance)

Below are the pictures of what I have done so far

DC to DC converter
[

]

USB 3G Modem

LCD

Whole system for now

The LCD part should be attempted tonight, as you can see, I have just one device connected, but I am waiting for some JST 2 PH 4 pin connector I ordered so I can DIY the ve direct to usb cable.

I already have things up and running and I have installed emoncms on my vps server so the Pi would be reporting to that. Internet should be fairly stable at least for the most part of the day.

[pb66 (Paul)]

@Thomas_Adeyinka

It pretty much works with all the interface I require beside the inverter, all my solar components are Vedirect compatible i.e battery monitor, and 2 solar charge controllers.

They are only available in the emonPi variant of emonhub, so the decision is pretty much made for you. If you are using the emonSD image, as I said previously, you should probably stick with the way it is intended to be rather than making big changes.

You found the thread to add the desktop environment packages to the emonpi/sd image, there is also a post or 2 on the old forum somewhere about running a browser in full-screen kiosk mode automatically at boot up for emoncms that might help you once you have the browser and desktop installed.

[Thomas_Adeyinka (bigbrovar)]

Thanks for the responses, they were of great help in finishing up on the project. Major issues I had included having to write a udev rules to permanent attachment of the usb serial to a particular tty (through symlink of cause) on a whole the system works fine and I have pretty much accomplished the goals I set out. For now I will continue to run emonpi headless its looking less and less important to attach an LCD to the Pi.

My finished dashboard is below. Thanks to everyone who contributed to this great project.

http://energy.bobby.com.ng/dashboard/view?id=1