Community
OpenEnergyMonitor

OpenEnergyMonitor Community

Email Alert when Input Threshold Reached

Tags: #<Tag:0x00007f681d5453f8>

I have a local Emoncms Version 10.2.6

I have a need to send an email alert. Such as: If Line 1 current goes above 10 AMPS, then send only one email, let’s say every hour.

I have tried to Add “Send Email” Process to the Feed, with several configurations.

My Swift Mailer is setup properly, it is working, and tested.

I can’t seem to control the “Send Email” Process. When I do get it to send, it usually sends multiple emails.

My four options are:

  1. Keep working with the“Send Email” Process to see if it can work.
  2. Create a script that checks value, using curl, Create a cronjob to run the script and send email.Such as: Alerts sending issue to an email from local host emoncms
  3. Use Mqttwarn for MQTT Alerts by jpmens/mqttwarn Jan-Piet Mens https://github.com/jpmens/mqttwarn
  4. Install Node-Red. I have read from the Community “The suggestion is to not run Node-RED on your emonPi (recent thread about a dead emonPi)” Setting up MQTT for the first time

I would appreciate any feedback with the best option to maintain a trouble-free EmonCMS. I am looking for the best method that does not affect the EmonCMS, and let it do what it does best. But I have a strong need for some sort of Alert System, Email or Text. If there is any alternative suggestion, please let me know.

Thanks in Advance.

This screenshot does not send an email at all. I have tried several different configurations.

@dlmpryor

If you were to use Option 2 then I think the following approach would work:

Simplify your Input Processing just to – Log to Feed and Publish to MQTT

Your script could then:

  • Subscribe to the MQTT topic
  • Contain simple logic based on IF statements to decide whether to send an email or text based on the topic value and on the time elapsed since the previous alert was sent

Recently I did something similar and struggled with Python and MQTT so wrote a bash script instead.

I think this is a great link:

Here are the relevant lines:


echo -n "" > response.txt # NOTE - response.txt will be created in /home/pi

mosquitto_sub -h "127.0.0.1" -v -u 'emonpi' -P 'emonpimqtt2016' -t "emon/openevse/state" -C 1 > response.txt # NOTE the: -C 1

read -r line < response.txt

feed=${line:}

echo $feed # For debugging – can be commented out

Clearly your MQTT topic will be different from emon/openevse/state

Your subsequent logic tests and actions would be based on the value of feed

Such a script can be tested in an SSH terminal and once it runs one iteration successfully, the code lines can then be put in a loop that runs forever.

Would probably be best to run the script as a Service so it runs automatically after each reboot.

Hope this is helpful.

Thanks John. This really helped me get started. Saved me hours of work.
It seems to be the most non-intrusive method so far.
I really want to keep my local emonCMS clean, trouble-free, and let it do what it does best.

  1. Running Script gave Error" -bash: mosquitto_sub: command not found
  2. Installed: sudo apt-get install mosquitto-clients
  3. Resolved Error
  4. Changed MQTT path From:emon/openevse/state To:emon/emontx3/DI2
  5. response.txt = emon/emontx3/DI2 1
  6. DI2 is a digital input indicating if a leak, DI2=1 means a Leak, DI2=0 no leak.

So far so good!!!

Work to be done:

A. Next step is to run in a loop.(Maybe not needed if I use a hourly cronjob)
B. Then Send email if DI2=1 Leak is detected.
 Prefer to Use Swiftmailer since it is EmonCMS preferred mailer.
 Swiftmailer is already installed and tested on my local EmonCMS
 Not sure how to send email? cronjob, Scripting
 Then, I only want one email, like every hour. Not sure if this can all be done in the
 script, or use a cronjob to run every hour?

Any help with the run in loop scripting, or the sending email would be appreciated.

@dlmpryor
I was unaware that SwiftMailer was installed as part emon. I stumbled into using msmtp when first learning about cronjobs and emailing. msmtp is well documented.

At a quick glance, the SwiftMailer website is sparse and php is a foreign language to me but Google is your friend.

Re: hourly cronjob or running a continuous loop? A cronjob would give you the status each hour whereas a continuous loop would alert you as soon as an out of spec condition occurred.

For help with loops in bash, check out the link below and scroll down to # Loops:

I don’t want to misspeak about SwiftMailer, it is NOT included as standard.
EmonCMS just supports installation of swiftmailer, as of a Post on March 4, 2020

Appreciate your comments.
I prefer to send email as soon as out of spec.

Script Outline:

if [  Leak Detection == 0 ]; then
	echo 'No Leak Detected '
	Leak Detection > response.txt
else
	echo 'Leak Detected Send Email '
	Send Email
	#Yikes! This will bomb me with emails. Need Timer!
fi

This if, then, else script outline would only run once.
I would need a cronjob running every so often.

Now it’s time to roll up the sleeves.

Thanks Again!

I’ve edited your post for presentation. For information, when posting code or output, please put 3 ‘backticks’ (normally found at the top left of the keyboard) on a line of their own before the code, and 3 more backticks also on a line of their own after the code.

```
code
```

If it is something like php you can add a language identifier after the first 3 backticks: ```php or even ```text if you don’t want any language markup applied.

I have a similar need, but used a different approach - largely because of the way I collect and log data. So probably not as useful as @johnbanks but maybe applicable to a different need.

I have a battery storage system that is largely under proprietary control, but I can interrogate the data from the web and pull out necessary values. In particular I want to be alerted when the state of charge (SoC) drops below a set value such as 50%. While this can be checked manually using the proprietary web interface, I want to get an email/SMS alert.

The approach I took was to:

  1. pull the data from the inverter using its html web interface
  2. put this data into a json string and write it to disk every 10 secs (overwrite the 700 byte files every time)
  3. read this file (using python)
  4. process and check the data I need
  5. if necessary call an email script and send via gmail
  6. if critical, send an email that triggers a text SMS to mobile phone.

So there are 3 small python scripts that get used:

The first uses requests to interrogate the inverter and download a text string with the data (current watts, daily watt hrs, SoC etc). This is basically a json string of key:value pairs. This is written to disk as a text file and updated every 10 seconds. (it is also inserted into a database by another script, but that is incidental to this application).

The second script reads the json file and pulls out the SoC key:value pair putting it into a list. It does this every 30 mins. The list length is capped at 10 values, so provides a 5 hr record of the SoC. While I want an alert when the SoC drops below e.g. 50%, I don’t want to be alerted every time it rises and falls around that value, so the list only triggers an alert when it drops below 50% after 9 continuous decreases. When triggered the alert calls a third script that is basically the emailer, passing the list string with the last 10 SoC values.

The emailer script is a short smtp script (using smtplib) setup for gmail. It is called using os.system(‘python3 pymail3.py string’) in the second script. This is an easy way of passing the list values as a string argument that is then sent as the subject or body of the email. I actually have 2 of these email scripts - the second one is used to trigger a text message, so has to have a special receiver/sender combination for my cellphone account.

It turns out you can have more than one gmail account, so for security I have a second gmail account just for this app - as password is not currently encrypted.

This is an example of the body of email that gets sent. The values are %SoC:

Check timestamp = 29-12-2020 09:00:18 ['pymail3.py', '[63.24,', '61.99,', '60.9,', '59.73,', '58.57,', '55.81,', '55.38,', '58.28,', '59.32,', '60.61]']

Currently all running on an rPi, which is used for node-red/mqtt.
I’m happy to share the scripts if anyone is interested - just need to cleanup and blank account details etc.

1 Like

Thanks Anthony,

If you don’t mind sharing your work, it would be greatly appreciated.
I am still deciding what approach I am going to use.
I was going to use shell scripting, but I am not sure how resource intensive they are when used as infinite while loops?
I am thinking about python all the way. But I would have to learn that language.
Either way, I will most likely need to call a emailer script.
Pondering about over night, I came came up with this untested shell script outline:

Script Outline2:

while;
   do
   if [  Leak Detection == 1 ];
   then
	echo 'Leak Detected '
	Leak Detection > response.txt
	send email #call python email script
	sleep 300 #5 min, time control to avoid email flooding.
	break; # exit loop, but not script
   fi

   echo 'No Leak Detected '
   echo 'keep script running'

done 

Hi @dlmpryor - Here are the 3 scripts that I am using. First is to get the data from the device - in my case a selectronic inverter.
This inverter also checks for updates etc every midnight, and I found that it can cause a problem if I am trying to pull down data during this time, as it seems to randomly check sometime between midnight and 1:00 am. So I just put my loop to sleep for 6300 seconds from midnight (arbitary length - probably only needs 30 mins).

If it is not in this midnight period, then use python requests to pull down the data, but using a try loop to manage the occasional data corruption. If there is a problem (an exception) basically sleep for 10 s and try again. The downloaded data is in json format. I change the timestamp format and round down some of the 15 decimal places(!) to 2 and write to a file. This file is very small and is updated every 10 seconds.

import dateutil.parser
from datetime import datetime, time
import pytz # not sure if needed here
begin_time = time(0,1)
end_time = time(1,45)
while True:
    dt = datetime.now().time()
    if (begin_time<=dt<=end_time):
        print('gone to sleep at ',dt)
        sleep(6300)
        print('woken and reconnected')
    else:
        pass 
    r = requests.get('http://192.168.20.xxx/cgi-bin/solarmonweb/devices/xyzetc/point')
    # need to manage errors if http fails with poor json
    try:
        datajson = r.json() # get the data in json
        extract = datajson['items']
        extract['timestamp'] = (datetime.fromtimestamp(extract['timestamp']).strftime('%Y-%m-%d %H:%M:%S'))
        extractcolumns = list(extract.keys())
        extract['soc'] = round(extract['battery_soc'],2)
        extract['load'] = round(extract['load_w'],2)
        extract['pvwatts'] = round(extract['solarinverter_w'],2)
        print(json.dumps(extract))
        with open('selectronic.json','w') as outfile:
            json.dump(extract,outfile)
    except:
        sleep(10)
    sleep(10)

The second script reads this data file and pulls out the SoC value I want to monitor (data[‘soc’]) and appends it to an empty list (‘files’). This list will continue to get longer unless trimmed, so I ‘pop’ the first value when the list reaches 10 entries and test if the list values are strictly decreasing (if yes, result will be True). The list (‘files’) is converted to a string and appended to a string ‘call_email’ that is run with os.system(call_email). As currently setup, an email is sent every 30 mins regardless of the SoC, but this can be turned off. Further down the loop the last SoC value is tested, and if less than 50 (%) triggers another email that can also activate an SMS text message. As shown below, it is just the normal email, but can be changed to the SMS script.

The last 6 lines of code run the schedule - which is effectively just printing a line of dots while waiting for the 30 min schedule to trigger. This probably could more simply done using sleep, but I am trying out schedule as well.

# 
# script to keep tabs on selectPro data and email when out of bounds.
# uses pymail2/3.py to send emails
# looks at selectronic.json which is written to file by soc_mqtt_mod1.py
# every 10 sec
# this json file can be used to check threshold values etc in almost real time (no data logged)
# lots of print statements to help in setting up - can be commented out
import time, datetime
import sys
import json
import schedule, time, datetime
import pathlib
import os
count = 0
files = []
fname = pathlib.Path('/home/tony/python/emon_select/selectronic.json')
assert fname.exists(), ('No such file: {fname}')  # check that the file exists
# test if SoC continuously decreasing - will return True or False
def strictly_decreasing(files):
    return all(files[i] > files[i+1] for i in range(len(files)-1))
# read json and put into dict
def read_json():
    #print('def read_json called')
    global count
    with open('selectronic.json') as json_file:
        data = json.load(json_file)
        #check if soc getting low
        print('json data: ',data)
        files.append(data['soc'])
        print('length of files = ',len(files))
        # make string for calling email
        call_email = 'python3 pymail3.py '+ str(files)
        os.system(call_email)
        if len(files) == 10:
            files.pop(0)
        result = strictly_decreasing(files)
        print(files)
        if len(files) == 9 and result and data['soc']<50:
            count += 1
            if count <= 1:
                os.system('python3 pymail3.py passedSoC')
                print('email alert sent!')
            else:
                print('all seems OK...')
                count = 0
            time.sleep(20)
        if data['soc'] < 50:
            print('SoC low alert! - send email...')
            
schedule.every(30).minutes.do(read_json)
while True:
    schedule.run_pending()
    sys.stdout.write('.')
    sys.stdout.flush()
    time.sleep(10)

The last script actually sends the email. All 3 scripts could probably be merged into 1, but I found it easier to troubleshoot as separate scripts - and the email script needed tuning.

import time, datetime
import smtplib, ssl
import sys
from email.message import EmailMessage

message = []
message = sys.argv
print('Argument List :', str(sys.argv))
print('Argument List #1:', message[1])

tstamp = datetime.datetime.now(tz=None)
date_time = tstamp.strftime('%d-%m-%Y %H:%M:%S')
sender_email = "[email protected]"
receiver_email = "[email protected]"
msg = EmailMessage()
msg.set_content('Stuff passed through ' + str(message) + ' timestamp = ' + str(date_time))

msg['Subject'] = 'Something like python test notification: ' 
msg['From'] = sender_email
msg['To'] = receiver_email

port = 465  # For SSL
password = 'my gmail pwd here' # or use manual input 
#input("Type your password and press enter: ")
# Create a secure SSL context
context = ssl.create_default_context()

with smtplib.SMTP_SSL("smtp.gmail.com", port, context=context) as server:
    server.login("[email protected]", password)
    server.send_message(msg)

This works with gmail, but should work with other smtp mail systems with a bit of tuning. It’s pretty simple to setup a second or third gmail account - just google it.

1 Like

Anthony,
I want to send my thanks for sharing. This is my first time programming in Python, so I am digesting your code. In addition, I am integrating it with MQTT, since I have plan to use MQTT in the future.
Not sure how long it will take, but will post it when finished.

I found python mqtt not too hard to setup - I used paho.mqtt.client and found some useful hints on the web. If you are using mqtt on emoncms you’ll need the standard user and password if you’ve not changed them. This is how I’ve used it - my data is in a python dictionary data_dict[“A9val”] which publishes to emon/rain:

import paho.mqtt.client as mqttClient

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print("Connected to broker")
        global Connected
        Connected = True
    else:
        print("Connection failed")

Connected = False # for state value

broker_address = "192.168.20.xxx" # your broker address
port = 1883
user = 'emonpi'
password = 'emonpimqtt2016'
client = mqttClient.Client("Python")
client.username_pw_set(user, password=password)
client.on_connect = on_connect
client.connect(broker_address, port = port)
client.loop_start()

client.publish("emon/rain/rainfall",data_dict["A9val"])

``

@dimpryor @icenov

I have found MQTT in a Python script too much of a challenge.

But there is a neat work around whereby individual bash commands can be included in a Python script:

Upfront in the Python script include:


import os

Then incorporate the bash MQTT command as shown in the example:


os.system("mosquitto_pub -h localhost -r -u emonpi -P emonpimqtt2016 -t emon/openevse/divertmode/set -m 1")

The bash command is simply inserted between the double quotes.

This may help …

1 Like

John and Anthony, thanks for all the feedback. Finally got around to finishing the program. Ran it for several days without any issues. I attached it as a pdf instruction for others to reference if they need an email alert.
Email Alert Instructions.pdf (67.3 KB)