rPi 3B+ getting hot. Redis high CPU usage. (emonSD-13Jun18 dev. specific)

Emoncms Installation
,
41

I got my python script working but it doesn’t seem to trigger from cron for some reason…

I’ll keep tinkering over the next few days.
Here’s what top looks like with a 0.1s sleep:

image
image789×202 12.6 KB

So its much better than the bash script.

42

Ok, it was a simple problem now fixed…
here’s the script @glyn.hudson if you want to give it a try instead of the bash script…

[EDIT] Updated the script - now cleanly handles being TERM’d by killall or kill

#!/usr/bin/python

import fcntl
import sys
import os
import redis
import subprocess
import time
import datetime
import signal

def handle_sigterm(sig, frame):
  print(datetime.datetime.now().isoformat(' ') +": Got Termination signal, exiting")
  fcntl.flock(l, fcntl.LOCK_UN)
  l.close()
  os.remove(pidfile)
  sys.exit(0)

## Used to update log viewer window in Emoncms admin
# Used in conjunction with: service-runner-update.sh and Emoncms admin module

scriptname = os.path.basename(sys.argv[0])
pidfile = '/tmp/' + scriptname

# lock it
try:
  l = open(pidfile, 'w')
  fcntl.flock(l, fcntl.LOCK_EX | fcntl.LOCK_NB)
except:
  sys.exit(1)

# Setup the signal handler to gracefully exit
signal.signal(signal.SIGTERM, handle_sigterm)
signal.signal(signal.SIGINT, handle_sigterm)

l.write("%s\n" % os.getpid())

print(datetime.datetime.now().isoformat(' ') +": Starting service-runner")
sys.stdout.flush()

server = redis.Redis()
while True:
  try:
    if server.exists('service-runner'):
      flag = server.lpop('service-runner')
      print(datetime.datetime.now().isoformat(' ') +": got flag: %s\n" % flag)
      sys.stdout.flush()
      script, logfile = flag.split('>')
      cmdstring = "{s} > {l}".format(s=script, l=logfile)
      print(datetime.datetime.now().isoformat(' ') + ": STARTING: " + cmdstring)
      sys.stdout.flush()
      subprocess.call(cmdstring, shell=True)
      if not (os.path.isfile(logfile)):
        f = open(logfile, 'a')
        f.close()
      print(datetime.datetime.now().isoformat(' ') + ": COMPLETE: " + cmdstring)
      sys.stdout.flush()
  except:
    print("Exception occurred", sys.exc_info()[0])
  time.sleep(0.2)

I’m no python expert, so feel free to modify!
I have noticed the print statements seem to be buffered and are not ending up in the log until the process ends, so maybe some improvement there. Also fixed now

Otherwise, it seems to work fine for triggering updates.

[EDIT] Here’s top with that 0.2s sleep interval
image

1 Like
43

For discussion purposes, I’ve submitted a PR for my service-runner re-write…

44

Did you get this resolved?

45

Yes I did

46

What was it?
chown 755?

47

It was a problem in the script - it was correctly being triggered by cron but it was causing an exception that crashed the script. I blame my lack of python familiarity, that being my first real python script. :slight_smile:

The script above (and in the github PR) works correctly as a drop-in replacement for the existing service-runner file as far as I can make out with my testing.

1 Like
48

Confirms for me that chown 755 isn’t necessary for python script to run by cron!
Congratz, I made my second python script recently. As far as languages go I think it’s a pleasure to use.

49

Ah, that really depends how you run it!
In this case, service-runner is already 775 and the way it is already called from cron requires it to be. My point was that is not what my problem was.

50

Oh, and you probably meant chmod, not chown :slight_smile:

51

OK interesting :slight_smile:

  subprocess.call(cmdstring, shell=True)

So this is the way to call another file, in this case the .sh updater, from python? if so, neat :slight_smile:

52

urh, probably. :wink:

Edit: Urh, Yes. Exactly what I mean. Good morning chmod :crazy_face:

53

Certainly one way, yes. With my (lack of) Python experience though, I wouldn’t like to comment on whether it is the way.

54

Understood :slight_smile:
Whatever works cleanly at the end of the day is alright.

55

Throughout this, off topic, I was wondering if it was possible to call scripts more directly from the browser. As a question it’d be… What’s the most direct javascript to .sh/.py connection possible?

56

Thanks for this @Greebo, I’ve just tested and have posted on the github PR:

57

Python service runner has now been merged into Emoncms core. Big thanks to @Greebo and everyone who helped testing.

2 Likes
58

Will this require some documentation on how to migrate?

59

Documentation is here:

Upgrade is not essential, the bash version will continue to work fine for older emonPi installs. New emonSD images will use the new python version.

60

I noticed that the updater shell script on the emonPi image adds a service-runner.sh crontab entry to the pi user’s crontab whenever it runs. Is that still the case if you update? That could cause two service-runner’s to be running (the python systemd service and the cron triggered version).