SBFSpot

def OrderedDict_to_JSON(): #https://docs.python.org/2/library/collections.html#collections.OrderedDict
    global smaJSONstring

    names_firstrow_ordered = OrderedDict(zip(names, rows[0]))

    smaJSONstring = json.dumps(OrderedDict(names_firstrow_ordered))

    print (smaJSONstring)

Produces:

{"TimeStamp": 1534325149, "Serial": 304991661, "Pdc1": 1154, "Pdc2": 1390, "Idc1": 5.033, "Idc2": 5.538, "Udc1": 229.46, "Udc2": 251.13, "Pac1": 828, "Pac2": 833, "Pac3": 831, "Iac1": 3.509, "Iac2": 3.508, "Iac3": 3.507, "Uac1": 236.01, "Uac2": 237.54, "Uac3": 237.17, "EToday": 4437, "ETotal": 13844060, "Frequency": 49.99, "OperatingTime": 12077.6, "FeedInTime": 11648.3, "BT_Signal": 0.0, "Status": "OK", "GridRelay": "Closed", "Temperature": 72.53}

In the JSON string, there are values containing alpha-numeric strings for “Status” and “GridRelay”.
How does emonCMS handle this? Shall I rid these Objects?

Recap/sanity check… My plan is to create a script which runs alongside SBFspot.
If I get that far it will:

  1. poll the SBFspot database every few seconds to find an updated record in table ‘SpotData’ at index[0].
  2. send the data to emonCMS locally or remotely, at the moment the method is to create a JSON string of that most recent record at index[0], somehow include the relevant API key, and post to emonCMS.

I think the main advantages of the method is that is caters for those who don’t have emonCMS running on their Pi, and those who do. Also, for local installs, having the script running locally means data is collected into emonCMS regardless of internet connectivity. Plus, as I see it, SBFspot is well-maintained for ethernet and bluetooth.

emoncms.org to have a wrapper to grab data from PVO is definitely also a good idea :slight_smile:

EDIT: I’m still unsure if that’s the proper use of the term ‘wrapper’.
EDIT EDIT: and having that wrapper on emonCMS locally also I see the benefit of, although I imagine it’ll be used less often.

Not quite, my suggestion of the “wrapper” AROUND emoncms was so that emoncms emulates PVOutput in that if you point a program written to talk to PVOutput to the wrapper on emonCMS it wouldn’t know the difference.

Don’t let this stop you in your current task though, sounds like you are on a route to success.

Couple of points to watch out for:

  1. MySQL (or MariaDB) will probably write to the SD card a lot so wear may be a problem and your Rasp PI dies :frowning:
  2. MySQL will eventually fill up the SD card, so think about truncating the data to keep a rolling week or so of data if you don’t intend to do anything with it except push into emoncms.
  3. You may be able to configure MySQL (see point 1) to use in-memory tables to save on disk space but this may be a limit on the raspberry pi
1 Like

Progress :sunny: :sunny: :sunny:

def url_stringing():
    global URLstring

    emoncmsaddress = "emoncms.org"
    node_id = "sbfspot"
    URLstring = "https://" + emoncmsaddress + "/input/post?node=" + node_id + "&fulljson=" + str(smaJSONstring) + "&apikey=" + apikey
    print URLstring

def post_to_emonCMS():
    r = requests.get(URLstring)
    print (r)

The problem I mentioned earlier of SBFspot not running after restart, was that the UploadDaemon was unable to write to a directory (I was unable to change the directory permissions).
The solution was to move the smadata folder to /home/pi/data/, which has permissions 777 set by default, and to change the SBF conf and UploadDaemon conf files to point to the new location.

New thread is here for redis CPU load:

1 Like

I’m trying to type a # in dataplicity on mac and it’s driving me nuts.

I have my python script initiated by a .sh which is initiated by cron upon initiation :smile: the script’s on the same interval as SBFspotUploadDaemon.

Why not start the Python script directly from your crontab?

That’d be simpler.
I’m trying also to make it start a few seconds after the upload daemon…

Would a sleep commmand near the start of your Python script do the trick?

It would.
I’ve realised though that a sleep command is in fact unnecessary.
The core SBFspot program is inputting to the database, it’s independent of the UploadDaemon, my script happening a few ms after the UploadDaemon will be grabbing the same data from the database. All good.

Done, I think, just in time for the weekend.

When the SD card image has been rebuilt for 3B+ I’ll look at using MySQL instead of SQLite. The existing users of sbfspot.3 will probably be using SQLite.

“I think”…
I had to change something. Needs testing a little longer and I’ll report back :slight_smile:

Seems to be working. :star_struck:
https://emoncms.org/graph/embed?graphid=13478

How do I point the HTTP request to localhost within the Pi?

http://localhost usually works

On the odd chance localhost isn’t defined in the /etc/hosts file, http://127.0.0.1 will work.

It worked although took a moment to figure out, instead of https://emoncms.org/, I had to enter http://localhost/emoncms/

I asked because I saw reference to webhost defined as 127.0.1.1 somewhere in the docs, which didn’t work for clear reasons now.

I’ve my script sending to both local and remote with a tidy extra couple of lines of code.

If you can remember where you saw it, let us know so it can be corrected.