Compiling emonPi SD card

Hi Forum!

I need to make a prepared image, but with one line modified in the EmonHub (remove Apikey!=32 check).
I wish to replicate the exact latests ‘pre built emonsd’ but with this change only.
Could someone point me the right direction to achieve this please?
I am currently looking at this link: emonpi/SD-card-build.md at master · openenergymonitor/emonpi · GitHub

Edit: Where can I download the correct/same RASPBIAN image? I see here is some offical older version, but it the 2015-11-21 date doesnt match?

Edit2: Is it possible to just download the latest image, boot it up on a usb stick, modify the code, and then re-save the image back to pc?

Thanks!

Sure! Latest image can be downloaded from here: emonSD pre built SD card Download & Change Log · openenergymonitor/emonpi Wiki · GitHub

Yes, but you would be better off “mounting” the usb stick image rather than “booting” as the image automatically updates itself on first boot so the image will not be “the same” image and will not subsequently auto update on future first-boots of any copies of this image.

The simplest way is to write the image to a 4gb usbstick (or 4gb sdcard & usbadapter), plug it into a working Pi (or Linux PC) and run

sudo mkdir ~/usbstick
sudo mount /dev/sda1 ~/usbstick
sudo nano ~/usbstick/home/pi/emonhub/src/interfacers/file_to_be_edited.py

“sda1” assumes it is the only or the first detected usb storage device. Once edited, you can sudo umount /dev/sda1 and make a copy of the usbstick image in your usual way, the saved image will be the same as the original but with the edit, even the logfiles and time etc will be the same.

By using a 4gb usbstick or sdcard you will also retain the smaller size, unless you use DD or similar to save your new image it will become the size of the usbstick used, and eg a 8gb image will take up twice as much space on your workstation and take twice as long to install to other sdcards, not to mention imposing a much higher minimum sdcard size.

Although IMO the better way forward is to show us your edit and where it is implemented in the code to weigh up if we can get the code updated since emoncms now accepts node ids over 32. This would eliminate any need for you to manually edit anything as the edit would get pulled in on the initial auto-update at first boot, it would also prevent any problems at a later date if emonhub is updated and your local changes prevent those changes being pulled in before manually stashing the edits and reapplying them afterwards.

That restriction was reluctantly added to the original emonhub as the 32 node restrictions in both JeeLib and emoncms (at the time) meant there was no good reason not to implement it and was basically another level of checks to reduce rogue inputs, but obviously could only ever stop rogue inputs outside the valid node id range which was not really that helpful. I already run my own version of emonhub without this “32 node” restriction but it is not the emonPi variant so my own changes are not relevant to the version on the sdcard.

EDIT - My apologies that node id check was already removed, I missed the “Apikey” which sent me off on a tangent about the node id, What I’ve said about the editing method still stands but it’s unlikely that code would be changed in main repo as all apikeys (to my knowledge) are 32 chars (128bit) do you have a different use case?

I have a different use case.
The key lengths are not fixed, so I need to remove this length check. I am currently just modifying it live via ssh.
Python doesn’t need to compile or build right?
Is there a script available to build this entire EmonPI SD from scratch in one go?

No, I’m afraid it’s a manual build following: emonpi/SD-card-build.md at master · openenergymonitor/emonpi · GitHub

What do you mean by different key lengths?

I am using Grafana as a front-end, and API keys are generated differently (and more securely)

No Python doesn’t need compiling or building, you can just edit the .py file and it’s interpreted/compiled at runtime so if you do it on a live system you will need to restart the emonhub service after the changes, using the “mount” method I suggested, it will just work from first boot.

So are you posting direct to Grafana or using the Grafana generated key as the emoncms apikey?

I introduced this very basic “is the apikey length 32” to emonhub as 32chars was the only apikey option at that time, much like the <32 node id check there was little reason not to add the check, but since it only checks the length it has limited use, it was just to spot the obvious and not send data that was obviously not going to be accepted by emoncms.

Looking at the emonpi variant though, there is no good functional reason for keeping this check, yes it can still create a log entry if the apikey is the wrong length, but not if any one of the 32 chars is wrong.

The check was designed to defer sending until the apikey is corrected. Even if the apikey is incorrect the “original” emonhub would not delete the buffered data until it got a confirmation from emoncms the data had been received, so removing the check only stops sending with a non-32char apikey that would still fail safe and not lose data if the apikey is incorrect.

Unfortunately the changes made to emonhub for the emonpi variant mean the data buffer is cleared every sendinterval regardless of a correct size apikey or even a network connection, so the check can currently only log the fairly obvious, it cannot save data from being discarded.

I would be interested to hear about your Grafana set up to, I have recently looked at it as an alternative front end because of it’s multi-organisational setup with 3 different admin/editor/reader user levels per org and it’s annotations would be invaluable to me. I could not find much info on mysql datasources and since it is not a trivial setup on a Pi I decided to wait until their hosted service was available.

Yeah Grafana looks cool. Not heard of it before. Impressive it’s open source http://grafana.org/

I am using the Grafana generated key. I have a micro-service that tests the key, and the continues to a function that parses the dataset into Influxdb.

I am not aware of that any mysql datasources are available. I could assist you with having a ‘custom hosted version’ if you want, write me in PM.