Archive: emonSD-07Nov16 RELEASE 🎉

[glyn.hudson]

I’ve put together a new emonSD pre build Raspberry Pi SD card incorporating many changes and some important security updates. It would be great if anyone could test the new image (@jon ?) and report back if it works ok for you.

Emocms data can be migrated using built-in Emoncms backup (import / export) module: Import / Backup - Guide | OpenEnergyMonitor

Full emonSD changelog & Download page (Wiki)

---

emonSD-07Nov16 BETA

File System

Stick with 4GB min SD card (8GB+ recommended). Expand partition with sudo emonSDexpand

Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  2.1G  1.2G  64% /
devtmpfs        483M     0  483M   0% /dev
tmpfs           487M     0  487M   0% /dev/shm
tmpfs           487M  6.6M  480M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           487M     0  487M   0% /sys/fs/cgroup
tmpfs            40M  3.8M   37M  10% /var/lib/openhab
tmpfs           1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
tmpfs           1.0M     0  1.0M   0% /var/lib/dhcp
tmpfs            50M  328K   50M   1% /var/log
tmpfs            30M   52K   30M   1% /tmp
/dev/mmcblk0p1   60M   21M   40M  35% /boot
/dev/mmcblk0p3  194M   37M  147M  21% /home/pi/data

Linux

• Update Linux kernal to at least 4.4.26-v7+ and get latest security & raspi firmware fixes (e.g. Dirty Cow) (dist-upgrade) forum topic

• $ apt-get clean all (free up unused packages, approx 700Mb)

Emoncms

• Latest Emoncms (currently V9.7.7)
• New graph module
• Lots of dashboard fixes and improvements

emonPi

• Remove personal GitHub credentials
• Fix Mosquitto MQTT server hanging after V1.4.10 update forum thread
• Add bash prompt RW indicator, add to /etc/bash.bashrc
• Install emonUpload to enable easier user emonTx, emonTH firmware updates
• Updated motd
• PlatformIO installed for on-device firmware compiling & updating. See blog post.

nodeRED

• Add weather underground nodeRED node + flow example

OpenHAB

• Update to Java 8 - fix my.openhab connection issue forum thread
• Disable OpenHAB Jetty server request logs. Stop filling up /var/log partition.

Emoncms Server Info

Server Information
Emoncms	Version	low-write 9.7.7 | 2016.10.29
Modules	app, config, dashboard, graph, wifi
Buffer	0 feed points pending write
Writer	Daemon is running with sleep 60s
Server	OS	Linux 4.4.26-v7+
Host	emonpi emonpi (127.0.1.1)
Date	2016-11-01 00:52:38 UTC
Uptime	00:52:38 up 5 min, 1 user, load average: 0.76, 0.65, 0.31
HTTP	Server	Apache/2.4.10 (Raspbian) HTTP/1.1 CGI/1.1 80
Database	Version	MySQL 5.5.52-0+deb8u1
Host	localhost (127.0.0.1)
Date	2016-11-01 00:52:38 (UTC 00:00‌)
Stats	Uptime: 5583 Threads: 3 Questions: 1699 Slow queries: 0 Opens: 61 Flush tables: 1 Open tables: 50 Queries per second avg: 0.304
Redis	Version	2.8.17
Host	localhost:6379 (127.0.0.1)
Size	0 keys (471.91K)Flush
Uptime	0 days
MQTT	Version	1.4.10
Host	localhost:1883 (127.0.0.1)
Pi	CPU Temp	41.86°CShutdownReboot
Release	emonSD-01Nov16
Memory	RAM	
Used 25.23%
Total: 973.11 MB Used: 245.53 MB Free: 727.58 MB
Disk	Mount	Stats
/	
Used 60.45%
Total: 3.33 GB Used: 2.01 GB Free: 1.16 GB
/boot	
Used 34.67%
Total: 59.95 MB Used: 20.78 MB Free: 39.16 MB
/home/pi/data	
Used 19.04%
Total: 193.66 MB Used: 36.87 MB Free: 146.8 MB
PHP	Version	5.6.27-0+deb8u1 (Zend Version 2.6.0)
Modules	Core   date   ereg   libxml   openssl   pcre   zlib   bcmath   bz2   calendar   ctype   dba   dom   hash   fileinfo   filter   ftp   gettext   SPL   iconv   mbstring   session   posix   Reflection   standard   shmop   SimpleXML   soap   sockets   Phar   exif   sysvmsg   sysvsem   sysvshm   tokenizer   wddx   xml   xmlreader   xmlwriter   zip   apache2handler   PDO   curl   dio   json   mcrypt   mosquitto   mysql   mysqli   pdo_mysql   readline   redis   mhash   Zend OPcache  

By default emonSD has a number of services running, if you don’t want to use these they can be disabled with:

pi@emonpi:~ $ sudo systemctl disable openhab.service  
pi@emonpi:~ $ sudo systemctl disable nodered.service  
pi@emonpi:~ $ sudo systemctl disable emonPiLCD.service  
pi@emonpi:~ $ sudo systemctl disable apache2.service  

[Dave]

Hi Glyn
Can the emonpi be updated just with using the update button or do you have to download a new image?

Kind Regards
Dave

[Andy_Taylor]

Downloading at the moment, I’ll see if I can do anything with the 1.5GB download size…

[glyn.hudson]

You will get all the important security updates and Emoncms updates by running:

Admin > emonpiupdate in local Emoncms and upgrading Debian packages

rpi-rw
sudo apt-get update && sudo apt-get upgrade
sudo apt-get dist-upgrade 
sudo reboot

When given the option choose to keep the version of php.ini currently installed

[Jon (Jon Murphy)]

@glyn.hudson

sudo apt-get upgrade && sudo apt-get upgrade

the 2nd command line is the same command twice… should be this?

rpi-rw
sudo apt-get update
sudo apt-get dist-upgrade
sudo reboot

[Andy_Taylor]

@glyn.hudson

rpi-rw
sudo apt-get update && sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo reboot

changes to the corrected lines…

[Andy_Taylor]

@Andy_Taylor

Downloading at the moment, I’ll see if I can do anything with the 1.5GB download size…

Crushed it down to an 825MB download…
825MB download version
MD5: 3961e96cf2e1ab46d750d0a0cae72a2e emonSD-07Nov16.zip
MD5: cf8537e90ffd98ffb5838fbe3c878d4d emonSD-07Nov16.img

[glyn.hudson]

Did you try and boot the image? Does your version boot ok?

[glyn.hudson]

@Jon

the 2nd command line is the same command twice… should be this?

Yes, correct. Update then Upgrade. I’ve fixed my post above

[Andy_Taylor]

Yes and yes, I’m planning to write some full instructions on how I manage to compress the image so far, its not that complex, but there are a few steps to it.

I’ll re-download your version and make up a full how to with all the steps in it, plus the commands etc, that should help you (or anyone else) understand how to really pull the size down prior to putting images up for download.

[Jon (Jon Murphy)]

Glyn - I started my testing last night but quickly found this was a 4GB image instead of the 8GB image discussed here: Increase emonSD pre-built SD card to 8GB min

I had been writing (trying to write?) a migration script that captures the items not grabbed by the backup/export script. And I quickly ran out of space on the /data directory:

/dev/mmcblk0p3  194M   37M  147M  21% /home/pi/data

Is it possible to create the 8GB image instead?

[Jon (Jon Murphy)]

I booted the emonPi with the beta 4GB image, let it do the first boot emonPi update, and then did the expand for an 8GB card (thank you Paul!):

sudo emonSDexpand

 
After the emonPi reboot I saw this on the display:

ERROR: MQTT
Not connected

 
It is easily corrected with:

sudo service emonPiLCD restart

I tried a sudo reboot and a paperclip shutdown. Sometimes it boots fine and other times I get the above mqtt display error.

[Jon (Jon Murphy)]

for the sudo raspi-config area - Can the en_US.UTF-8 UTF-8 be clicked on by default?

[Jon (Jon Murphy)]

Can the update-nodejs-and-nodered command be run as part of the build?

Updates:

• Node-RED version: v0.15.2 (I think the Nov build already had v0.15.2)
• node.js version: v4.6.2 (from version 0.10.x)
• npm version: v2.15.11 (from version 1.4.x)

PS - my apologies for the rapid fire posts!

[Jon (Jon Murphy)]

This error is in the first boot emonPi Update:

(I do not have a huawei device)

[glyn.hudson]

@Andy_Taylor

Crushed it down to an 825MB download…
825MB download version
MD5: 3961e96cf2e1ab46d750d0a0cae72a2e emonSD-07Nov16.zip
MD5: cf8537e90ffd98ffb5838fbe3c878d4d emonSD-07Nov16.img

Hi @Andy_Taylor, just tested your packed down version. It seems to work great for me. It would be interesting to hear what you did to it to shrink it down!

---

Hi @Jon , thanks a lot for your testing. Sorry for the delay in getting back I’ve been away.

@Jon

Glyn - I started my testing last night but quickly found this was a 4GB image instead of the 8GB image discussed here: Increase emonSD pre-built SD card to 8GB min

As mentioned in the change log for the emonSD 07Nov16, I decided to keep the image 4GB. This was party due to @Andy_Taylor 's space optimisations. The image can be expanded to fill 8GB with sudo emonSDexpand.

@Jon

After the emonPi reboot I saw this on the display:

ERROR: MQTT
Not connected

Interesting, did the message say forever or if you left it for 5min or so would MQTT connect?

@Jon

for the sudo raspi-config area - Can the en_US.UTF-8 UTF-8 be clicked on by default?

What benefit would this have?

@Jon

Can the update-nodejs-and-nodered command be run as part of the build?

Nodejs and nodered should be updated via apt for the stable versions. IMO I would prefer to stick to the stable builds released via apt and the Debian packages. Keen users can always update manually if desired.

@Jon

This error is in the first boot emonPi Update:

Thanks for letting me know, however this is not a big issue since there have been no changes to the huawei repo, however I have fixed the permission error during the update script:

fix permissions

[glyn.hudson]

Download file has been updated to @Andy_Taylor compressed 825MB image:

https://github.com/openenergymonitor/emonpi/wiki/emonSD-pre-built-SD-card-Download-&-Change-Log#emonsd-07nov16

[Jon (Jon Murphy)]

@glyn.hudson

Interesting, did the message say forever or if you left it for 5min or so would MQTT connect?

small forever - the ERROR: MQTT message was there for a few hours.

 

@glyn.hudson

What benefit would this [en_US.UTF-8 UTF-8 clicked on] have?

I am assuming this is a locales configuration related to the US:

I believe this is something like the format of mm/dd/yyyy versus dd/mm/yyyy.

Is this a bad?!? Should I not be picking the US locales?

 

@glyn.hudson

Nodejs and nodered should be updated via apt for the stable versions. IMO I would prefer to stick to the stable builds released via apt and the Debian packages. Keen users can always update manually if desired.

Per the Node-RED website: “We recommend the use of node.js LTS.”

NodeJS v0.10 does not work with some nodes (like the Nest node: node-red-contrib-nest v0.1.9)

[dweston (Dion Weston)]

I’m a rank newbie with emon. By way of feedback though; I managed to download and build the emonSD-07Nov16 BETA SD card. It ran flawlessly through the install, partition expansion and initial registration of Emoncms 9.7.7 | 2016.10.29.

Architecture:          armv7l
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    4
Socket(s):             1
Model name:            ARMv7 Processor rev 5 (v7l)