New emonSD release: emonSD-26Oct17 🎉

glyn.hudson · 27 October 2017 16:44

We have just published an emonSD update: emonSD-26Oct2017.

IMG_20171027_174103

All emonPi / emonBase units purchased via the shop will be shipped with this version

This update is an incremental update with mainly security fixes including KRACK patch.

Existing users can apply the KRACK WAP vulnerability patch by following the instructions:

There is no need for existing users running emonSD-07Nov16 to upgrade to the new image. However if your running a version old then emonSD-07Nov16 I would recomend updating to this latest version: emonSD-26Oct17. You will need to backup your data using Emoncms backup, see backup / import user guide.

Changelog

See below for the key changes to emonSD-26Oct17 compared to the previous release emonSD-07Nov16. This change log has bee copied from the emonSD changelog and download page (wiki), see this page for further download and flashing details:

emonSD-26Oct17

Download (1.4GB)
Canadian Mirror (1.4GB)

(.img) MD5: 3b7c2ebc299239a5d9a9d43b25beabe8
(.zip) MD5: 40ec7c1e8bcdd81d4035ff640b07a788

New changes compared with previous release, SD-card-build.md has been updated:

Based on Debian Raspbian Jessie minimal, updated to latest packages, kernel and firmware. Includes patch for KRACK WPA vulnerability:

$ uname -a
Linux emonpi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux
$ sudo /opt/vc/bin/vcgencmd version
Jul  3 2017 14:17:30 
version 4139c62f14cafdb7d918a3eaa0dbd68cf434e0d8 (tainted) (release)

Automatic NTP time update: see forum thread and changes.
Fix random seed: improved HTTPS / SSH security. See forum thread.
Use dtoverlay=pi3-miniuart-bt instead of dtoverlay=pi3-disable-bt in /boot/config.txt
This re-maps RasPi3 bluetooth to software serial/dev/ttyS0 instead of disabling it.

File System

4GB min SD card (8GB+ recommended). If SD card is larger than 4GB, expand data partition with sudo emonSDexpand

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       3.4G  2.0G  1.2G  63% /
devtmpfs        481M     0  481M   0% /dev
tmpfs           486M     0  486M   0% /dev/shm
tmpfs           486M  6.6M  479M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           486M     0  486M   0% /sys/fs/cgroup
tmpfs            40M  6.1M   34M  16% /var/lib/openhab
tmpfs           1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
/dev/mmcblk0p1   60M   22M   39M  37% /boot
tmpfs           1.0M     0  1.0M   0% /var/lib/dhcp
tmpfs            50M  480K   50M   1% /var/log
tmpfs            30M  152K   30M   1% /tmp
/dev/mmcblk0p3  3.5G   39M  3.3G   2% /home/pi/data

Emoncms Server Information

Emoncms	Version	low-write 9.8.10 | 2017.08.17
Modules	Administration | App v1.0.0 | Backup v1.0.0 | EmonHub Config v1.0.0 | Dashboard v1.1.1 | EventProcesses | Feed | Graph v1.0.0 | Input | postprocess | CoreProcess | Schedule | setup | Time | User | Visualisation | WiFi v1.0.0
Buffer	0 feed points pending write
Writer	Daemon is running with sleep 60s
Server	OS	Linux 4.9.35-v7+
Host	emonpi emonpi (127.0.1.1)
Date	2017-10-27 16:04:08 UTC
Uptime	16:04:08 up 6 min, 1 user, load average: 0.09, 0.17, 0.09
HTTP	Server	Apache/2.4.10 (Raspbian) HTTP/1.1 CGI/1.1 80
Database	Version	MySQL 5.5.57-0+deb8u1
Host	localhost (127.0.0.1)
Date	2017-10-27 16:04:08 (UTC 00:00‌)
Stats	Uptime: 82668 Threads: 3 Questions: 196 Slow queries: 0 Opens: 59 Flush tables: 1 Open tables: 51 Queries per second avg: 0.002
Redis	Version	2.8.17
Host	localhost:6379 (127.0.0.1)
Size	13 keys (473.56K)Flush
Uptime	0 days
MQTT	Version	1.4.14
Host	localhost:1883 (127.0.0.1)
Pi	CPU Temp	40.78°CShutdownReboot
Release	emonSD-26Oct17
File-system	Set root file-system temporarily to read-write, (default read-only)Read-Write Read-Only
Memory	RAM	
Used 25.03%
Total: 970.93 MB Used: 242.99 MB Free: 727.94 MB
Disk	Mount	Stats
/	
Used 59.18%
Total: 3.33 GB Used: 1.97 GB Free: 1.2 GB
/boot	
Used 36.32%
Total: 59.95 MB Used: 21.77 MB Free: 38.17 MB
/home/pi/data	
Used 1.09%
Total: 3.46 GB Used: 38.69 MB Free: 3.25 GB
PHP	Version	5.6.30-0+deb8u1 (Zend Version 2.6.0)
Modules	apache2handler | bcmath | bz2 | calendar | Core v5.6.30-0+deb8u1 | ctype | curl | date v5.6.30-0+deb8u1 | dba | dio v0.0.4RC4 | dom v20031129 | ereg | exif v1.4 | fileinfo v1.0.5 | filter v0.11.0 | ftp | gettext | hash v1.0 | iconv | json v1.3.6 | libxml | mbstring | mcrypt | mhash | mosquitto v0.3.0 | mysql v1.0 | mysqli v0.1 | openssl | pcre | PDO v1.0.4dev | pdo_mysql v1.0.2 | Phar v2.0.2 | posix | readline v5.6.30-0+deb8u1 | redis v2.2.7 | Reflection | session | shmop | SimpleXML v0.1 | soap | sockets | SPL v0.2 | standard v5.6.30-0+deb8u1 | sysvmsg | sysvsem | sysvshm | tokenizer v0.1 | wddx | xml | xmlreader v0.1 | xmlwriter v0.1 | Zend OPcache v7.0.6-devFE | zip v1.12.5 | zlib v2.0 | 





https://github.com/openenergymonitor/emonpi/wiki/emonSD-pre-built-SD-card-Download-&-Change-Log#emonsd-26oct17

Nolan_Garrett · 24 November 2017 08:24

Crazy question but - does this build work? I’ve tried writing it to multiple SD cards, and all fail to mount /home/pi/data. I’ve tried multiple imagers as well, on two different computers.

glyn.hudson · 24 November 2017 12:28

Yes it works. What process are you using to flash the image to the SD card. Recommend using Linux dd or etcher. Some tools don’t flash all the partitions.

We sell a pre-flashed SD card with this image pre-loaded if you’re still having trouble;

Nolan_Garrett · 24 November 2017 21:20

So far I’ve tried Etcher (latest version) and Win32DiskImager on two different Windows machines, both attempting to use two different microSD cards. I’ve also tried swapping SD to microSD adapters.

I also just tried an OS X system and using dd, with exactly the same behavior. On all three systems I downloaded the image from the site and verified the MD5 sum of the .img file. Both SD cards have no problems if I write Raspbian to them and boot them.

In every case I get this error on boot:

Failed to mount /home/pi/data.
See 'systemctl status home-pi-data.mount' for details.

When I run the systemctl command, I see home-pi-data.mount exited with status 32, and mount said
mount: wrong fs type, bad option, bad superblock on /dev/mmcblk0p3

I will keep messing with this but I’m quickly running out of ideas!

Nolan_Garrett · 24 November 2017 22:40

I believe I can demonstrate that the emonSD-26Oct17 image available for download here at is in fact corrupt in some way. Please note that the MD5 sum of the emonSD-26Oct17.img file is 3b7c2ebc299239a5d9a9d43b25beabe8.

fdisk -l ./emonSD-26Oct17.img
Disk ./emonSD-26Oct17.img: 3.6 GiB, 3838837248 bytes, 7497729 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0003ea3e

Device                Boot   Start      End Sectors  Size Id Type
./emonSD-26Oct17.img1 *       8192   131071  122880   60M  6 FAT16
./emonSD-26Oct17.img2       131072  7288831 7157760  3.4G 83 Linux
./emonSD-26Oct17.img3      7288832 14786559 7497728  3.6G 83 Linux

Using the following command I can mount the first two partitions:
mount -t auto -o loop,offset=$((512*8192)) emonSD-26Oct17.img /mnt
mount -t auto -o loop,offset=$((512*131072)) emonSD-26Oct17.img /mnt

However, mounting the third partition fails:
mount -t auto -o loop,offset=$((512*7288832)) emonSD-26Oct17.img /mnt
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error

In some cases useful info is found in syslog - try
dmesg | tail or so.

You’ll also notice that while this image should support a 4GB microSD card, the total size of the partitions found by fdisk calculates out to just over 7GB.

I can perform all of these commands on the older Nov16 image and everything seems to work.

Any idea where I can get a “good” copy of the Oct17 image?

rbrt · 25 November 2017 10:39

I have also been having issues flashing the emonSD-26Oct17 image to a SD card, I’ve tried an 8GB, 16GB and 32GB SanDisk Ultra with both the Win32DiskImager and Etcher without much luck. As soon as I boot from the SD card it seems to go into systemd-fsck for hours and get stuck:

I don’t think it can be the SD cards as I’ve previously used all 3 with Jessie and Stretch on the Pi3 without issue.

I’ll give the older image a try and see if it’s any more successful.

Update: just flashed emonSD-07Nov16 and it worked first time

Chad · 25 November 2017 12:03

As a brand new user, I decided to flash a card while I wait for my hardware.
I too couldn’t use the Oct image. I spent about half a day trying to get it working.

I eventually realised there was a Nov build, and it worked the first time.

Exangel · 25 November 2017 12:03

Hello,

It’s the third time I’m trying to install from scratch emoncms into my raspberry 3 without any success.

I’ve got an error of systemd-fsck {250} at the startup and a number increasing without stopping; impossible to go further.

I’ve been able to install the previous version without any issue.
I’ve tried 2 times to download the .img without any success with this release.

I’m the only one facing this issue ?

Thanks.

pb66 · 25 November 2017 15:08

Sorry you guys are having trouble installing the latest image, I have notified @glyn.hudson (he maintains the emonSD images) you are having issues, but haven’t heard back yet, but I’m sure he will take a look at it asap.

glyn.hudson · 27 November 2017 18:06

Hi Guys,

I’ve managed to test today. I downloaded the Oct17 image from the download link and checked the MD5, it matches the md5 on the download page.

I flashed the image to a couple of 8GB Kingston sd card using dd and Etcher (…very impressed with Etcher, very slick indeed).

I tested both SD card in a RasPi 3, both booted up fine. I’m sorry I have been unable to replicate the issue

The Oct17 image on the download link certainly works…at least for me! We used the same image in the factory for shop purchase assembled emonBase /emonPi

Any ideas how we can try and debug this further, it’s very strange that a number of users have reported issues.

Exangel · 27 November 2017 21:27

Hi,

I’ve done it once again.
The checksum of the img seems to be correct. I use this time Etcher instead of dd command and I still get the same error at the end.

Could it be because of the fash disk itself ? I have also a sandisk ultra 16Go.

no other clue …

rbrt · 27 November 2017 22:10

I’ve only been testing on SanDisk cards so far, I think it’s probably worth trying a different brand to eliminate that as a possible cause. Unfortunately I only have SanDisk cards here at home but might be able to find something different at work tomorrow to try.

Nolan_Garrett · 27 November 2017 22:28

I’ve tested with a 4GB Transcend card (the one that came with the original emonPi) and a 16GB Kingston card. Both fail to successfully boot. I am going to pull down the image, md5 it again, then image it to the 4GB Transcend to test once more.

EDIT: Tried again, same problem. Used Etcher and also validated the MD5, which was correct. Also, I can apparently no longer add replies to this topic since I’ve hit a limit of 3 replies for new users. I guess I will just keep editing this post?

@glyn.hudson what do you see for output of fdisk -l on the Oct17 image? Mine still shows this, which I really think has to indicate the problem:

Disk emonSD-26Oct17.img: 3.6 GiB, 3838837248 bytes, 7497729 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0003ea3e

Device Boot Start End Sectors Size Id Type
emonSD-26Oct17.img1 * 8192 131071 122880 60M 6 FAT16
emonSD-26Oct17.img2 131072 7288831 7157760 3.4G 83 Linux
emonSD-26Oct17.img3 7288832 14786559 7497728 3.6G 83 Linux

The fact that the third partition shows a size much larger than the expected 4GB SD card size seems like an issue worth investigating?

Robert.Wall · 27 November 2017 23:24

Hopefully that’s fixed - I’ve raised your Trust by one level.

Nolan_Garrett · 27 November 2017 23:36

That seems to have worked! It made me do a password reset and activation (I’d been using the “Sign in With Google” option), but it’s all good. Thank you!

glyn.hudson · 28 November 2017 17:19

Ok, I think I’ve managed to find and solve the issue. The issue was that there was a lot of free space at end of the third partition included in the image file. This was no problem when flashing the image back onto the same model of SD card that the image was created on (as I was doing) since the image fit perfectly. However if using a different brand of SD card the actual size of an 8GB card can vary causing the image not to boot correctly.

This Gparted screen shots illustrate the issue:

I’ve trimmed the third partition to be 1GB, this should allow the image to easily fit on any 8GB SD card and reduce the size.

Once the RasPi has booted image it’s possible to run $ sudo emonSDexpand to expand the third /data partition to fill the available SD card size. The third data partition is the RW partition (mounted in /home/pi/data) where Emoncms logging data is stored.

~~I’ve uploaded a the cut down version of emonSD-26Oct to test: slim-emonSD-26Oct.zip.~~

Update: This ‘sim’ image is now the standard image, please download using the normal URL: emonSD pre built SD card Download & Change Log · openenergymonitor/emonpi Wiki · GitHub

Please download the new and report experience on the thread. If all goes well this version will replace the standard emonSD-26Oct download.

md5sum:

slim-emonSD-26Oct17.img: 88f8ff9a5f7bc0e9b07012895a5cdd95
slim-emonSD-26Oct17.img.zip: 6726564f379d0127052e8c30a3ffa534

This post explains the issue well: How to trim disk images to partition size | Linux M0nk3ys

Nolan_Garrett · 28 November 2017 18:40

Awesome, I am going to try this in a few moments! I am assuming from this note above that the new standard minimum emonSD storage size will be 8GB, up from the 4GB minimum.

Nolan_Garrett · 28 November 2017 19:16

Looks like I can confirm this boots! I’ll mess with it later today and ensure the whole image is working, but the boot was successful, so I think we’ve found the root cause of that problem. Thank you, @glyn.hudson for your work on this! I’ll post back if I find any other problems as I am setting it up.

Exangel · 28 November 2017 21:28

It seems to work also on my side. It’s booting right now.
It should be fine then.
Thanks.