emonSD-17Oct19 (original beta testing thread)

emonSD
1

Following on from the beta release of the EmonScripts automated emoncms build script. This image is the first pre built emonSD image release that uses the scripts for the full image build. The image is currently in Beta @glyn.hudson and I will be testing this image on our own home systems over the next couple of weeks to see if we can spot any issues. If you would like to help us please feel free to download this image and let us know how you get on. Special thanks to @borpin for all his help and guidance throughout the development process.

Feel free to build the image from scratch rather than download the image. This can be done using the automated build script, see: GitHub - openenergymonitor/EmonScripts: Emoncms Stack Installation and Update scripts

emonSD-17Oct19 (Beta - currently in testing)

Download (1.1 GB)

(.img) MD5: a7d12ac6b589ae0d470c4a6f1ce38414
(.zip) MD5: 52ecf81c2ad4afbd9da42a6e703b5c59
  • Built using EmonScripts emoncms installation script, see
    https://github.com/openenergymonitor/EmonScripts.
  • Based on Raspbian Buster 2019-09-26-raspbian-buster-lite.zip
  • Compatible with Raspberry Pi 3, 3B+ & 4
  • Emoncms data is logged to low-write ext2 partition mounted in /var/opt/emoncms
  • Log partition /var/log mounted as tmpfs using log2ram, now presistant after reboot
  • SSH access disabled by default, long press emonPi LCD push button for 5s to enable. Or create file /boot/ssh in FAT partition.

Kernel

$ uname -a
Linux emonpi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

$ sudo /opt/vc/bin/vcgencmd version
Sep 24 2019 17:37:47 
Copyright (c) 2012 Broadcom
version 6820edeee4ef3891b95fc01cf02a7abd7ca52f17 (clean) (release) (start_cd)

File System

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       4.0G  1.9G  2.0G  49% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M  6.6M  482M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgroup
tmpfs           1.0M  4.0K 1020K   1% /var/lib/php/sessions
tmpfs           1.0M     0  1.0M   0% /var/tmp
tmpfs            30M   16K   30M   1% /tmp
/dev/mmcblk0p3   10G  5.3M  9.5G   1% /var/opt/emoncms
/dev/mmcblk0p1  253M   52M  201M  21% /boot
log2ram          50M  2.1M   48M   5% /var/log
tmpfs            98M     0   98M   0% /run/user/1000

Emoncms

Server Information
-----------------------

Emoncms
	Version :	 low-write 10.1.9
	Modules :	 Administration | App v2.0.7 | Backup v2.1.4 | EmonHub Config v2.0.4 | Dashboard v2.0.5 | DemandShaper v1.0.2 | Device v2.0.2 | EventProcesses | Feed | Graph v2.0.5 | Input | Postprocess v2.1.1 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
	Git :	 
		URL :	 https://github.com/emoncms/emoncms.git
		Branch :	 * stable
		Describe :	 10.1.9

Server
	OS :	 Linux 4.19.75-v7+
	Host :	 emonpi | emonpi | (192.168.0.109)
	Date :	 2019-10-17 13:10:53 BST
	Uptime :	 13:10:53 up 15 min,  1 user,  load average: 0.10, 0.11, 0.09

Memory
	RAM :	 Used: 19.37%
		Total :	 975.62 MB
		Used :	 188.99 MB
		Free :	 786.63 MB
	Swap :	 Used: 0.00%
		Total :	 100 MB
		Used :	 0 B
		Free :	 100 MB

Disk
	/ :	 Used: 46.42%
		Total :	 3.92 GB
		Used :	 1.82 GB
		Free :	 1.91 GB
		Write Load :	 n/a
	/var/opt/emoncms :	 Used: 0.05%
		Total :	 9.98 GB
		Used :	 5.27 MB
		Free :	 9.47 GB
		Write Load :	 n/a
	/boot :	 Used: 20.55%
		Total :	 252.05 MB
		Used :	 51.79 MB
		Free :	 200.26 MB
		Write Load :	 n/a
	/var/log :	 Used: 4.20%
		Total :	 50 MB
		Used :	 2.1 MB
		Free :	 47.9 MB
		Write Load :	 n/a

HTTP
	Server :	 Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL
	Version :	 5.5.5-10.3.17-MariaDB-0+deb10u1
	Host :	 localhost:6379 (127.0.0.1)
	Date :	 2019-10-17 13:10:52 (UTC 01:00‌​)
	Stats :	 Uptime: 899  Threads: 14  Questions: 1757  Slow queries: 0  Opens: 70  Flush tables: 1  Open tables: 36  Queries per second avg: 1.954

Redis
	Version :	 
		Redis Server :	 5.0.3
		PHP Redis :	 5.0.2
	Host :	 localhost:6379
	Size :	 114 keys (810.42K)
	Uptime :	 0 days
MQTT Server
	Version :	 Mosquitto 1.5.7
	Host :	 localhost:1883 (127.0.0.1)

PHP
	Version :	 7.3.9-1~deb10u1 (Zend Version 3.3.9)
	Modules :	 apache2handler | calendar v7.3.9-1~deb10u1 | Core v7.3.9-1~deb10u1 | ctype v7.3.9-1~deb10u1 | curl v7.3.9-1~deb10u1 | date v7.3.9-1~deb10u1 | dom v20031129 | exif v7.3.9-1~deb10u1 | fileinfo v7.3.9-1~deb10u1 | filter v7.3.9-1~deb10u1 | ftp v7.3.9-1~deb10u1 | gd v7.3.9-1~deb10u1 | gettext v7.3.9-1~deb10u1 | hash v7.3.9-1~deb10u1 | iconv v7.3.9-1~deb10u1 | json v1.7.0 | libxml v7.3.9-1~deb10u1 | mbstring v7.3.9-1~deb10u1 | mosquitto v0.4.0 | mysqli v7.3.9-1~deb10u1 | mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $ | openssl v7.3.9-1~deb10u1 | pcre v7.3.9-1~deb10u1 | PDO v7.3.9-1~deb10u1 | pdo_mysql v7.3.9-1~deb10u1 | Phar v7.3.9-1~deb10u1 | posix v7.3.9-1~deb10u1 | readline v7.3.9-1~deb10u1 | redis v5.0.2 | Reflection v7.3.9-1~deb10u1 | session v7.3.9-1~deb10u1 | shmop v7.3.9-1~deb10u1 | SimpleXML v7.3.9-1~deb10u1 | sockets v7.3.9-1~deb10u1 | sodium v7.3.9-1~deb10u1 | SPL v7.3.9-1~deb10u1 | standard v7.3.9-1~deb10u1 | sysvmsg v7.3.9-1~deb10u1 | sysvsem v7.3.9-1~deb10u1 | sysvshm v7.3.9-1~deb10u1 | tokenizer v7.3.9-1~deb10u1 | wddx v7.3.9-1~deb10u1 | xml v7.3.9-1~deb10u1 | xmlreader v7.3.9-1~deb10u1 | xmlwriter v7.3.9-1~deb10u1 | xsl v7.3.9-1~deb10u1 | Zend OPcache v7.3.9-1~deb10u1 | zlib v7.3.9-1~deb10u1

Pi
	Model :	 Raspberry Pi 3 Model B+ Rev 1.3 - 1GB (Sony UK)
	Serial num. :	 78A9D9F
	Temperature :	 48.31°C - 47.8°C
	emonpiRelease :	 emonSD-17Oct19
	File-system :	 read-write
2 Likes
3

@glyn.hudson @TrystanLea could you test and see if the logrotate is working please? I’ve only checked on my Ubuntu setup so far but the permissions on the additional conf files are not correct on that.

4

yes looks like its working here, but there’s an issue with the emonPiLCD looking for file /home/pi/data/emonbase…

image
image1251×307 76 KB

1 Like
5

Not sure why the emonPiLCD script got that far on my system, there is a earlier check to see if the system has an LCD connected that it somehow got past even though I’ve no LCD present. After restarting the script it stopped in the right place. I’ve added a directory check as a second check just in case: check if dir exists before trying to create file · openenergymonitor/emonpi@c3ad4a5 · GitHub

6

Discovered another issue, not sure how I missed this one last time, there is excessive write load on the OS partition relating to writing and updating php session files when a browser window is open with an active emoncms session running (e.g looking at the feed list). Placing the /var/lib/php/sessions directory in a tmpfs solves the issue, see:
Write load from php sessions · Issue #50 · openenergymonitor/EmonScripts · GitHub

It’s an easy update to /etc/fstab if anyone is testing with this image already:

sudo nano /etc/fstab

add line at the bottom of the file:

tmpfs           /var/lib/php/sessions  tmpfs   nodev,nosuid,size=1M,mode=1777  0  0

then reboot.

7

Write load testing

I’ve added my write load testing script to emoncms/usefulscripts
usefulscripts/writeloadtest at master · emoncms/usefulscripts · GitHub

to install:

cd /opt/emoncms/modules/usefulscripts
git checkout master
git pull

add apikey to writeloadtest.php:

nano usefulscripts/writeloadtest/writeloadtest.php

install service:

sudo ln -s /opt/emoncms/modules/usefulscripts/writeloadtest/writeloadtest.service /lib/systemd/system
sudo systemctl enable writeloadtest.service
sudo systemctl start writeloadtest.service

the write load stats then show up in emoncms inputs:

image
image1588×321 20.3 KB

log them all to feeds at 60s interval so that you can keep an eye on write load over time, e.g:

image
image698×355 17.4 KB

the high write load in the above screenshot was caused by the php session file updates, over night I did not have a browser window open, hence lower write load.

8

Worth switching off the logrotate for a while and trawling the system logs for things like this.

1 Like
9

5 posts were split to a new topic: Disabling emonhub, releasing serial port

10

I have updated the emonSD-02Oct19 image to include the fstab modification, fixing the php session write load issue.

1 Like
11

I’ve added a new feature to make write load testing easier. The write load is now shown in the emoncms admin page:

image
image1399×762 58.4 KB

The period over which the write load is recorded can be reset by clicking on the Write Load Period: Reset button.

This is currently in the emoncms master branch, I will be merging to stable soon.

1 Like
12

@glyn.hudson and I have been working on fixing a couple of issues with the emonPiLCD script and emonPi LCD button operation on the new image.

We’ve also fixed the issue resulting from the new VirtualHost configuration where hostname/emoncms/feed/list is now hostname/feed/list by adding a htaccess redirect so that old dashboards and API’s will automatically redirect to the new locations.

We will be doing a fresh image build this week with latest changes and then repeating another set of tests - before we can hopefully launch this image properly.

2 Likes
13

Did that include making it Python 3 compatible (as 2 is dying very soon)?

I’ll get my hat…

14

The latest image build is now available: emonSD-17Oct19. See details in first post above.

We are in the process of repeating our testing to ensure that its all working, looking good so far.

3 Likes
15

Was this from the Stable EmonScripts branch?

16

yes, latest stable for all repositories

17

once the image is considered stable, let me know and I’ll copy it on my box as alternative dl site.

1 Like
18

thanks @bidouilleur! much appreciated

19

I meant the EmonScript branch used?

20

yes also stable branch v1.0.1-beta

1 Like