emonSD first boot

I just downloaded the latest Oct18 image and stuck it in a Pi 3, Not something I do often so here’s a couple of observations/questions

When I first navigated to http://emonpi I got this screen

Since I am already connected via wifi as I put a copy of wpa_supplicant.conf in the boot partition before putting the sdcard in in the Pi I don’t need either of those options. Could we have a “Skip” option for users that use the same method as I used, plus some users will opt for Ethernet cable rather than wifi.

Clicking log in takes me to the registration screen.

Why is it essential I provide an email address for a device that has no email server or client installed? I understand why the emoncms.org server insists, but this is not really required here. IIRC it’s just a setting in settings.php, so why is it not off by default for the emonSD?

image

Mmmmh! So why am I not packing the latest emoncms?

when I download the update log from the admin page I get a file containing one line which is “/home/pi/data/emonpiupdate.log does not exist!” so why did first boot not run?

So I’m rebooting now, maybe “firstboot” will run on the “Second boot”? I’ll keep you posted!

image

Aha! that looks better,

Oh! that not so much!

Lets look at the update log, I’ll attach it here

Ok, it’s gonna be one of those days! It is a .txt file !!!

Here’s the log file content, it raises a few questions.

Starting emonPi Update >
via service-runner-update.sh
Service Runner update script V1.1.1
EUID: 1000
Argument: 
Sun 17 Mar 19:08:58 UTC 2019
#############################################################

emonSD version: emonSD-30Oct18

emonSD base image check passed...continue update

#############################################################
I2C LCD NOT DETECTED on either 0x['27', '3f'] ...exiting LCD script
/home/pi/emonpi/service-runner-update.sh: line 46: rpi-rw: command not found
git pull /home/pi/emonpi
  enable-ssh
  enable-ssh-button
  gpiozero-enable-ssh
  greeebs-patch-1
* master
  python-systemd-servicerunner
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
fatal: unable to access 'https://github.com/openenergymonitor/emonpi.git/': gnutls_handshake() failed: Error in the pull function.
git pull /home/pi/RFM2Pi
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
Already up-to-date.
git pull /home/pi/emonhub
* emon-pi
On branch emon-pi
Your branch is up-to-date with 'origin/emon-pi'.
nothing to commit, working tree clean
From https://github.com/openenergymonitor/emonhub
   899e526..1fde439  emon-pi                -> origin/emon-pi
 * [new branch]      redis_config_interface -> origin/redis_config_interface
 * [new branch]      unitless_decoder       -> origin/unitless_decoder
 * [new tag]         v2.1.2                 -> v2.1.2
Updating 899e526..1fde439
Fast-forward
 README.md                                          |   8 +-
 conf/emonhub.conf                                  |   6 +-
 conf/emonpi.default.emonhub.conf                   |   2 +-
 .../modbus/modbusTCP.emonhub.conf                  | 162 +--------------
 conf/interfacer_examples/modbus/readme.md          |  43 +---
 configuration.md                                   |  43 +++-
 install                                            |  12 +-
 install.systemd                                    |   4 +-
 src/emonhub.py                                     |   4 +-
 src/interfacers/EmonHubJeeInterfacer.py            |  44 +++-
 src/interfacers/EmonHubTx3eInterfacer.py           |  13 +-
 src/interfacers/EmonModbusTcpInterfacer.py         | 224 +++++++++++++++++++++
 src/interfacers/__init__.py                        |   2 +-
 src/interfacers/tmp/EmonModbusTcpInterfacer.py     | 147 --------------
 14 files changed, 340 insertions(+), 374 deletions(-)
 create mode 100644 src/interfacers/EmonModbusTcpInterfacer.py
 delete mode 100644 src/interfacers/tmp/EmonModbusTcpInterfacer.py
git pull /home/pi/usefulscripts
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
From https://github.com/emoncms/usefulscripts
   7f1ac2a..5221f56  master     -> origin/master
Updating 7f1ac2a..5221f56
Fast-forward
 sdpart/sdpart_imagefile | 8 ++++----
 update_emoncms.sh       | 2 ++
 2 files changed, 6 insertions(+), 4 deletions(-)
git pull /home/pi/huawei-hilink-status
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
Already up-to-date.

Start emonPi Atmega328 firmware update:

=================================
EmonPi update started
=================================

EUID: 1000

Requirement already up-to-date: paho-mqtt in /usr/local/lib/python2.7/dist-packages
Get:1 http://archive.raspberrypi.org/debian stretch InRelease [25.4 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15.0 kB]
Get:3 http://archive.raspberrypi.org/debian stretch/main armhf Packages [214 kB]
Get:4 http://archive.raspberrypi.org/debian stretch/ui armhf Packages [44.4 kB]
Get:5 http://raspbian.raspberrypi.org/raspbian stretch/main armhf Packages [11.7 MB]
Get:6 http://raspbian.raspberrypi.org/raspbian stretch/non-free armhf Packages [95.5 kB]
Fetched 12.1 MB in 13s (862 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  python-colorzero
Suggested packages:
  python-gpiozero-docs
The following NEW packages will be installed:
  python-colorzero
The following packages will be upgraded:
  python-gpiozero python-rpi.gpio
2 upgraded, 1 newly installed, 0 to remove and 103 not upgraded.
Need to get 133 kB of archives.
After this operation, 261 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.
Stopping OpenEnergyMonitor emonHub: emonhub has been stopped ok.
Start ATmega328 serial upload using avrdude with latest.hex
Discrete Sampling
avrdude -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/home/pi/emonpi/firmware/compiled/latest.hex
avrdude-original: Using autoreset DTR on GPIO Pin 7
avrdude-original: stk500_getsync() attempt 1 of 10: not in sync: resp=0xfc
avrdude-original: stk500_getsync() attempt 2 of 10: not in sync: resp=0xe0
avrdude-original: stk500_getsync() attempt 3 of 10: not in sync: resp=0x8e
avrdude-original: stk500_getsync() attempt 4 of 10: not in sync: resp=0x1c
avrdude-original: stk500_getsync() attempt 5 of 10: not in sync: resp=0x8e
avrdude-original: stk500_getsync() attempt 6 of 10: not in sync: resp=0xe0
avrdude-original: stk500_getsync() attempt 7 of 10: not in sync: resp=0x0e
avrdude-original: stk500_getsync() attempt 8 of 10: not in sync: resp=0xf8
avrdude-original: stk500_getsync() attempt 9 of 10: not in sync: resp=0xe0
avrdude-original: stk500_getsync() attempt 10 of 10: not in sync: resp=0xe0

avrdude-original done.  Thank you.

strace: |autoreset: Broken pipe
Starting OpenEnergyMonitor emonHub: emonhub has been started ok.


Start emonhub update script:

=================================
EmonPi update started
=================================
Running emonhub automatic node addition script
EUID: 1000
EUID: 1000
[[5]]
Node 5 already present
[[6]]
Node 6 already present
[[7]]
Node 7 already present
[[8]]
Node 8 already present
[[9]]
Node 9 already present
[[10]]
Node 10 already present
[[11]]
Node 11 already present
[[12]]
Node 12 already present
[[13]]
Node 13 already present
[[14]]
Node 14 already present
[[19]]
Node 19 already present
[[20]]
Node 20 already present
[[21]]
Node 21 already present
[[22]]
Node 22 already present
[[23]]
Node 23 already present
[[24]]
Node 24 already present
[[25]]
Node 25 already present
[[26]]
Node 26 already present

Start emoncms update:

=================================
Emoncms update started
Emoncms update script V1.2.0

Sun 17 Mar 19:10:39 UTC 2019

#############################################################

emonSD version: emonSD-30Oct18

EUID: 1000

current settings.php md5: bfcadb4356440ffc96c06a5d91aa2a02
Default settings.php md5: bfcadb4356440ffc96c06a5d91aa2a02

git pull /var/www/emoncms
  emrysr-fix/issue-1078-scolling-horizontal-mobile-devices-v1
  emrysr-fix/widen-unit-dropdown-width-in-feed-view
  master
* stable
  systmdRunner
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/emoncms
   e520645b..d0db7a57  stable       -> origin/stable
 * [new branch]        data_checker -> origin/data_checker
   6cc7ffd5..68888047  hierarchical_settings_php -> origin/hierarchical_settings_php
   215557a3..90955023  master       -> origin/master
   730c62b4..97d5316e  mqttauth     -> origin/mqttauth
 * [new tag]           9.9.3        -> 9.9.3
 * [new tag]           9.9.4        -> 9.9.4
 * [new tag]           9.9.5        -> 9.9.5
 * [new tag]           9.9.6        -> 9.9.6
 * [new tag]           9.9.7        -> 9.9.7
 * [new tag]           9.9.8        -> 9.9.8
Updating e520645b..d0db7a57
Fast-forward
 .travis.yml                                        |    6 +-
 COPYRIGHT.txt                                      |    2 +-
 Lib/misc/autocomplete.css                          |   34 +
 Lib/misc/autocomplete.js                           |   86 ++
 Lib/misc/sidebar.css                               |    3 +-
 Lib/misc/sidebar.js                                |   12 +-
 Lib/responsive-linked-tables.js                    |   25 +-
 Lib/tablejs/custom-table-fields.js                 |   27 +-
 Lib/units.php                                      |    3 +-
 Lib/user_locale.js                                 |   28 +
 Modules/admin/admin_controller.php                 |    8 +-
 Modules/admin/admin_main_view.php                  |  125 +-
 Modules/admin/locale/fr_FR/LC_MESSAGES/messages.mo |  Bin 3386 -> 3981 bytes
 Modules/admin/locale/fr_FR/LC_MESSAGES/messages.po |  120 +-
 Modules/admin/pi-model.json                        |   38 +-
 Modules/admin/userlist_view.php                    |   30 +-
 Modules/feed/Views/feedlist_view_v2.php            | 1229 ++++++++++----------
 Modules/feed/engine/CassandraEngine.php            |   11 +-
 Modules/feed/engine/MysqlTimeSeries.php            |   12 +-
 Modules/feed/engine/PHPFina.php                    |  200 +++-
 Modules/feed/engine/PHPFiwa.php                    |    7 +-
 Modules/feed/engine/PHPTimeSeries.php              |    9 +-
 Modules/feed/engine/RedisBuffer.php                |   10 +-
 Modules/feed/engine/TemplateEngine.php             |   10 +-
 Modules/feed/feed_controller.php                   |  196 ++--
 Modules/feed/feed_model.php                        |    9 +-
 Modules/feed/locale/fr_FR/LC_MESSAGES/messages.mo  |  Bin 12939 -> 14507 bytes
 Modules/feed/locale/fr_FR/LC_MESSAGES/messages.po  |  370 +++---
 Modules/input/Views/device_view.php                |  603 ++++++----
 Modules/input/input_controller.php                 |  188 +--
 Modules/input/input_methods.php                    |   13 +-
 Modules/input/locale/fr_FR/LC_MESSAGES/messages.mo |  Bin 14290 -> 15289 bytes
 Modules/input/locale/fr_FR/LC_MESSAGES/messages.po |  167 ++-
 Modules/process/Views/process_ui.js                |   16 +-
 Modules/process/Views/process_ui.php               |   65 +-
 Modules/user/locale/cy_GB/LC_MESSAGES/messages.mo  |  Bin 3015 -> 4653 bytes
 Modules/user/locale/cy_GB/LC_MESSAGES/messages.po  |  441 +++++--
 Modules/user/profile/profile.php                   |    5 +-
 Modules/vis/Views/vis_main_view.php                |    4 +-
 Modules/vis/visualisations/bargraph.php            |   17 +-
 Modules/vis/visualisations/editdaily.php           |    2 +-
 Modules/vis/visualisations/editrealtime.php        |    3 +-
 .../vis/visualisations/multigraph/multigraph.js    |    1 +
 Modules/vis/visualisations/smoothie.php            |   12 +-
 Theme/basic/emon-blue.css                          |  148 ++-
 Theme/basic/emon-standard.css                      |  154 ++-
 Theme/basic/emon-sun.css                           |  144 ++-
 Theme/basic/menu_view.php                          |  247 ++--
 Theme/basic/theme.php                              |  440 +++----
 default.emonpi.settings.php                        |    3 +-
 default.settings.php                               |    4 +-
 docs/LinuxInstall.md                               |   13 +-
 docs/RaspberryPi/MQTT.md                           |   26 +-
 docs/RaspberryPi/read-only.md                      |   12 +-
 docs/Upgrading.md                                  |    2 +-
 index.php                                          |   53 +-
 process_settings.php                               |    1 +
 readme.md                                          |   22 +-
 scripts/feedwriter.php                             |    2 +-
 scripts/logger/install.sh                          |   22 +-
 scripts/mqtt_input.service                         |    2 +-
 scripts/phpmqtt_input.php                          |   15 +-
 scripts/services/emoncms_mqtt/emoncms_mqtt.php     |  385 ++++++
 scripts/services/emoncms_mqtt/emoncms_mqtt.service |   61 +
 scripts/services/feedwriter/feedwriter.service     |   61 +
 scripts/services/install-service-emoncmsmqtt.md    |   23 +
 scripts/services/install-service-feedwriter.md     |   24 +
 scripts/services/install-service-runner-update.md  |   34 +-
 scripts/services/service-runner/service-runner.py  |   56 +-
 .../services/service-runner/service-runner.service |   11 +-
 version.txt                                        |    2 +-
 71 files changed, 3880 insertions(+), 2234 deletions(-)
 create mode 100644 Lib/misc/autocomplete.css
 create mode 100644 Lib/misc/autocomplete.js
 create mode 100644 Lib/user_locale.js
 create mode 100644 scripts/services/emoncms_mqtt/emoncms_mqtt.php
 create mode 100644 scripts/services/emoncms_mqtt/emoncms_mqtt.service
 create mode 100644 scripts/services/feedwriter/feedwriter.service
 create mode 100644 scripts/services/install-service-emoncmsmqtt.md
 create mode 100644 scripts/services/install-service-feedwriter.md

NEW default settings.php md5: 468b110786bf0ab60a856fa3c0036f0f
settings.php has NOT been user modifed
Update required to settings.php...
settings.php autoupdated


git pull /var/www/emoncms/Modules/app
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/app
   3fbdf07..8edcc30  stable     -> origin/stable
   0692abe..f3a980e  master     -> origin/master
 * [new tag]         1.2.1      -> 1.2.1
Updating 3fbdf07..8edcc30
Fast-forward
 Lib/config.js                                      |  2 +-
 Lib/feed.js                                        | 13 +++-
 Views/app_view.php                                 |  2 +-
 Views/css/config.css                               | 13 ++++
 app_model.php                                      | 16 +++-
 apps/OpenEnergyMonitor/costcomparison/rates.js     | 86 ++++++++++++++++++++++
 apps/OpenEnergyMonitor/myelectric/myelectric.php   | 33 ++++++---
 apps/OpenEnergyMonitor/myelectric2/myelectric2.php | 30 ++++++--
 apps/OpenEnergyMonitor/myenergy/myenergy.php       |  1 +
 apps/OpenEnergyMonitor/myheatpump/myheatpump.php   |  1 +
 apps/OpenEnergyMonitor/mysolarpv/mysolarpv.php     | 29 ++++++--
 .../mysolarpvdivert/mysolarpvdivert.php            |  1 +
 apps/OpenEnergyMonitor/timeofuse/timeofuse.php     |  1 +
 apps/OpenEnergyMonitor/timeofuse2/timeofuse2.php   |  2 +
 apps/OpenEnergyMonitor/timeofusecl/timeofusecl.php |  2 +
 module.json                                        |  2 +-
 16 files changed, 201 insertions(+), 33 deletions(-)
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/config
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/config
   486d4d3..5e96633  stable     -> origin/stable
 * [new branch]      editor     -> origin/editor
   f38b465..52b2f17  master     -> origin/master
 * [new tag]         1.1.0      -> 1.1.0
Updating 486d4d3..5e96633
Fast-forward
 config_controller.php | 31 ++++++++++++++++---------------
 module.json           |  2 +-
 2 files changed, 17 insertions(+), 16 deletions(-)
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/wifi
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/wifi
   16a2296..2f37618  stable     -> origin/stable
   16a2296..bbf4912  master     -> origin/master
 * [new tag]         1.3.1      -> 1.3.1
Updating 16a2296..2f37618
Fast-forward
 module.json | 2 +-
 wifi.php    | 8 +++++++-
 2 files changed, 8 insertions(+), 2 deletions(-)
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.
git pull /var/www/emoncms/Modules/dashboard
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/dashboard
   9b03e57..8be171b  stable     -> origin/stable
   c1ae9f6..0309ef7  master     -> origin/master
 * [new tag]         1.3.2      -> 1.3.2
 * [new tag]         1.3.3      -> 1.3.3
Updating 9b03e57..8be171b
Fast-forward
 Readme.md                                   |    2 +
 Views/dashboard_edit_view.php               |    4 +-
 Views/dashboard_list.php                    |    4 +-
 Views/dashboard_view.php                    |    1 +
 Views/js/designer.js                        |   76 +-
 Views/loadwidgets.php                       |    2 +-
 dashboard_controller.php                    |    7 +-
 dashboard_langjs.php                        |   14 +-
 locale/en_GB/LC_MESSAGES/messages.mo        |  Bin 518 -> 512 bytes
 locale/en_GB/LC_MESSAGES/messages.po        | 1015 ++++++++++++++++++++++-----
 locale/fr_FR/LC_MESSAGES/messages.mo        |  Bin 14689 -> 15636 bytes
 locale/fr_FR/LC_MESSAGES/messages.po        |  435 ++++++------
 module.json                                 |    2 +-
 widget/bar/bar_render.js                    |   45 +-
 widget/bar/bar_widget.php                   |   15 -
 widget/battery/battery_render.js            |   60 +-
 widget/button/button_render.js              |   17 +-
 widget/dial/dial_render.js                  |   46 +-
 widget/dial/dial_widget.php                 |   15 -
 widget/feedtime/feedtime_render.js          |   21 +-
 widget/feedtime/feedtime_widget.php         |   15 -
 widget/feedvalue/feedvalue_render.js        |   14 +-
 widget/feedvalue/feedvalue_widget.php       |   15 -
 widget/isactivefeed/isactivefeed_widget.php |   15 -
 widget/jgauge/jgauge_render.js              |   41 +-
 widget/jgauge2/jgauge2_render.js            |   42 +-
 widget/signal/signal_render.js              |   59 +-
 widget/sun/sun_render.js                    |   57 +-
 widget/thermometer/thermometer_render.js    |   60 +-
 widget/thresholds/thresholds_render.js      |   18 +-
 30 files changed, 1505 insertions(+), 612 deletions(-)
 delete mode 100644 widget/bar/bar_widget.php
 delete mode 100644 widget/dial/dial_widget.php
 delete mode 100644 widget/feedtime/feedtime_widget.php
 delete mode 100644 widget/feedvalue/feedvalue_widget.php
 delete mode 100644 widget/isactivefeed/isactivefeed_widget.php
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/graph
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/graph
   7a2fc18..91d4db5  stable     -> origin/stable
   f4dabbc..8ae4e7d  master     -> origin/master
 * [new branch]      revert-43-feature/issue-40-multi-id-data-fetch -> origin/revert-43-feature/issue-40-multi-id-data-fetch
 * [new tag]         1.2.3      -> 1.2.3
 * [new tag]         1.2.2      -> 1.2.2
Updating 7a2fc18..91d4db5
Fast-forward
 Lib/timezone-js                      |   1 +
 embed.php                            |  13 +-
 graph.css                            |  94 ++++++++--
 graph.js                             | 336 ++++++++++++++++++++++++++---------
 group_view.php                       |   4 +-
 locale/cy_GB/LC_MESSAGES/messages.mo | Bin 0 -> 3539 bytes
 locale/cy_GB/LC_MESSAGES/messages.po | 326 +++++++++++++++++++++++++++++++++
 locale/fr_FR/LC_MESSAGES/message.po  | 327 ++++++++++++++++++++++++++++++++++
 locale/fr_FR/LC_MESSAGES/messages.mo | Bin 0 -> 3714 bytes
 module.json                          |   2 +-
 readme.md                            |   2 +-
 view.php                             | 303 +++++++++++++++++++------------
 12 files changed, 1190 insertions(+), 218 deletions(-)
 create mode 160000 Lib/timezone-js
 create mode 100644 locale/cy_GB/LC_MESSAGES/messages.mo
 create mode 100644 locale/cy_GB/LC_MESSAGES/messages.po
 create mode 100644 locale/fr_FR/LC_MESSAGES/message.po
 create mode 100644 locale/fr_FR/LC_MESSAGES/messages.mo
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.

git pull /home/pi/postprocess
Already on 'emonpi'
Your branch is up-to-date with 'origin/emonpi'.
From https://github.com/emoncms/postprocess
   d392f91..e548367  emonpi     -> origin/emonpi
 * [new tag]         1.0.0      -> 1.0.0
Updating d392f91..e548367
Fast-forward
 common.php                                    |  29 +++++
 postprocess-module/postprocess_controller.php | 105 ++++++++++++++++--
 postprocess-module/view.js                    |  98 +++++++++++++++--
 postprocess-module/view.php                   |  46 +++++++-
 processes/basic_formula.php                   | 151 ++++++++++++++++++++++++++
 processes/constantflow_tokwh.php              | 107 ++++++++++++++++++
 processes/liquidorairflow_tokwh.php           | 118 ++++++++++++++++++++
 processes/powertokwh.php                      |  38 +++++--
 8 files changed, 660 insertions(+), 32 deletions(-)
 create mode 100644 processes/basic_formula.php
 create mode 100644 processes/constantflow_tokwh.php
 create mode 100644 processes/liquidorairflow_tokwh.php
git pull /home/pi/sync
Already on 'master'
Your branch is up-to-date with 'origin/master'.
From https://github.com/emoncms/sync
   30bba0a..82fb0bb  master     -> origin/master
 * [new tag]         1.0.1      -> 1.0.1
Updating 30bba0a..82fb0bb
Fast-forward
 sync-module/sync_controller.php | 4 +++-
 sync-module/sync_model.php      | 5 ++++-
 sync-module/sync_view.php       | 2 +-
 3 files changed, 8 insertions(+), 3 deletions(-)
git pull /var/www/emoncms/Modules/device
Already on 'master'
Your branch is up-to-date with 'origin/master'.
From https://github.com/emoncms/device
   a1e063b..da7fa4e  master                 -> origin/master
   db9ec31..08eea77  emoncmsorg             -> origin/emoncmsorg
 * [new branch]      remove_user_dependency -> origin/remove_user_dependency
 * [new tag]         1.2.1                  -> 1.2.1
Updating a1e063b..da7fa4e
Fast-forward
 Views/device_dialog.js                             |  4 +-
 Views/device_dialog.php                            |  2 +-
 Views/device_view.php                              | 13 +++-
 data/Control/smartplug.json                        |  3 +-
 data/{OpenEnergyMonitor => OpenEVSE}/openevse.json | 79 +++++++++++++++-------
 device_controller.php                              | 25 +++++--
 device_model.php                                   | 45 ++++++------
 device_template.php                                | 12 ++--
 module.json                                        |  2 +-
 9 files changed, 120 insertions(+), 65 deletions(-)
 rename data/{OpenEnergyMonitor => OpenEVSE}/openevse.json (57%)
git pull /home/pi/backup
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
From https://github.com/emoncms/backup
   0aa1a3b..3dcb153  stable     -> origin/stable
 * [new branch]      multienv   -> origin/multienv
 * [new tag]         1.1.6      -> 1.1.6
Updating 0aa1a3b..3dcb153
Fast-forward
 backup/module.json |  2 +-
 emoncms-copy.sh    | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 emoncms-export.sh  |  7 +++--
 readme.md          |  1 +
 4 files changed, 98 insertions(+), 3 deletions(-)
 create mode 100755 emoncms-copy.sh
Already on 'stable'
Your branch is up-to-date with 'origin/stable'.

update mqtt_input systemd unit file

Update Emoncms database
[]

Restarting Services...
Restarting OpenEnergyMonitor emonHub: emonhub has been restarted ok.
Log is turned off
Restarting feedwriter



set log rotate config owner to root
Restarting Services...


Starting emonPi LCD service..

/home/pi/emonpi/service-runner-update.sh: line 132: rpi-ro: command not found
Sun 17 Mar 19:11:15 UTC 2019


...................
emonPi update done
restarting service-runner\n
service-runner: no process found

There are a few errors, it tells me emonhub has been restarted, more than once and that the service runner process cannot be found, and yet it has been started !!!

I’m sensing this isn’t going to be a quicky, maybe I’ll comeback to it another time.

1 Like

The fundamental issue is that the EmonSD is heavily weighted towards the EmonPi hardware. This is not an incorrect position, but it causes a number of problems as a result.

I had a similar experience and was part way through writing up a comment on it but I got distracted!

Personally I think the whole structure is fatally flawed from 2 github organisations, through to some modules being ‘core’ and others being optional (but actually essential) and out of date documentation. This is the root cause of many of the issues we are seeing.

At a minimum, I think it really needs 3 images or a setup script that is done on the basis of target hardware:

  • Basic SD - core EmonCMS requirements - no RFM module
  • EmonBase
  • EmonPi

I think a setup script is the way forward rather than a pre-prepared SD Card. The exception may be 2 images specifically for an EmonPi or an EmonBase on an RPi (although this could be done via a script).

The setup page should only display if wpa_supplicant is empty of wifi configuration. I tested this last week and it did work fine so that is a bit strange. I will make a note to check again. It should also show the Ethernet option.

Sure, its there since emoncms can be used on many different target platforms with different requirements. E.g self install on remote server where user wants password recovery emails. The email requirement at register is not a setting in settings.php yet. Email verification is a setting, essentially you can enter any email in that box on emonSD as the email will not be verified.

I will double check a fresh image and see if I can replicate the issues that you saw - I’m not aware of those issues and ran the update process on a number of stock images week before last.

@TrystanLea was an RFM card installed on your test? Was it a Pi3?

Yes both RFM2Pi and Pi3 b+ Im aware the

avrdude-original: stk500_getsync() attempt 10 of 10: not in sync: resp=0xe0

errors will show, but everything else should be fine on the first boot update. I will run another test here

An Ethernet option will appear if an Ethernet cable is plugged in.

The only error I can see is rpi-rw: command not found this is nothing to worry about and has been fixed recently:

It should run on first boot provided Internet connectivity is available. This first boot updates takes some time since it’s a sizable update. It’s usually ran in the factory on shop purchased units so most users will have the latest version ready to go on first boot. If using an emonPi the LCD screen will indicate that an update is in progress.

The message is also nothing to worry about, since I imagine you are running emonSD on an RFM69Pi system, the firmware update will only work on an emonPi.

I’ve wondered this since I got my EmonBase and read about all of the confusion between the two “Update” buttons on the admin web page… Is there not a way to determine if the hardware is an EmonBase or an EmonPi (or neither) and do “the right thing” for each?

If there’s no software way to tell, perhaps a circuit modification could enable the detection? Something like a GPIO check for connection to ground on a specific pin that is different for the different add-on boards?

I’d suggest testing the image without the RFM board.

It’s necessary to choose the “wrong” one when you have a custom sketch in the front end, otherwise it gets overwritten back to the ‘default’ sketch when you update.

That works because a different baud rate is used to communicate with the RFM12[69]Pi.

The only way to detect would be to test the different baud rates.

However, updating the RFM69Pi firmware is unlikely to be needed, there has been any changes to the RFM60Pi firmware since 2016: Release V1.3.0 · openenergymonitor/RFM2Pi · GitHub. Therefore, as an emonBase user is doesn’t really matter if you run emonPi update instead of emonBase update since there is no new FW updates.

I agree, it would be good to make the microcontroller FW update a separate process from the emoncms update

1 Like

Just tested this here and it all went ok, here’s the logfile: emonpiupdate.txt (31.2 KB)

1 Like

I’m all for this. It would also reduce the risk of hitting the “wrong” update button if you had a custom sketch loaded (as @Robert.Wall posted above).

3 Likes

I agree. It’s the principal reason why I advise people against trying to calibrate/customise the front end, because it’s far too easy to lose it and have to repeat the work.

I’ve never understood why the front-end sketch has to be reloaded each time emoncms is updated - surely it should be an option, normally defaulting to ‘no’ and defaulting to ‘yes’ only when there’s an update to the front end sketch.

1 Like

I think the key thing here is the fact I put a wpa_supplicant.conf file in the boot partition before installing to the Pi.

You may well be testing with an Ethernet cable or WiFi APmage to set up wifi since the Rasbian method I used was not possible on previous emonSD images due to the RO FS.

I believe (but have not confirmed or tested further) that the wifi network might not have auto-configured early enough for the firstboot to run properly. If this is the case, then whilst I understand why it might be this way due to the previous images being RO, moving forward, the emonSD should really support this route to setting up wifi. It is by far the easiest method.

Hopefully, if the firstboot script becomes a systemd unit to get rid of rc.local, we will have more control over what happens when. the “firstboot” service can be triggered after there is a network connection, regardless of how it comes about.

That is significantly different to mine. I would have expected the first time the updater runs (regardless of being at first boot or second) to be exactly the same if the image and update script versions are the same (aside from the lcd and rfm2pi/emonpi fw which could be different for very obvious reasons).

I did cite 2 errors in the text you quoted, emonhub is reported as not running in emoncms despite “it tells me emonhub has been restarted, more than once” and at the end of the script it tells me the “service runner process cannot be found, and yet it has been started” and yet in emoncms it shows as running. Isuspect the latter is just a hangover from the recent changes to the service-runner perhaps. It is running so it is NOT the problem the log reports. But emonhub has failed to be restarted for some reason (or crashed out very soon after the update).

Then there is the 2 minor errors about rpi-rw at the start and rpi-ro towards the end, as you say nothing to worry about and easily fixed.

I am also not concerned about the fw not updating, I have no rfm2pi or emonpi board on the bare RPi at this point. That was expected, as was the lcd warning.

But if you look closer there are these errors

Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
fatal: unable to access 'https://github.com/openenergymonitor/emonpi.git/': gnutls_handshake() failed: Error in the pull function.

and

The following additional packages will be installed:
  python-colorzero
Suggested packages:
  python-gpiozero-docs
The following NEW packages will be installed:
  python-colorzero
The following packages will be upgraded:
  python-gpiozero python-rpi.gpio
2 upgraded, 1 newly installed, 0 to remove and 103 not upgraded.
Need to get 133 kB of archives.
After this operation, 261 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.

In both cases there was obviously a working network connection, both seem odd, but the point here is that this is automated, I didn’t do anything to interfere with these operations and the network was up at this time. I have just checked and the apt-get installs (https://github.com/openenergymonitor/emonpi/blob/master/emonpiupdate#L18-L19) were not done, but succeeded first time when I just manually tried it.

And the emonpi repo wasn’t up to date either.

pi@emonpi:~/emonpi $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
pi@emonpi:~/emonpi $ git describe
emonSD-22Dec2015-709-g4c1d84f
pi@emonpi:~/emonpi $ git pull
remote: Enumerating objects: 139, done.
remote: Counting objects: 100% (139/139), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 210 (delta 124), reused 121 (delta 116), pack-reused 71
Receiving objects: 100% (210/210), 58.16 KiB | 0 bytes/s, done.
Resolving deltas: 100% (135/135), completed with 24 local objects.
From https://github.com/openenergymonitor/emonpi
   4c1d84f..56ce5b2  master                  -> origin/master
   18b8859..003ad11  cydynni                 -> origin/cydynni
 * [new branch]      feature/update-tmpfiles -> origin/feature/update-tmpfiles
 * [new branch]      remove-reset-hack       -> origin/remove-reset-hack
 * [new branch]      service_correction      -> origin/service_correction
 * [new tag]         2.9.1                   -> 2.9.1
Updating 4c1d84f..56ce5b2
Fast-forward
 emoncmsupdate                | 565 +++++++++++++++++++++++++++----------------
 emonhub-sudoers              |   2 +-
 emonhubupdate                |  53 +++-
 emonpiupdate                 |   7 +-
 firmware/platformio.ini      |   1 +
 firmware/src/src.ino         |   2 +
 firmware/src/temperature.ino |  77 ++++--
 lcd/emonPiLCD.cfg            |  36 +++
 lcd/emonPiLCD.py             | 518 +++++++++++++++++++++++----------------
 rc.local_jessieminimal       |   2 +
 service-runner-update.sh     |  12 +-
 stretch/motd                 |   2 +-
 stretch/rc.local             |   3 +-
 13 files changed, 832 insertions(+), 448 deletions(-)
 create mode 100644 lcd/emonPiLCD.cfg
pi@emonpi:~/emonpi $

If the emonpi repo is not successfully updated, there really in’t much point in continuing the update process as you end up in between versions or crossed version might be a better term. A warning and abort would be better at that point if not successful.

The fact it was updating from an old version of the emonpi repo is why the update is significantly different to Trystans example.

What ever the reasons behind the fail, maybe it was network, maybe this will be better managed in systemd, either way, I am just reporting what happened when I simply booted to a new image, I had not interfered with anything other than pre-adding the wifi details as explained above.

I have just booted this up again to check a couple of things (it was powered down after posting the other night) and when I navigate to the IP of the RPi I got the setup screen again

this time there is a “24” in the top left of the screen. I have to click “log in” to get the log in box.

The wpa_supplican.conf file is present and the content is as I left it on the boot partition.

pi@emonpi:~/emonpi $ ls -la /etc/wpa_supplicant
total 52
drwxr-xr-x  2 root root  4096 Oct 30 16:59 .
drwxr-xr-x 97 root root  4096 Mar 17 19:10 ..
-rwxr-xr-x  1 root root   937 Oct 14  2017 action_wpa.sh
-rwxr-xr-x  1 root root 25619 Oct 14  2017 functions.sh
-rwxr-xr-x  1 root root  4696 Oct 14  2017 ifupdown.sh
-rw-------  1 root root   556 Aug  8  2018 wpa_supplicant.conf
pi@emonpi:~/emonpi $ 

Whilst I don’t necessarily think you are wrong, I do not think that is the issue here. Or at least not the one I was referring too. I expected the FW update to crash out ungracefully, but as others have posted here, this is actually the “officially supported method” to avoid overwriting custom FW or temporarily bricking a rfm12 based rfm2pi.

In which case why don’t we just do away with the update emonbase button to reduce the chance of screwing up an rfm12 based rfm2pi?

Since Trystan is keen to bring the methods of updating for emonsd and non-emonsd systems closer. Why do we not just change the button to “update” and in the settings.php we have a “update path” setting that is “/home/pi/emonpi/update_emonpi” by default in the emonpi.default.settings.php (and even in process_settings.php if we must) .

then users can edit that line to $update_path = /home/pi/emonpi/update_rfm69pi or $update_path = /home/pi/emonpi/update_rfm12pi if they have an emonbase, (if they click update before changing it will fail to update due to emonpi defaults and not brick thier rfm12pi) OR the rest of us can edit it to something like $update_path = /home/randomuser/myownrepo/update_just_emoncms or some official offering perhaps?

This is safer, simpler, more flexible and broadens the use of the update button to non-emonsd installs.

[edit]

eg when the emonSTM comes out, just drop a update_emonSTM file into the emonPi repo (maybe it’s time the emonpi FW was seperated from the emonSD stuff!)

Also just fior some insight as to how these multiple scripts might work, I imagine something like

#!/bib/bash

source emonsd_update.sh

avrdude -v -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 38400 -U flash:w:/home/pi/RFM2Pi/firmware/RFM69CW_RF_Demo_ATmega328/RFM12_Demo_ATmega328.cpp.hex

where emonsd_update.sh is a universal update script called from each of the various individual update scripts that can be called from emoncms.

This also accommodates optional FW installs to the emonpi for legacy or CM when it lands. eg $update_path = /home/pi/emonpi/update_emonpiCM or $update_path = /home/pi/emonpi/update_emonpiBETA etc etc

Interesting, it seemed the git pull on emonPi failed because of a gnutls_handshake() error. I’ve not seen this before. Searching the web for possible answers indicates this could be a network issue or some sort of firewall or proxy. Whenever emonSD runs the first update in the factory we have never had this error before, I will see if I can find any more info on this error.

The first boot updates checks for an Internet connection (by pinging google) before starting the update. Firstboot update will not run until an Internet connection is detected, this may not be the first boot if not connection was present initially