Questions about emoncms 10.0.2 upgrade

Hi,

Where is update log?

Missatge de Brian Orpin via Community [email protected] del dia dt., 28 de maig 2019 a les 13:45:

Admin page

# cat emonpiupdate.log
Starting emonPi Update >
via service-runner-update.sh
Service Runner update script V1.1.1
EUID: 1000
Argument: all
Sun 26 May 18:20:05 CEST 2019

Thanks @klxout is that the full content of your update log?, if you click on “Copy Log to Clipboard” it should copy the whole thing, should be several hundred lines.

Right, all log, but i think that update was executed two times to confirm that latest update was applied

It shouldnt matter how many times you run update, it should run through the whole update again.

The missing backup module in the menu sounds to me like the backup module has not updated on your system. If you are comfortable with SSH you can check its status with the following commands:

cd /home/pi/backup
git status

you could then try updating it with:

git pull 

if you have a local change it may not update and that may be our problem, could you let me know what you see?

Another option to try is to run a full update manually you can do this with the manual update steps described at the bottom of the page here: https://community.openenergymonitor.org/t/upgrading-emonsd-07nov16-to-latest-via-emonpi-emonbase-update/11051

Hi,

output

cd /home/pi/backup

root@emonpi:/home/pi/backup# git status
On branch stable
Your branch is up-to-date with ‘origin/stable’.
nothing to commit, working tree clean
root@emonpi:/home/pi/backup# git pull
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), done.
From https://github.com/emoncms/backup

  • [new branch] log_location → origin/log_location
    Already up-to-date.

seems all ok… any suggestion?

Thanks

Missatge de Trystan Lea via Community [email protected] del dia dc., 29 de maig 2019 a les 22:39:

About this url:
https://community.openenergymonitor.org/t/upgrading-emonsd-07nov16-to-latest-via-emonpi-emonbase-update/11051

with a raspberry pi2 waht option can I use?

Run the automatic update process (emonpi hardware):

/home/pi/emonpi/service-runner-update.sh all emonpi

emonbase (rfm69pi) hardware:

/home/pi/emonpi/service-runner-update.sh all rfm69pi
what?

Missatge de klxout [email protected] del dia dc., 29 de maig 2019 a les 22:57:

which hardware apart from pi2 do you have? You can actually skip the firmware upload step with the following version of the command if you dont have emonpi or rfm69pi hardware

/home/pi/emonpi/service-runner-update.sh all custom

Executed update:

# /home/pi/emonpi/service-runner-update.sh all emonpi

I have this same issue. Updated my stock emonpi to 10.0.2 and now on to 10.1 and I have no backup option. Tried running manual update, no change. The backup option is not available in the cog dropdown…

image

pi@emonpi:~/backup $ cd /home/pi/backup
pi@emonpi:~/backup $ git status
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working tree clean
pi@emonpi:~/backup $ git pull
Already up-to-date.
pi@emonpi:~/backup $

From manual update:

    ------------------------------------------
    Updating backup module
    ------------------------------------------
    - git branch: stable
    - git tags: 2.0.0-12-g67a1f1d
    - no local changes
    - running: git pull origin stable

    From https://github.com/emoncms/backup
     * branch            stable     -> FETCH_HEAD
    Already up-to-date.

I have the same issue following a Full Update. The config files for the backup need to be set up again with the correct paths, which got backup working again, but I still don’t have a link to it.

The more serious issue is that Emonhub is not starting, and it appears to be a similar issue, as the config appears to be out of date and the paths in the config files don’t match my system. I am trying to run the update again, and at the moment the log is filling up with lines saying:

Writing | avrdude-original: stk500_loadaddr(): (a) protocol error, expect=0x14, resp=0x1c

avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_cmd(): programmer is out of sync
avrdude-original: stk500_recv(): programmer is not responding
avrdude-original: stk500_recv(): programmer is not responding

I had the same lines in the first run of the update, which does eventually complete, but it looks like I might have to wipe the system and start again if it does not work this time.

After a lot of playing around, I have got Emonhub working again. The config in emonhub.service file did not match the config in my system, so the service was not starting (for example, it was not looking in /home/pi/data for emonhub.conf, which is where the Emonhub link on the web interface pulls it from). The links in the emonhub.service were:

Process: 3892 ExecStart=/usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log (code=exited, status=203/EXEC)

After all that, there is still no link to the Backup!

No emonhub issues for me. Not sure if anyone can point us in the right direction to rediscover backup however. Happy to post any and all config or logs that may lead to a clue…

@TrystanLea - something potentially not quite right with the update script.

Can you tell us what the base system is? Is it an EmonSD (the version will be in the admin screen)?

It is an Emonpi, and it was updated early this year, so the base system is now emonSD-30Oct18.

It is working normally again (apart from the missing Backup link), so the issue appears to have been the incorrect links in the emonhub.service file.

Thanks @julesb yes there has been a change to the emonhub service config but the update script should handle that update process correctly - at least it did when I last tested. Perhaps something tripped it up during the update. Is sounds like you have it sorted. If anyone else experiences this please let me know.

The backup module is installed at /home/pi/backup and the emoncms UI part of it /home/pi/backup/backup-module should be symlinked to /var/www/emoncms/Modules/backup. To update the module manually you could try:

cd /home/pi/backup
git pull
ln -sf /home/pi/backup/backup-module /var/www/emoncms/Modules/backup

I will double check the update process, this should all be handled automatically, at least it did last time I tested.

Do you by any chance have a copy of your emonpiupdate.log for the last update @julesb? Its available from the administration page.

I did an install 3 times. The first one is the one that stopped Emonhub from running, and it had similar issues with avrdude. The first log was overwritten then I attempted the update again. This got stuck on ATMega328 upating and was truncated when I shutdown the device, although I had copied the log lines before rebooting. That one is shown below.

<details><summary>LAST ENTRIES ON THE LOG FILE</summary><br />
#############################################################<br />
Starting emonPi Update ><br />
via service-runner-update.sh<br />
Service Runner update script V1.1.1<br />
EUID: 1000<br />
Argument: all<br />
Tue 18 Jun 11:34:49 UTC 2019<br />
#############################################################<br />
<br />
emonSD version: emonSD-30Oct18<br />
<br />
emonSD base image check passed...continue update<br />
<br />
#############################################################<br />
I2C LCD DETECTED Ox3f<br />
git pull /home/pi/emonpi<br />
  enable-ssh<br />
  enable-ssh-button<br />
  gpiozero-enable-ssh<br />
  greeebs-patch-1<br />
* master<br />
  python-systemd-servicerunner<br />
On branch master<br />
Your branch is behind 'origin/master' by 145 commits, and can be fast-forwarded.<br />
  (use "git pull" to update your local branch)<br />
Changes not staged for commit:<br />
  (use "git add ..." to update what will be committed)<br />
  (use "git checkout -- ..." to discard changes in working directory)<br />
<br />
	modified:   lcd/emonPiLCD.cfg<br />
<br />
no changes added to commit (use "git add" and/or "git commit -a")<br />
error: Your local changes to the following files would be overwritten by merge:<br />
	lcd/emonPiLCD.cfg<br />
Please commit your changes or stash them before you merge.<br />
Aborting<br />
Updating bc7ebd1..0a0c6ba<br />
git pull /home/pi/RFM2Pi<br />
* master<br />
On branch master<br />
Your branch is up-to-date with 'origin/master'.<br />
nothing to commit, working tree clean<br />
Already up-to-date.<br />
git pull /home/pi/emonhub<br />
* emon-pi<br />
On branch emon-pi<br />
Your branch is up-to-date with 'origin/emon-pi'.<br />
Changes not staged for commit:<br />
  (use "git add ..." to update what will be committed)<br />
  (use "git checkout -- ..." to discard changes in working directory)<br />
<br />
	modified:   conf/default/emonhub<br />
	modified:   service/emonhub.service<br />
<br />
no changes added to commit (use "git add" and/or "git commit -a")<br />
Already up-to-date.<br />
git pull /home/pi/usefulscripts<br />
* master<br />
On branch master<br />
Your branch is up-to-date with 'origin/master'.<br />
nothing to commit, working tree clean<br />
Already up-to-date.<br />
git pull /home/pi/huawei-hilink-status<br />
* master<br />
On branch master<br />
Your branch is up-to-date with 'origin/master'.<br />
nothing to commit, working tree clean<br />
Already up-to-date.<br />
<br />
Start emonPi Atmega328 firmware update:<br />
<br />
=================================<br />
EmonPi update started<br />
=================================<br />
<br />
EUID: 1000<br />
<br />
Requirement already up-to-date: paho-mqtt in /usr/local/lib/python2.7/dist-packages<br />
Hit:1 http://archive.raspberrypi.org/debian stretch InRelease<br />
Get:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15.0 kB]<br />
Fetched 15.0 kB in 1s (9,090 B/s)<br />
Reading package lists...<br />
Reading package lists...<br />
Building dependency tree...<br />
Reading state information...<br />
rng-tools is already the newest version (2-unofficial-mt.14-1).<br />
python-gpiozero is already the newest version (1.5.0).<br />
python-rpi.gpio is already the newest version (0.6.5~stretch-1).<br />
0 upgraded, 0 newly installed, 0 to remove and 118 not upgraded.<br />
Start ATmega328 serial upload using avrdude with latest.hex<br />
Discrete Sampling<br />
avrdude -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/home/pi/emonpi/firmware/compiled/latest.hex<br />
avrdude-original: Using autoreset DTR on GPIO Pin 7<br />
<br />
avrdude-original: AVR device initialized and ready to accept instructions<br />
<br />
Reading | ################################################## | 100% 0.00s<br />
<br />
avrdude-original: Device signature = 0x1e950f (probably m328p)<br />
avrdude-original: NOTE: "flash" memory has been specified, an erase cycle will be performed<br />
                  To disable this feature, specify the -D option.<br />
avrdude-original: erasing chip<br />
avrdude-original: reading input file "/home/pi/emonpi/firmware/compiled/latest.hex"<br />
avrdude-original: input file /home/pi/emonpi/firmware/compiled/latest.hex auto detected as Intel Hex<br />
avrdude-original: writing flash (17886 bytes):<br />
<br />
Writing | avrdude-original: stk500_loadaddr(): (a) protocol error, expect=0x14, resp=0x1c<br />
<br />
avrdude-original: stk500_paged_write(): (a) protocol error, expect=0x14, resp=0xe0<br />
avrdude-original: stk500_cmd(): programmer is out of sync<br />
avrdude-original: stk500_cmd(): programmer is out of sync<br />
avrdude-original: stk500_cmd(): programmer is out of sync<br />
[snipped a further 100 lines]
avrdude-original: stk500_recv(): programmer is not responding<br />
avrdude-original: stk500_recv(): programmer is not responding<br />
avrdude-original: stk500_recv(): programmer is not responding<br />
[snipped a further 261 lines which stop when device was shutdown]

The after a reboot I updated again and this appeared to be more successful, but did not resolve the issues with Emonhub.

> #############################################################
> Starting emonPi Update >
> via service-runner-update.sh
> Service Runner update script V1.1.1
> EUID: 1000
> Argument: emonhub
> Tue 18 Jun 12:10:07 UTC 2019
> #############################################################
> 
> emonSD version: emonSD-30Oct18
> 
> emonSD base image check passed...continue update
> 
> #############################################################
> I2C LCD DETECTED Ox3f
> git pull /home/pi/emonpi
>   enable-ssh
>   enable-ssh-button
>   gpiozero-enable-ssh
>   greeebs-patch-1
> * master
>   python-systemd-servicerunner
> On branch master
> Your branch is behind 'origin/master' by 145 commits, and can be fast-forwarded.
>   (use "git pull" to update your local branch)
> Changes not staged for commit:
>   (use "git add <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
> 
> 	modified:   lcd/emonPiLCD.cfg
> 
> no changes added to commit (use "git add" and/or "git commit -a")
> error: Your local changes to the following files would be overwritten by merge:
> 	lcd/emonPiLCD.cfg
> Please commit your changes or stash them before you merge.
> Aborting
> Updating bc7ebd1..0a0c6ba
> git pull /home/pi/RFM2Pi
> * master
> On branch master
> Your branch is up-to-date with 'origin/master'.
> nothing to commit, working tree clean
> Already up-to-date.
> git pull /home/pi/emonhub
> * emon-pi
> On branch emon-pi
> Your branch is up-to-date with 'origin/emon-pi'.
> Changes not staged for commit:
>   (use "git add <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
> 
> 	modified:   conf/default/emonhub
> 	modified:   service/emonhub.service
> 
> no changes added to commit (use "git add" and/or "git commit -a")
> Already up-to-date.
> git pull /home/pi/usefulscripts
> * master
> On branch master
> Your branch is up-to-date with 'origin/master'.
> nothing to commit, working tree clean
> Already up-to-date.
> git pull /home/pi/huawei-hilink-status
> * master
> On branch master
> Your branch is up-to-date with 'origin/master'.
> nothing to commit, working tree clean
> Already up-to-date.
> 
> Start emonPi Atmega328 firmware update:
> 
> =================================
> EmonPi update started
> =================================
> 
> EUID: 1000
> 
> Requirement already up-to-date: paho-mqtt in /usr/local/lib/python2.7/dist-packages
> Hit:1 http://archive.raspberrypi.org/debian stretch InRelease
> Hit:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease
> Reading package lists...
> Reading package lists...
> Building dependency tree...
> Reading state information...
> rng-tools is already the newest version (2-unofficial-mt.14-1).
> python-gpiozero is already the newest version (1.5.0).
> python-rpi.gpio is already the newest version (0.6.5~stretch-1).
> 0 upgraded, 0 newly installed, 0 to remove and 118 not upgraded.
> Start ATmega328 serial upload using avrdude with latest.hex
> Discrete Sampling
> avrdude -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/home/pi/emonpi/firmware/compiled/latest.hex
> avrdude-original: Using autoreset DTR on GPIO Pin 7
> 
> avrdude-original: AVR device initialized and ready to accept instructions
> 
> Reading | ################################################## | 100% 0.00s
> 
> avrdude-original: Device signature = 0x1e950f (probably m328p)
> avrdude-original: NOTE: "flash" memory has been specified, an erase cycle will be performed
>                   To disable this feature, specify the -D option.
> avrdude-original: erasing chip
> avrdude-original: reading input file "/home/pi/emonpi/firmware/compiled/latest.hex"
> avrdude-original: input file /home/pi/emonpi/firmware/compiled/latest.hex auto detected as Intel Hex
> avrdude-original: writing flash (17886 bytes):
> 
> Writing | ################################################## | 100% 2.48s
> 
> avrdude-original: 17886 bytes of flash written
> avrdude-original: verifying flash memory against /home/pi/emonpi/firmware/compiled/latest.hex:
> avrdude-original: load data flash data from input file /home/pi/emonpi/firmware/compiled/latest.hex:
> avrdude-original: input file /home/pi/emonpi/firmware/compiled/latest.hex auto detected as Intel Hex
> avrdude-original: input file /home/pi/emonpi/firmware/compiled/latest.hex contains 17886 bytes
> avrdude-original: reading on-chip flash data:
> 
> Reading | ################################################## | 100% 1.86s
> 
> avrdude-original: verifying ...
> avrdude-original: 17886 bytes of flash verified
> strace: |autoreset: Broken pipe
> strace: |autoreset: Broken pipe
> strace: |autoreset: Broken pipe
> strace: |autoreset: Broken pipe
> strace: |autoreset: Broken pipe
> 
> avrdude-original done.  Thank you.
> 
> strace: |autoreset: Broken pipe
> 
> 
> Start emonhub update script:
> 
> =================================
> emonHub update started
> =================================
> emonhub.service already installed
> 
> Running emonhub automatic node addition script
> EUID: 1000
> [[5]]
> Node 5 already present
> [[6]]
> Node 6 already present
> [[7]]
> Node 7 already present
> [[8]]
> Node 8 already present
> [[9]]
> Node 9 already present
> [[10]]
> Node 10 already present
> [[11]]
> Node 11 already present
> [[12]]
> Node 12 already present
> [[13]]
> Node 13 already present
> [[14]]
> Node 14 already present
> [[19]]
> Node 19 already present
> [[20]]
> Node 20 already present
> [[21]]
> Node 21 already present
> [[22]]
> Node 22 already present
> [[23]]
> Node 23 already present
> [[24]]
> Node 24 already present
> [[25]]
> Node 25 already present
> [[26]]
> Node 26 already present
> 
> Start emoncms update:
> 
> ===========================================
> Emoncms update started
> Emoncms update script V1.2.1 (6th Feb 2019)
> Tue 18 Jun 12:10:39 UTC 2019
> ===========================================
> 
> username: pi
> 
> Checking environment:
> - User directory /home/pi found
> - pi user and pi user directory found
> - emonSD version: emonSD-30Oct18
> - emoncms directory: /var/www/emoncms
> 
> EUID: 1000
> /home/pi/emonpi/emonhub-sudoers: parsed OK
> 
> emonhub service control sudoers entry installed
> www-data : www-data video systemd-journal
> 
> current settings.php md5: 0e4803a641b7784ed23edf87506ce761
> Default settings.php md5: 0e4803a641b7784ed23edf87506ce761
> 
> Checking status of /var/www/emoncms git repository
> - git branch: stable
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/emoncms
>  * branch              stable     -> FETCH_HEAD
> Already up-to-date.
> 
> NEW default settings.php md5: 0e4803a641b7784ed23edf87506ce761
> settings.php has NOT been user modifed
> settings.php not updated
> 
> ==========================================
> UPDATING EMONCMS MODULES
> ==========================================
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/app module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.2.0-108-g6e5016a
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/app
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/config module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.0.0-63-g1d9238e
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/config
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/dashboard module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.3.1-115-gabd1a0b
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/dashboard
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/device module
> ------------------------------------------
> - git branch: master
> - git tags: 1.1.1-48-g2ea6e74
> - no local changes
> - running: git pull origin master
> 
> From https://github.com/emoncms/device
>  * branch            master     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/graph module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.2.1-151-gc7f4ea9
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/graph
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating /var/www/emoncms/Modules/wifi module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.2.0-39-gb67783d
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/wifi
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating postprocess module
> ------------------------------------------
> - git branch: emonpi
> fatal: No names found, cannot describe anything.
> - git tags: 
> - no local changes
> - running: git pull origin emonpi
> 
> From https://github.com/emoncms/postprocess
>  * branch            emonpi     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating sync module
> ------------------------------------------
> - git branch: master
> - git tags: 1.0.0-23-gc5eb931
> - no local changes
> - running: git pull origin master
> 
> From https://github.com/emoncms/sync
>  * branch            master     -> FETCH_HEAD
> Already up-to-date.
> 
> ------------------------------------------
> Updating backup module
> ------------------------------------------
> - git branch: stable
> - git tags: 1.1.5-54-g67a1f1d
> - no local changes
> - running: git pull origin stable
> 
> From https://github.com/emoncms/backup
>  * branch            stable     -> FETCH_HEAD
> Already up-to-date.
> 
> ==========================================
> AUTO INSTALL EMONCMS MODULES
> ==========================================
> - graph module already installed
> - device module already installed
> - postprocess module already installed
> - sync module already installed
> - backup module already installed
> 
> 
> ==========================================
> SERVICES
> ==========================================
> emoncms_mqtt.service already installed
> feedwriter.service already installed
> service-runner.service already installed
> ------------------------------------------
> Update Emoncms database
> []
> 
> Restarting Services...
> - sudo systemctl restart feedwriter.service
> --- ActiveState=active ---
> - sudo systemctl restart emoncms_mqtt.service
> --- ActiveState=active ---
> - sudo systemctl restart emonhub.service
> --- ActiveState=active ---
> Installing emoncms logrotate...
> 
> Now setting up Logrotate...
> Backing up old logrotate configuration...
> Linked to new logrotate configuration...
> Backing up old logrotate cron job...
> mv: cannot stat '/etc/cron.daily/logrotate': No such file or directory
> Linked to new logrotate cron job...
> 
> Completed
> 
> 
> Completed
> 
> Running logrotate...
> 
> set log rotate config owner to root
> 
> ------------------------------------------
> Emoncms update script complete
> ------------------------------------------
> 
> 
> Starting emonPi LCD service..
> 
> Tue 18 Jun 12:11:01 UTC 2019
> 
> 
> ...................
> emonPi update done
> restarting service-runner
> service-runner: no process found