Community
OpenEnergyMonitor

Community

Solved: Emonhub failed after update (to python3?) - python3-pip fails to install

Tags: #<Tag:0x00007f6e108fbad0> #<Tag:0x00007f6e108fb3f0>

We had a power cut, and my emonPi3 didn’t do to well after power was restored. It’s running emonSD-Oct19 and was updated not too long ago to 10.2.3

It looks like it decided it was a first run, and so ran the update, and it looks like it has tried to update emonhub to the python 3 version.

However, the upgrade fails. When I ran it manually, it failed to install pyphon3-pip

The following packages have unmet dependencies:

python3-pip : Depends: python-pip-whl (= 18.1-5) but 18.1-5+rpt1 is to be installed
Recommends: python3-dev (>= 3.2) but it is not going to be installed

Recommends: python3-setuptools but it is not going to be installed

Recommends: python3-wheel but it is not going to be installed

E: Unable to correct problems, you have held broken packages.

sudo: pip3: command not found

It actually also failed to install python3-serial and python3 configobj, but I installed those manually before eventually understanding what had changed (emonhub now python3).

The emonpi update log /var/log/emoncms/emonpiupdate.log is empty

Any help appreciated

Is your /var/log full?

Couple of other threads on issues with the update.

Hi @borpin /var/log has plenty of space:

[email protected]:~ $ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        4114268 2218440   1693084  57% /
devtmpfs          494908       0    494908   0% /dev
tmpfs             499516       0    499516   0% /dev/shm
tmpfs             499516   12896    486620   3% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             499516       0    499516   0% /sys/fs/cgroup
tmpfs              30720    1412     29308   5% /tmp
tmpfs               1024       0      1024   0% /var/tmp
tmpfs               1024       0      1024   0% /var/lib/php/sessions
/dev/mmcblk0p3  10464739  342090   9590937   4% /var/opt/emoncms
/dev/mmcblk0p1    258095   53034    205061  21% /boot
log2ram            51200   31416     19784  62% /var/log
tmpfs              99900       0     99900   0% /run/user/1000

Additional information I forgot to add last night:

Server Information

Server Information

Services

  • emonhub :- Failed Failed
  • emoncms_mqtt :- Active Running
  • feedwriter :- Active Running - sleep 300s 0 feed points pending write
  • service-runner :- Active Running
  • emonPiLCD :- Inactive Dead
  • redis-server :- Active Running
  • mosquitto :- Active Running
  • demandshaper :- Active Running

Emoncms

  • Version :- low-write 10.2.3
  • Modules :- Administration | App v2.1.6 | Backup v2.2.1 | EmonHub Config v2.0.5 | Dashboard v2.0.7 | DemandShaper v1.2.5 | Device v2.0.5 | EventProcesses | Feed | Graph v2.0.9 | Input | Postprocess v2.1.4 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
  • Git :-

Server

  • OS :- Linux 4.19.75-v7+
  • Host :- emonpi | emonpi | (192.168.0.18)
  • Date :- 2020-06-15 07:57:42 BST
  • Uptime :- 07:57:42 up 9:52, 1 user, load average: 0.26, 0.09, 0.02

Memory

  • RAM :- Used: 24.91%
    • Total :- 975.62 MB
    • Used :- 243 MB
    • Free :- 732.63 MB
  • Swap :- Used: 0.00%
    • Total :- 100 MB
    • Used :- 0 B
    • Free :- 100 MB
      Write Load Period

Disk

  • / :- Used: 53.92%
    • Total :- 3.92 GB
    • Used :- 2.12 GB
    • Free :- 1.61 GB
    • Write Load :- 8.22 KB/s (9 hours 24 mins)
  • /var/opt/emoncms :- Used: 3.27%
    • Total :- 9.98 GB
    • Used :- 334.07 MB
    • Free :- 9.15 GB
    • Write Load :- 5.56 KB/s (9 hours 24 mins)
  • /boot :- Used: 20.55%
    • Total :- 252.05 MB
    • Used :- 51.79 MB
    • Free :- 200.25 MB
    • Write Load :- 0.33 B/s (9 hours 24 mins)
  • /var/log :- Used: 61.39%
    • Total :- 50 MB
    • Used :- 30.7 MB
    • Free :- 19.3 MB
    • Write Load :- n/a

HTTP

  • Server :- Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

  • Version :- 5.5.5-10.3.17-MariaDB-0+deb10u1
  • Host :- localhost:6379 (127.0.0.1)
  • Date :- 2020-06-15 07:57:42 (UTC 01:00‌​)
  • Stats :- Uptime: 36207 Threads: 13 Questions: 34960 Slow queries: 0 Opens: 60 Flush tables: 1 Open tables: 54 Queries per second avg: 0.965

Redis

  • Version :-
    • Redis Server :- 5.0.3
    • PHP Redis :- 5.0.2
  • Host :- localhost:6379
  • Size :- 214 keys (849.53K)
  • Uptime :- 0 days

MQTT Server

  • Version :- Mosquitto 1.5.7
  • Host :- localhost:1883 (127.0.0.1)

PHP

  • Version :- 7.3.9-1~deb10u1 (Zend Version 3.3.9)
  • Modules :- apache2handlercalendar Core ctype curl date dom v20031129exif fileinfo filter ftp gd gettext hash iconv json v1.7.0libxml mbstring mosquitto v0.4.0mysqli mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $openssl pcre PDO pdo_mysql Phar posix readline redis v5.0.2Reflection session shmop SimpleXML sockets sodium SPL standard sysvmsg sysvsem sysvshm tokenizer wddx xml xmlreader xmlwriter xsl Zend OPcache zlib

Pi

  • Model :- Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)

  • Serial num. :- 8574B2A6

  • CPU Temperature :- 54.77°C

  • GPU Temperature :- 54.8°C

  • emonpiRelease :- emonSD-17Oct19

  • File-system :- read-write

Client Information

Client Information

HTTP

  • Browser :- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0
  • Language :- en-GB,en;q=0.5

Window

  • Size :- 1263 x 597

Screen

  • Resolution :- 1280 x 720

And emoncms.log contains the following:

2020-06-15 06:57:23.383|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:23.404|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:24.013|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:24.371|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:24.700|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:25.316|ERROR|index.php|Not Admin|admin/emonpi/getupdatelog
2020-06-15 06:57:25.352|ERROR|index.php|Not Admin|admin/view

I am assuming the log is in GMT, as 06:57 GMT is about the time I accessed the Admin page in emoncms this morning.

Odd. As an aside, if you paste code or SSH output, please use 3 ‘backticks’ on a line before and after (if you ‘edit’ your post you will see what I have done.

Can you

cd /opt/openenergymonitor/EmonScripts/update
git pull
./emonhub.sh

@IanDavies did you fix this?

Hi Brian, just about to carry on looking at it … thanks for you support

Well that didn’t take long - not good…

[email protected]:/opt/openenergymonitor/EmonScripts/update $ git pull
error: object file .git/objects/de/c912b043f14ba0f0bf97f4a228acd3ae115d27 is empty
fatal: loose object dec912b043f14ba0f0bf97f4a228acd3ae115d27 (stored in .git/objects/de/c912b043f14ba0f0bf97f4a228acd3ae115d27) is corrupt

Before I ran the git pull, I looked at the files in that directory:

[email protected]:/opt/openenergymonitor/EmonScripts/update $ ls -l
total 32
-rwxr-xr-x 1 pi pi 4374 Oct 17  2019 emoncms_core.sh
-rwxr-xr-x 1 pi pi 2631 Oct 17  2019 emoncms_modules.sh
-rwxr-xr-x 1 pi pi    0 Jun 14 06:17 emonhub.sh
-rwxr-xr-x 1 pi pi  620 Oct 17  2019 emonpi.sh
-rwxr-xr-x 1 pi pi    0 Jun 14 06:17 emontxv3cm.sh
-rwxr-xr-x 1 pi pi 1144 Oct 17  2019 firstbootupdate
-rw-r--r-- 1 pi pi  386 Oct 17  2019 load_config.sh
-rwxr-xr-x 1 pi pi    0 Jun 14 06:17 main.sh
-rw-r--r-- 1 pi pi  736 Oct 17  2019 readme.md
-rwxr-xr-x 1 pi pi    0 Jun 14 06:17 rfm12pi.sh
-rwxr-xr-x 1 pi pi 1368 Oct 17  2019 rfm69pi.sh
-rwxr-xr-x 1 pi pi    0 Jun 14 06:17 service-runner-update.sh

Several are zero length, and all at the same time 06:17, which is not when the power was cut, but seems to be when the autoupdate tried to run. Power was restored at 05:37.

Using dmesg seems to show some errors on the sdcard, looks like the power cut has scuppered the card. Probably need to rebuild.

Yep, get that backup…

On the other hand, maybe I found a solution (there might still be issues with the SD Card, but here is what I did to get emonhub working again, in case others are hit by the same issue with upgrading…

Recap: my Pi3 was failing to update emonhub, complaining about dependencies for python3-pip:

python3-pip : Depends: python-pip-whl (= 18.1-5) but 18.1-5+rpt1 is to be installed
Recommends: python3-dev (>= 3.2) but it is not going to be installed
Recommends: python3-setuptools but it is not going to be installed
Recommends: python3-wheel but it is not going to be installed

E: Unable to correct problems, you have held broken packages.

sudo: pip3: command not found

What worked for me was removing python-pip-whl:

sudo apt-get remove python-pip-whl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  gir1.2-glib-2.0 libgirepository-1.0-1 libjs-sphinxdoc libjs-underscore libpython-all-dev python-all python-all-dev python-asn1crypto python-cffi-backend
  python-configparser python-crypto python-cryptography python-dbus python-entrypoints python-enum34 python-gi python-ipaddress python-keyring
  python-keyrings.alt python-secretstorage python-wheel python-xdg
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  python-pip python-pip-whl
0 upgraded, 0 newly installed, 2 to remove and 119 not upgraded.
After this operation, 2,790 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 51965 files and directories currently installed.)
Removing python-pip (18.1-5+rpt1) ...
Removing python-pip-whl (18.1-5+rpt1) ...
Processing triggers for man-db (2.8.5-2) ...

I then manually installed python3-pip which installed cleanly, although I realised after that I didn’t need to do that, because I then re-ran the install.sh script, which installs python3-pip anyway! The script also downloads other packages (most of which I had manually downloaded whilst trying to fix this yesterday), and starts up the emonhub service, which is now working - Yay!

The re-install of python3-pip, and when I ran the install.sh script, both listed a bunch of packages as no longer needed: I assume they are the python 2.7 versions as they don’t have a “3” in their name? I haven’t removed any, but might be useful to know if we still need them, or, why does apt think they are not needed anymore?

[email protected]:/tmp/emonhub $ ./install.sh
Apply raspberrypi serial configuration? 1=yes, 0=no:
You entered 1

Reading package lists... Done
Building dependency tree
Reading state information... Done
python3-configobj is already the newest version (5.0.6-3).
python3-pip is already the newest version (18.1-5).
python3-serial is already the newest version (3.4-4).
The following packages were automatically installed and are no longer required:
  libjs-sphinxdoc libjs-underscore libpython-all-dev python-all python-all-dev python-asn1crypto python-cffi-backend python-configparser python-crypto
  python-cryptography python-dbus python-entrypoints python-enum34 python-gi python-ipaddress python-keyring python-keyrings.alt python-secretstorage
  python-wheel python-xdg
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 114 not upgraded.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting paho-mqtt
  Downloading https://www.piwheels.org/simple/paho-mqtt/paho_mqtt-1.5.0-py3-none-any.whl (61kB)
    100% |████████████████████████████████| 61kB 903kB/s
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.21.0)
Installing collected packages: paho-mqtt
Successfully installed paho-mqtt-1.5.0
Unit /etc/systemd/system/[email protected] is masked, ignoring.
useradd: user 'emonhub' already exists
- installing emonhub.service
- Service ActiveState=active

I’d rather folk didn’t do this.

Somewhat concerned this has not been done from the right folder and so will break in future.

Can you post the output from

systemctl status emonhub.service

just to be sure.

Hi Brian,

Here you are:

[email protected]:~ $ systemctl status emonhub.service
● emonhub.service - emonHub service description
   Loaded: loaded (/opt/openenergymonitor/emonhub/service/emonhub.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-06-15 22:30:26 BST; 23h ago
  Process: 14094 ExecStartPre=/bin/mkdir -p /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 14095 ExecStartPre=/bin/chgrp -R emonhub /var/log/emonhub/ (code=exited, status=0/SUCCESS)
  Process: 14096 ExecStartPre=/bin/chmod 775 /var/log/emonhub/ (code=exited, status=0/SUCCESS)
 Main PID: 14097 (python3)
    Tasks: 4 (limit: 2319)
   Memory: 13.5M
   CGroup: /system.slice/emonhub.service
           └─14097 python3 /usr/local/bin/emonhub/emonhub.py --config-file=/etc/emonhub/emonhub.conf --logfile=/var/log/emonhub/emonhub.log

Jun 15 22:30:26 emonpi systemd[1]: Starting emonHub service description...
Jun 15 22:30:26 emonpi systemd[1]: Started emonHub service description.

Thanks for your support with this

Ian

1 Like