Brand new emonPi emonCMS not working

SJM81 · 2019-10-24T16:02:29.171Z

Help! - I’ve got a brand new emonPi delivered today, but it does not seem to be happy - there are various symptoms.

Wifi scan did not work - even after ~30m. I had to directly edit the wpa_supplicant.conf to get it to join my network. On the WiFi config page it is continuously “Scanning for networks…”, and the Network Log has an error Error: Cannot find /home/pi/emonpi/wifiap/networklog.sh
Once on my wifi, I am unable to run any updates. Any update logs /bin/sh: 1: /home/pi/emonpi/service-runner-update.sh: not found
I can view the inputs correctly, but when I try to configure a “log to feed” process I get ERROR: Feed could not be created, could not write meta data file fopen(/home/pi/data/phpfina/8.meta): failed to open stream: No such file or directory
The backup page has Error: missing backup config.cfg

It looks like the UI is looking in the wrong place (e.g. I can find service-runner-update.sh in /opt/openenergymonitor/EmonScripts/update/service-runner-update.sh)

Any help gratefully received!!!

emoncms.txt (5.2 KB)

Server Information

Services

emonhub :- Active Running
emoncms_mqtt :- Active Running
feedwriter :- Active Running - sleep 60s 0 feed points pending write
service-runner :- Active Running
emonPiLCD :- Active Running
redis-server :- Active Running
mosquitto :- Active Running
demandshaper :- Active Running

Emoncms

Version :- low-write 10.1.10
Modules :- Administration | App v2.0.8 | Backup v2.1.6 | EmonHub Config v2.0.4 | Dashboard v2.0.5 | DemandShaper v1.0.3 | Device v2.0.3 | EventProcesses | Feed | Graph v2.0.5 | Input | Postprocess v2.1.1 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * stable
- Describe :- 10.1.10

Server

OS :- Linux 4.19.75-v7+
Host :- emonpi | emonpi | (192.168.6.175)
Date :- 2019-10-24 16:27:59 BST
Uptime :- 16:27:59 up 1:20, 1 user, load average: 0.29, 0.20, 0.12

Memory

RAM :- Used: 21.75%
- Total :- 975.62 MB
- Used :- 212.16 MB
- Free :- 763.46 MB
Swap :- Used: 0.00%
- Total :- 100 MB
- Used :- 0 B
- Free :- 100 MB
  Write Load Period

Disk

/ :- Used: 43.20%
- Total :- 3.92 GB
- Used :- 1.7 GB
- Free :- 2.04 GB
- Write Load :- 1.96 KB/s (1 hours 17 mins)
/var/opt/emoncms :- Used: 0.04%
- Total :- 9.98 GB
- Used :- 3.77 MB
- Free :- 9.47 GB
- Write Load :- 0 B/s (1 hours 17 mins)
/boot :- Used: 20.55%
- Total :- 252.05 MB
- Used :- 51.79 MB
- Free :- 200.26 MB
- Write Load :- 0 B/s (1 hours 17 mins)
/var/log :- Used: 3.16%
- Total :- 50 MB
- Used :- 1.58 MB
- Free :- 48.42 MB
- Write Load :- n/a

HTTP

Server :- Apache/2.4.38 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

Version :- 5.5.5-10.3.17-MariaDB-0+deb10u1
Host :- localhost:6379 (127.0.0.1)
Date :- 2019-10-24 16:27:58 (UTC 01:00‌)
Stats :- Uptime: 15161 Threads: 15 Questions: 8785 Slow queries: 0 Opens: 46 Flush tables: 1 Open tables: 40 Queries per second avg: 0.579

Redis

Version :-
- Redis Server :- 5.0.3
- PHP Redis :- 5.0.2
Host :- localhost:6379
Size :- 58 keys (805.35K)
Uptime :- 0 days

MQTT Server

Version :- Mosquitto 1.5.7
Host :- localhost:1883 (127.0.0.1)

PHP

Version :- 7.3.9-1~deb10u1 (Zend Version 3.3.9)
Modules :- apache2handler | calendar v7.3.9-1~deb10u1 | Core v7.3.9-1~deb10u1 | ctype v7.3.9-1~deb10u1 | curl v7.3.9-1~deb10u1 | date v7.3.9-1~deb10u1 | dom v20031129 | exif v7.3.9-1~deb10u1 | fileinfo v7.3.9-1~deb10u1 | filter v7.3.9-1~deb10u1 | ftp v7.3.9-1~deb10u1 | gd v7.3.9-1~deb10u1 | gettext v7.3.9-1~deb10u1 | hash v7.3.9-1~deb10u1 | iconv v7.3.9-1~deb10u1 | json v1.7.0 | libxml v7.3.9-1~deb10u1 | mbstring v7.3.9-1~deb10u1 | mosquitto v0.4.0 | mysqli v7.3.9-1~deb10u1 | mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: 7cc7cc96e675f6d72e5cf0f267f48e167c2abb23 $ | openssl v7.3.9-1~deb10u1 | pcre v7.3.9-1~deb10u1 | PDO v7.3.9-1~deb10u1 | pdo_mysql v7.3.9-1~deb10u1 | Phar v7.3.9-1~deb10u1 | posix v7.3.9-1~deb10u1 | readline v7.3.9-1~deb10u1 | redis v5.0.2 | Reflection v7.3.9-1~deb10u1 | session v7.3.9-1~deb10u1 | shmop v7.3.9-1~deb10u1 | SimpleXML v7.3.9-1~deb10u1 | sockets v7.3.9-1~deb10u1 | sodium v7.3.9-1~deb10u1 | SPL v7.3.9-1~deb10u1 | standard v7.3.9-1~deb10u1 | sysvmsg v7.3.9-1~deb10u1 | sysvsem v7.3.9-1~deb10u1 | sysvshm v7.3.9-1~deb10u1 | tokenizer v7.3.9-1~deb10u1 | wddx v7.3.9-1~deb10u1 | xml v7.3.9-1~deb10u1 | xmlreader v7.3.9-1~deb10u1 | xmlwriter v7.3.9-1~deb10u1 | xsl v7.3.9-1~deb10u1 | Zend OPcache v7.3.9-1~deb10u1 | zlib v7.3.9-1~deb10u1

Pi

Model :- Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)
Serial num. :- 1FE8D2F
Temperature :- 48.31°C - 48.9°C
emonpiRelease :- emonSD-17Oct19
File-system :- read-write

Client Information

HTTP

Browser :- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36
Language :- en-GB,en-US;q=0.9,en;q=0.8

Window

Size :- 1280 x 906

Screen

Resolution :- 2560 x 1080

borpin · 2019-10-24T17:33:38.401Z

SJM81:

I’ve got a brand new emonPi

@glyn.hudson @TrystanLea this is a new image?

@SJM81 try emailing the shop in this instance, you may get a quicker response. Looks like an issue with the new image. I’ll burn one here and see what happens.

borpin · 2019-10-24T18:14:05.339Z

Hi @SJM81, I have just burnt that image and all seems fine.

Have you edited either settings.php or settings.ini?

TrystanLea · 2019-10-24T18:17:37.766Z

O dear! Im not sure what’s gone wrong there, I havent seen any issues like this in my testing. Very happy to send a replacement SD card of course, if we cant get to the bottom of it another way I will send one out to you tomorrow.
If you don’t mind could you list the content of the emoncms folder:

cd /var/www/emoncms
ls -l

and then copy and paste here the content of /var/www/emoncms/settings.ini assuming it exists:

cat /var/www/emoncms/settings.ini

SJM81 · 2019-10-24T18:37:21.733Z

Thanks guys - here’s the emoncms folder:

pi@emonpi:/var/www/emoncms $ ls
composer.json  default.emonpi.settings.php  docs                  index.php    locale.php  php-info.php          route.php         settings.ini  Theme
COPYRIGHT.txt  default-settings.ini         example.settings.ini  Lib          Modules     process_settings.php  scripts           settings.php  version.txt
core.php       default-settings.php         example.settings.php  LICENSE.txt  param.php   readme.md             settings.env.ini  tests
pi@emonpi:/var/www/emoncms $ cat settings.ini
emoncms_dir = "/opt/emoncms"
openenergymonitor_dir = "/opt/openenergymonitor"

[sql]
server = "127.0.0.1"
database = "emoncms"
username = "emoncms"
password = "emonpiemoncmsmysql2016"
; Skip database setup test - set to false once database has been setup.
dbtest   = true

[redis]
enabled = true
prefix = ''

[mqtt]
enabled = true
user = 'emonpi'
password = 'emonpimqtt2016'

[feed]
engines_hidden = [0,6,10]
redisbuffer[enabled] = true
redisbuffer[sleep] = 300
phpfina[datadir] = '/var/opt/emoncms/phpfina/'
phptimeseries[datadir] = '/var/opt/emoncms/phptimeseries/'

[interface]
enable_admin_ui = true
feedviewpath = "graph/"
favicon = "favicon_emonpi.png"

[log]
; Log Level: 1=INFO, 2=WARN, 3=ERROR
level = 2
pi@emonpi:/var/www/emoncms $

TrystanLea · 2019-10-24T18:44:00.387Z

Thanks @SJM81, there’s a settings.php file present which should not be there. Could you copy and paste the content of settings.php and bash history?

cat /var/www/emoncms/settings.php
history

The errors might all go away if you remove the php settings file:

rm /var/www/emoncms/settings.php

SJM81 · 2019-10-24T18:50:18.681Z

I’ve done a rename on settings.php to settings.php.old (just in case ;)) - it appears to have fixed the issue! Wifi scanning returns in ~5s, updates work, as does backup and I’m just configuring my input processes now Thank you for your help!

Contents of settings.php:

<?php

//1 #### Mysql database settings
    $server   = "127.0.0.1";
    $database = "emoncms";
    $username = "emoncms";
    $password = "emonpiemoncmsmysql2016";
    $port     = "3306";
    // Skip database setup test - set to false once database has been setup.
    $dbtest = true;


//2 #### Redis
    $redis_enabled = true;
    $redis_server = array( 'host'   => 'localhost',
                           'port'   => 6379,
                           'auth'   => '',
                           'prefix' => '');


//3 #### MQTT
    // The 'subscriber' topic format is rx/* - where * is the emoncms input node number.
    // The 'publisher' topic format is user selectable from the 'Publish to MQTT' input process, for example power/solar
    $mqtt_enabled = true;            // Activate MQTT by changing to true
    $mqtt_server = array( 'host'     => 'localhost',
                          'port'     => 1883,
                          'user'     => 'emonpi',
                          'password' => 'emonpimqtt2016',
                          'basetopic'=> 'emon',
                          'client_id' => 'emoncms'
                          );


//4 #### Engine settings
    $feed_settings = array(
        // Supported engines. Uncommented engines will not be available for user to create a new feed using it. Existing feeds with a hidden engine still work.
        // Place a ',' as the first character on all uncommented engines lines but first.
        // If using emoncms in low-write mode, ensure that PHPFIWA is disabled by removing the leading //, from the PHPFIWA entry
        'engines_hidden'=>array(
            Engine::MYSQL,           // 0  Mysql traditional
            //Engine::MYSQLMEMORY,   // 8  Mysql with MEMORY tables on RAM. All data is lost on shutdown
            //Engine::PHPTIMESERIES, // 2
            //Engine::PHPFINA,       // 5
            Engine::PHPFIWA,         // 6  PHPFIWA disabled for compatibility with Low-write mode
            Engine::CASSANDRA        // 10 Apache Cassandra disabled by default for emonpi, enable if you wish to use
        ),

        // Redis Low-write mode
        'redisbuffer'=>array(
            'enabled' => true      // If enabled is true, requires redis enabled and feedwriter service running
            ,'sleep' => 60          // Number of seconds to wait before write buffer to disk - user selectable option
        ),

        'csvdownloadlimit_mb' => 25,     // Max csv download size in MB

        // Engines working folder. Default is /var/lib/phpfiwa,phpfina,phptimeseries
        // On windows or shared hosting you will likely need to specify a different data directory--
        // Make sure that emoncms has write permission's to the datadirectory folders
        'phpfiwa'=>array(
            'datadir' => '/home/pi/data/phpfiwa/'
        ),
        'phpfina'=>array(
            'datadir' => '/home/pi/data/phpfina/'
        ),
        'phptimeseries'=>array(
            'datadir' => '/home/pi/data/phptimeseries/'
        ),
        'cassandra'=>array(
            'keyspace' => 'emoncms'
        )
    );

    $homedir = "/home/pi";

    // Max number of allowed different inputs per user. For limiting garbage rf data
    $max_node_id_limit = 32;
    // Datapoint limit. Increasing this effects system performance but allows for more data points to be read from one api call
    $max_datapoints = 8928;


//5 #### User Interface settings
    // gettext  translations are found under each Module's locale directory
    $default_language = 'en_GB';

    // Theme location (folder located under Theme/, and must have the same structure as the basic one)
    $theme = "basic";
    // Theme colour options: "standard", "blue", "sun"
    $themecolor = "blue";

    // Favicon filenme in Theme/$theme
    $favicon = "favicon_emonpi.png";

    // Use full screen width
    $fullwidth = true;

    // Main menu collapses on lower screen widths
    $menucollapses = false;

    // Enable multi user emoncms.
    // If set to false, emoncms will automatically remove the register form and
    // ability to create further users after the first user has been created
    $enable_multi_user = false;

    // Enable remember me feature
    $enable_rememberme = true;

    // Allow user to reset his password
    $enable_password_reset = false;

    // Email address to email proccessed input values
    $default_emailto = 'root@localhost';

    // (OPTIONAL) Email SMTP, used for password reset or other email functions
    $smtp_email_settings = array(
      'host'=>"smtp.gmail.com",
      'port'=>"465",  // 25, 465, 587
      'from'=>array('noreply@emoncms.org' => 'EmonCMS'),
      // comment lines below that dont apply
      'encryption'=>"ssl", // ssl, tls
      'username'=>"yourusername@gmail.com",
      'password'=>"yourpassword"
    );

    // Default controller and action if none are specified and user is anonymous
    $default_controller = "user";
    $default_action = "login";

    // Default controller and action if none are specified and user is logged in
    $default_controller_auth = "feed";
    $default_action_auth = "list";

    // Public profile functionality
    // Allows http://yourdomain.com/[username]/[dash alias] or ?id=[dash id]
    // Alternative to http://yourdomain.com/dashboard/view?id=[dash id]
    // Add optional '&embed=1' in the end to remove header and footer
    $public_profile_enabled = true;
    $public_profile_controller = "dashboard";
    $public_profile_action = "view";

    // Default feed viewer: "vis/auto?feedid=" or "graph/" - requires module https://github.com/emoncms/graph
    $feedviewpath = "graph/";


//6 #### Other settings
    // Log file configuration
    $log_enabled = true;
    $log_location = "/var/log/emoncms";
    // Log Level: 1=INFO, 2=WARN, 3=ERROR
    $log_level = 2;

    // If installed on Emonpi, allow admin menu tools
    $allow_emonpi_admin = true;

    //experimental feature for virtual feeds average, default is true, set to false to activate average agregation with all data points, will be slower
    $data_sampling = false;

    // Show all fatal PHP errors
    $display_errors = true;

    // CSV export options for the number of decimal_places, decimal_place_separator and field_separator
    // The thousands separator is not used (specified as "nothing")
    // NOTE: don't make $csv_decimal_place_separator == $csv_field_separator
    // Adjust as appropriate for your location
    // number of decimal places
    $csv_decimal_places = 2;

    // decimal place separator
    $csv_decimal_place_separator = ".";

    // field separator
    $csv_field_separator = ",";

    // set true on docker installations
    $allow_config_env_vars = false;

    // Dont change - developer updates this when the config format changes
    $config_file_version = "10";

    // Set to true to run database update without logging in
    // URL Example: http://localhost/emoncms/admin/db
    $updatelogin = false;

    // Applicaton name
    $appname = "emoncms";

and

pi@emonpi:/var/www/emoncms $ history
    1  sudo su
    2  ls
    3  ls /
    4  sudo su
    5  sudo cat /etc/wpa_supplicant/wpa_supplicant.conf
    6  sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
    7  wpa_cli -i wlan0 reconfigure
    8  sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
    9  wpa_cli -i wlan0 reconfigure
   10  find / -name update.sh
   11  sudo find / -name update.sh
   12  sudo find / -name *update.sh
   13  /opt/openenergymonitor/EmonScripts/update/service-runner-update.s
   14  sudo /opt/openenergymonitor/EmonScripts/update/service-runner-update.sh
   15  /opt/openenergymonitor/EmonScripts/update/service-runner-update.sh
   16  nano /opt/openenergymonitor/EmonScripts/update/service-runner-update.sh
   17  rpi-rw && rm /home/pi/data/emonpiupdate.log && /home/pi/emonpi/firstbootupdate
   18  date
   19  rpi-rw
   20  sudo su
   21  /opt/openenergymonitor/EmonScripts/update/service-runner-update.sh
   22  timedatectl status
   23  ip a
   24  ping www.google.com
   25  systemctl status emonhub
   26  sudo apt-get update
   27  sudo apt-get upgrade
   28  sudo reboot now
   29  exit
   30  cd /var/www/emoncms/
   31  ls
   32  cat settings.ini
   33  cat settings.
   34  cat settings.php
   35  mv settings.php settings.php.old
   36  history

TrystanLea · 2019-10-24T19:05:02.340Z

ok great, very strange that the settings.php file should be there, looks like a stock version of the old settings file. Its also strange that your history does not provide the full image build history.

This is what history should look like (providing full transparency on the build):

1  uptime
2  df -h
3  wget https://raw.githubusercontent.com/openenergymonitor/EmonScripts/stable/install/init_resize.sh
4  chmod +x init_resize.sh
5  sudo mv init_resize.sh /usr/lib/raspi-config/init_resize.sh
6  sudo mv /boot/cmdline2.txt /boot/cmdline.txt
7  sudo reboot
8  sudo resize2fs /dev/mmcblk0p2
9  sudo mkfs.ext2 -b 1024 /dev/mmcblk0p3
10  wget https://raw.githubusercontent.com/openenergymonitor/EmonScripts/stable/defaults/etc/fstab
11  sudo mv fstab /etc/fstab
12  sudo mkdir /var/opt/emoncms
13  sudo chown www-data /var/opt/emoncms
14  sudo rebooot
15  sudo reboot
16  wget https://raw.githubusercontent.com/openenergymonitor/EmonScripts/stable/install/init.sh
17  chmod +x init.sh && ./init.sh
18  LS
19  ls
20  sudo systemctl disable hostapd.service 
21  sudo wifiAP start
22  sudo reboot
23  ls
24  sudo rm /var/log/emoncms/emonpiupdate.log
25  sudo rm /var/log/emoncms/emonupdate.log 
26  sudo rm /var/log/emoncms/emoncms.log 
27  ls
28  sudo update-rc.d ssh disable
29  sudo invoke-rc.d ssh stop
30  sudo dpkg-reconfigure openssh-server
31  sudo halt
32  cd /var/log/emoncms/
33  ls
34  cat emonpiupdate.log 
35  cd
36  history

All I can think is that somehow your pi got an SD card from the wrong box here in the shop but then the 17Oct19 image was only written to a small number of SD cards for testing… and I don’t remember doing a test with the old settings file and then clearing the history on the 17th of October image… so its a bit perplexing!

I’ve just taken 10 emonpi’s built today off the shelf and checked for a present settings.php file and any issues during update and they all appear fine.

Really sorry that you should get a faulty one! Happy to send another SD card for your peace of mind.

SJM81 · 2019-10-24T19:10:26.237Z

That’s OK, I’m not worried at all - it’s working great now! Thank you for the quick assistance, now to deploy emoncms in a kubernetes cluster for my home automation!

Glad to see other people make mistakes like this too sudo rebooot

Also, it’s dark now…

image1273×746 70.6 KB

martin · 2020-07-28T17:16:12.032Z

Just to note that I also received a brand new emonPi today, and saw exactly the same symptoms as described at the top of this thread. I was able to SSH in and move the settings.php file, and everything worked from thereon out.

TrystanLea · 2020-07-28T17:27:30.679Z

Thanks @martin, we just worked this out as well, there was a misunderstanding in the office whilst updating the master image last week that has left the settings.php file mistakenly in place. The quick fix is exactly as you have done.

We will be emailing everyone who has ordered a unit in the last week to offer either a replacement SD card or the steps to fix as you have done.

Ardmhacha24 · 2020-08-02T22:09:50.262Z

Looks like I may have the same issue? I bought a prebuilt SD card last week and am unable to initialize and setup feeds correctly or run updates…

Ardmhacha24 · 2020-08-02T22:10:45.798Z

Removing the .php file appears to have allowed me to now run updates…

Ardmhacha24 · 2020-08-02T22:12:14.542Z

Sam - would love to understand more about your k8 install?