Sync of daily feeds to local EmonPi

Sprky · 2018-06-05T09:28:30.618Z

You can find an EmonPi update button in the menu Setup > Administration.

JideyF · 2018-06-05T12:20:53.810Z

I did the update
When I try to sync Inputs I have following message
using host https://emoncms.org + my account

Authentication failure, username or password incorrect

Log:
2018-06-05 12:19:11.956|WARN|sync_model.php|remote save
2018-06-05 12:19:12.572|WARN|sync_model.php|{“success”:false,“message”:“Incorrect authentication”}
2018-06-05 12:19:27.737|WARN|sync_model.php|remote save
2018-06-05 12:19:28.520|WARN|sync_model.php|{“success”:false,“message”:“Incorrect authentication”}

What I can notice is that the time is not right, this can perhaps affect the https connection …
how do I get the real and right time configured on the pi ?

any other idea ?

JideyF · 2018-06-07T16:36:01.845Z

Don’t know how to force another branch for update… here my update Log…
Notice I have an old pi2 and RFM2pi as RF module

Filesystem is unlocked - Write access
type ' rpi-ro ' to lock
I2C LCD NOT DETECTED on either 0x['27', '3f'] ...exiting LCD script
Starting emonPi Update >
via service-runner-update.sh
Service Runner update script V1.0.0
EUID: 1000
Argument: emonpi
Thu  7 Jun 16:26:16 UTC 2018
#############################################################

emonSD version: emonSD-26Oct17

emonSD base image check passed...continue update

#############################################################
Filesystem is unlocked - Write access
type ' rpi-ro ' to lock
git pull /home/pi/emonpi
* master
  wifiap
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	1
	hardware/emonpi/emonpi2c/

nothing added to commit but untracked files present (use "git add" to track)
Updating 5c08fc4..223cd0a
Fast-forward
 emoncms-wifiscan.sh  |  3 +++
 emoncms_wifiscan.php | 27 +++++++++++++++++++++++++++
 service-runner       | 29 ++++++++++++++---------------
 3 files changed, 44 insertions(+), 15 deletions(-)
 create mode 100755 emoncms-wifiscan.sh
 create mode 100644 emoncms_wifiscan.php
git pull /home/pi/RFM2Pi
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
Already up-to-date.
git pull /home/pi/emonhub
  K0den-wibeee_interface
* emon-pi
  hmm01i-syslogging
On branch emon-pi
Your branch is up-to-date with 'origin/emon-pi'.
nothing to commit, working directory clean
Already up-to-date.
git pull /home/pi/oem_openHab
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
Already up-to-date.
git pull /home/pi/usefulscripts
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
Already up-to-date.
git pull /home/pi/huawei-hilink-status
* master
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
Already up-to-date.

Start emonPi Atmega328 firmware update:

=================================
EmonPi update started
=================================

EUID: 1000

Requirement already up-to-date: paho-mqtt in /usr/local/lib/python2.7/dist-packages
Stopping OpenEnergyMonitor emonHub: emonhub has been stopped ok.
Start ATmega328 serial upload using avrdude with latest.hex
Discrete Sampling
avrdude -c arduino -p ATMEGA328P -P /dev/ttyAMA0 -b 115200 -U flash:w:/home/pi/emonpi/firmware/compiled/latest.hex
avrdude-original: Using autoreset DTR on GPIO Pin 7
Starting OpenEnergyMonitor emonHub: emonhub has been started ok.
node-red-node-emoncms@0.2.1 node_modules/node-red-node-emoncms


Start emonhub update script:

=================================
EmonPi update started
=================================
Running emonhub automatic node addition script
EUID: 1000
EUID: 1000
[[5]]
Node 5 already present
[[6]]
Node 6 already present
[[7]]
Node 7 already present
[[8]]
Node 8 already present
[[9]]
Node 9 already present
[[10]]
Node 10 already present
[[11]]
Node 11 already present
[[12]]
Node 12 already present
[[13]]
Node 13 already present
[[14]]
Node 14 already present
[[19]]
Node 19 already present
[[20]]
Node 20 already present
[[21]]
Node 21 already present
[[22]]
Node 22 already present
[[23]]
Node 23 already present
[[24]]
Node 24 already present
[[25]]
Node 25 already present
[[26]]
Node 26 already present

Start emoncms update:

=================================
Emoncms update started
Emoncms update script V1.1.1

Thu  7 Jun 16:28:34 UTC 2018

#############################################################

emonSD version: emonSD-26Oct17

emonSD base image check pass...continue update

#############################################################
EUID: 1000
Checking cron tab for service runner entry...
service runner crontab entry already installed

current settings.php md5: b895b8f85cf79351470a1f5b14eace91
Default settings.php md5: f9fa74b4e54adb7d760e875e0764955e

git pull /var/www/emoncms
  dev-mosquitto-php
  master
* stable
  symlinked_modules
On branch stable
Your branch is up-to-date with 'origin/stable'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	emoncms

nothing added to commit but untracked files present (use "git add" to track)
Already up-to-date.

NEW default settings.php md5: f9fa74b4e54adb7d760e875e0764955e
settings.php HAS been user modified
settings.php not updated


git pull /var/www/emoncms/Modules/app
  9.0
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/config
  9.0
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/wifi
  9.0
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.
git pull /var/www/emoncms/Modules/dashboard
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.

git pull /var/www/emoncms/Modules/graph
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.

git pull /home/pi/postprocess
Your branch is up-to-date with 'remotes/origin/emonpi'.
Updating c827999..7cc1498
Fast-forward
 postprocess-module/postprocess_controller.php | 26 +++++---------------------
 1 file changed, 5 insertions(+), 21 deletions(-)
git pull /home/pi/sync
Your branch is up-to-date with 'origin/master'.
Updating e88a887..adaf7f7
Fast-forward
 sync-module/sync_controller.php | 19 +++++++++++++++++--
 sync-module/sync_model.php      | 15 ---------------
 2 files changed, 17 insertions(+), 17 deletions(-)
git pull /home/pi/backup
  master
* stable
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean
Already up-to-date.
Your branch is up-to-date with 'origin/stable'.

update mqtt_input systemd unit file

Update Emoncms database
[]

Restarting Services...
Restarting OpenEnergyMonitor emonHub: emonhub has been restarted ok.
Log is turned off
Restarting feedwriter

Restarting openhab (via systemctl): openhab.service.


set log rotate config owner to root
Restarting Services...


Starting emonPi LCD service..

Filesystem is locked - Read Only access
type ' rpi-rw ' to unlock
Thu  7 Jun 16:29:52 UTC 2018


...................
emonPi update done

emonpiupdate.zip (1.9 KB)

JideyF · 2018-06-15T07:15:59.906Z

Has anyone an Idea how I can get an update Version where the Sync Module works ?
What should I change to get another Version that the Stable one when I call update ?

Thanks

Vster · 2018-06-16T14:15:00.535Z

Hi,
I’m having similar issues.
I updated the Sync module by ssh’ing into the Pi and getting the new version from GitHub.
From memory (I’m not at home)
rpi-rw
cd sync
git pull

That said, I’m still not able to get all the data, dashes and Processes out of EmonCMS.org correctly yet. I think @TrystanLea is still working on it.

TrystanLea · 2018-06-21T20:04:16.204Z

JideyF:

Has anyone an Idea how I can get an update Version where the Sync Module works ?
What should I change to get another Version that the Stable one when I call update ?

The sync module should update ok as part of the emonpi update process, here’s an snippet out of my emonpiupdate.log from my latest call to emonpi update:

git pull /home/pi/postprocess
Your branch is up-to-date with 'remotes/origin/emonpi'.
Already up-to-date.
git pull /home/pi/sync
Your branch is up-to-date with 'origin/master'.
Already up-to-date.

Vster:

That said, I’m still not able to get all the data, dashes and Processes out of EmonCMS.org correctly yet.

What are you still missing?

JideyF · 2018-06-22T07:18:07.646Z

Hi Tristan,
I pulled all and it says it is uptodate
I stil have the same issue with Authentication Failure …
I don’t know why … Thanks

2018-06-09 10:18:50.461|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2018-06-13 12:21:47.067|WARN|sync_model.php|remote save
2018-06-13 12:21:47.722|WARN|sync_model.php|{"success":false,"message":"Incorrect authentication"}

TrystanLea · 2018-06-22T12:36:22.923Z

I know its asking the obvious, are you using the username and password from the emoncms.org account? could you try re-entering them?

JideyF · 2018-06-22T13:55:39.950Z

Yes i tried with emoncms.org user and pwd
i tried to change the pwd.
But no results. Still the same error.

JideyF · 2018-07-04T09:53:25.746Z

Hi,

I finaly managed to sync my local emoncms.
after requesting a new password for my emoncms.org account.
I just noticed that it did not sync the process of each input, I created them again manually.
I also had to set again the feed valus to be used for each dashboard element.

Now I just need to get all my historic data for the last 4 years. I hope using usefulscripts ?

JideyF · 2018-07-04T11:37:04.208Z

I executed the backup script and got all my remote feeds into my local emoncms
but…
When I try to access a feed graph, i have followind error:

Request error
Notice: Undefined offset: 0 in /var/www/emoncms/Modules/feed/feed_model.php on line 557
Notice: Undefined offset: 1 in /var/www/emoncms/Modules/feed/feed_model.php on line 557
Warning: Cannot modify header information - headers already sent by (output started at /var/www/emoncms/Modules/feed/feed_model.php:557) in /var/www/emoncms/index.php on line 247
{“success”:false,“message”:“Error reading meta data feedid=19258”}

@TrystanLea
Could you help me solve this hopefully last problem ?
Thanks

JideyF · 2018-08-27T11:44:04.474Z

Hi
Has someone the same problem with Graphs after Sync beetween local EmonCMS and .org ?
It to have a problem with the imported Feed IDs …

Thanks

Robert.Wall · 2018-08-27T11:48:55.878Z

Your local emonCMS and emoncms.org are different, but if you look at other threads, moves are afoot to bring the two versions closer together. If you watch that/those threads, there might be updates coming that will solve your problem.

It would appear that @TrystanLea didn’t see your July 4th post and question.

TrystanLea · 2018-08-27T14:05:24.577Z

@JideyF Did you get the sync module to work in the end, or did you download the data with usefulscripts? The error is resulting from this line of code here which is to do with the redis buffer, I wonder if you could try temporarily turning the redis buffer off in settings.php?

The redis buffer is important for the emonpi as it helps reduce SD card wear increasing livespan so we can hopefully resolve the situation so that you can turn it back on again.

JideyF · 2018-08-28T09:22:11.437Z

Hi Trystan,

I reset my emonCMS.org user passwd and after this I got the sync module to work yes.
All data was synced, feed size are the same on local and remote emonCMS
What has not being synced are the process lists …
So I recreated them and configured them to write to the synced feed

But now I have problems with reading these imported feeds Request error
Notice : Undefined offset: 0 in /var/www/emoncms/Modules/feed/feed_model.php on line 557

Notice : Undefined offset: 1 in /var/www/emoncms/Modules/feed/feed_model.php on line 557

Warning : Cannot modify header information - headers already sent by (output started at /var/www/emoncms/Modules/feed/feed_model.php:557) in /var/www/emoncms/index.php on line 247
{“success”:false,“message”:“Error reading meta data feedid=31570”}

Log says
WARN|PHPFina.php|post_bulk_prepare() failed to fetch meta feedid=233060
.

I tryed to turn redis to false in settings but then I get an error …

settings.php file error

buffered writing requires redis but its disabled, check settings: settings[‘redisbuffer’][‘enabled’], redis_enabled

JideyF · 2018-09-21T12:23:57.400Z

+1
Has someone the final solution for syncing all the history data from EmonCMS.org to Local EmonCMS ?
I still can’t access the data synced using the Sync Module …

Thanks

TrystanLea · 2018-09-21T15:29:16.278Z

@JideyF you mention the feed sizes appear the same for local and remote feeds, do you see any data in the graph or is that when you see the error you mention?

JideyF · 2018-09-24T06:14:33.000Z

Hi Tristan,

Now I think all is OK

What I did:

update Local EmonPi in Administration Tab
deleted all the inputs and feeds locally
started again a Sync with my eMonCMS.org account

I had to re-assign the Inputs to the Local Feeds in the Process Lists

I had to recreate the graphs

Then I had to re-assign the components on my different Dashboards to the local Feeds/Graphs

Now All is working Locally ! Great !!!

Many thanks !

Le ven. 21 sept. 2018 à 17:34, Trystan Lea noreply@community.openenergymonitor.org a écrit :

TrystanLea · 2018-09-24T17:06:37.676Z

Great to hear!

pb66 · 2019-05-08T09:55:34.367Z

2 posts were merged into an existing topic: Missing postprocess and sync menu entries emoncms 9.9.9