Dataplicity for remote access

Radoslav_dataplicity · 2017-12-07T13:41:48.851Z

The steps below show how to access your emonPi remotely

1. Use SSH to access the terminal on emonPi.

2. Create an account with dataplicity.

21253×726 469 KB

31257×775 258 KB

3. Run the installation command.

4659×608 17.8 KB

4. Log in to your account and access your device remotely.

51144×212 34.1 KB

61026×719 83.1 KB

pb66 · 2017-12-07T14:17:32.950Z

Hi @Radoslav_dataplicity, are you directly affiliated with Dataplicity? Are you a dev or maintainer? Are you here to offer support or just promoting Dataplicity?

Have you seen the many threads here on the forum for Dataplicity?

If you check /etc/rc.local on your emonPi/emonSD you find the Dataplicity log files are recreated at boot time to counter the fact all log files are lost on reboot as they are held in RAM on this read-only image. Unless Dataplicity (or Supervisor) have altered the way the log files are only created at installation and never tested for after that, your installation above would have failed following a reboot.

Are you able to confirm if this issue has now been resolved in Dataplicity?
@barfle (Elliot Mackenzie) said back in January that they were working on a solution.

Radoslav_dataplicity · 2017-12-07T15:23:59.789Z

Hey @pb66 I’m part of Dataplicity and I’m here to offer support. I did see the other threads on this forum and I was hoping that by starting this one I could solve problems such as the one with the logs.

I have my emonPi here beside me and have done multiple reboots after going through the installation steps above. Everything works fine and emonPi is still connected.

pb66 · 2017-12-07T16:03:14.007Z

Radoslav_dataplicity:

I have my emonPi here beside me and have done multiple reboots after going through the installation steps above. Everything works fine and emonPi is still connected.

pb66:

If you check /etc/rc.local on your emonPi/emonSD you find the Dataplicity log files are recreated at boot time to counter the fact all log files are lost on reboot

I believe it is only working after a reboot because we have already put a rough fix in rc.local. I believe your guide and installation script would not work without those changes that were there prior to you calling your install script. This fix is less than ideal but this workaround is needed for Dataplicity to function on an emonSD.

Can you confirm if there has been any progress on this issue?

If you remove "supervisor/supervisord.log" from lines 12 and 13 of /etc/rc.local and remove/comment out line 20 completely does it still work after a reboot?

github.com

openenergymonitor/emonpi/blob/master/rc.local_jessieminimal#L11-L20


      
          sleep 1
          ifup wlan0
          
          # Will only run if /var/log is mounted in tmpfs
          if ( mount | grep "on /var/log "| grep -q "^tmpfs " )
          then
            for i in "redis" "apache2" "mysql" "openhab" "logrotate" "mosquitto" "supervisor" "emonhub" "emoncms"; do mkdir /var/log/"$i"; done
            for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do touch /var/log/"$i"; done
            for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do ""chmod 666"" /var/log/"$i"; done
            chown -R root:adm /var/log/apache2

Radoslav_dataplicity · 2017-12-07T16:24:41.950Z

I have made the adjustments as you suggested and emonPi is still visible online on dataplicity device list after a reboot. Would you like to give it a try as well if you don’t mind ?

pb66 · 2017-12-07T16:58:36.426Z

Thanks for testing. I don’t have an emonPi up and running right now. But it sounds like the issue might have been addressed, the only way to be sure would be to start with a fresh image and remove those bits from rc.local before running the install script and then check the log files after a reboot. Unless of course, you know specifically that this has been addressed.

Can you check if /var/log/supervisor/supervisord.log is there with the right ownership and permissions after rebooting.

barfle · 2017-12-07T17:24:03.656Z

Hi Paul,
Unfortunately we’ve just confirmed that your rc.local fix is still required…

The issue here stems from [feature request] options to deal with bad logging situation · Issue #709 · Supervisor/supervisor · GitHub. Dataplicity uses supervisor, but unfortunately upstream has concluded that this is a fix for the distribution, not for supervisor (I can see the point, though I don’t entirely agree with the conclusion). It does also appear that you have discovered a number of other packages that exhibit the same behaviour here - nginx, logrotate etc.

My question for you is what do you think is the correct solution for emonpi here? Short of a fix in the Raspbian repositories, is your fix here adequate do you think, or should we ask the Raspbian guys politely if they will adjust this in their distribution? Note that in general /var/log on Pi is not on tmpfs, and that the adjustment to tmpfs, while common, is in this case particular to the distribution/setup of emonpi.

Best,
Elliot.

pb66 · 2017-12-07T17:55:16.612Z

Hi Elliot, discussions keep popping up about this, the latest is on the “Mosquitto won't start on boot after raspbian and emonsd update” thread.

Whilst I firmly believe it’s in the hands of the SW providers to make their offerings as robust as possible and being able to overcome a deleted/moved log file would benifit their SW, I also understand we are unlikely to persuade the likes of MySQL, Redis and Apache2 to adopt a change so I also believe it to be a part of the RO optimizations to accommodate this anomaly that exists when moving the log files to tempfs. I do not like the idea of hard coding each SW’s requirements into the emonPi image, but that is how it is done at the moment.

IMO the read-only optimisations should include something to automate the process of replicating any additional log files at boot time regardless of the SW they belong to.

The reason I brought it up here was firstly for clarification as to whether that was still the case, and secondly to make it clear that following the guide outlined in the first post is dependent on the pre-existing “fix” (although I do not think this is a proper fix as the use of rc.local in this way is not recommended and it is depreciated in Stretch) just in case someone lands here via a different route, for example if they partially follow the self-build guide for the emonSD image they may well not have those entries in rc.local.

The guide (although clear and very helpful) only suits pre-built emonSD or non-read-only setups, additional steps are required if installed on a home-brew read-only OS based setup to ensure the logfiles are present before Supervisord service runs.

It would have been nice to have one less SW to worry about, but it’s not the end of the world as we have to accommodate others too as you point out.

glyn.hudson · 2018-01-26T18:01:58.158Z

Thanks @Radoslav_dataplicity

I’ve also tested setting up dataplicity using our off-the-shelf latest emonPi SD card image emonSD-26Oct17. It works great. All the issue mentioned above have been resolved.

If you don’t mind I will use these screenshots to post up a blog post and user guide page for how to use and setup dataplicity for remote access.

barfle · 2018-01-26T18:11:58.044Z

By all means
Please let us know if and when we can be of further assistance.

Best,
Elliot.

glyn.hudson · 2018-01-26T19:02:11.190Z

I’ve just create a page on our user guide detailing how to setup Dataplicity an an emonPi:

Guide | OpenEnergyMonitor

Remote Access

Remote access to emonPi

Please let me know if you spot any errors.

Note: I’ve just pushed up the page, it will take about 5min from when I post this for the link to go live!

JohnSchols · 2018-03-26T15:05:55.204Z

glyn.hudson:

I’ve just create a page on our user guide detailing how to setup Dataplicity an an emonPi:

Remote Access — OpenEnergyMonitor 0.0.1 documentation

Glyn,

Hi, I’ve probably got the complete wrong end of the stick. I was hoping this would let me see my local emoncms dashboard. From what I’ve seen I don’t think this is so? if that is the case should I uninstall Dataplicity?
John

glyn.hudson · 2018-03-26T15:34:14.110Z

JohnSchols:

Hi, I’ve probably got the complete wrong end of the stick. I was hoping this would let me see my local emoncms dashboard. From what I’ve seen I don’t think this is so? if that is the case should I uninstall Dataplicity?

John

Yes it will, you will need to enable a duplicity ‘wormhole’ to give you https access to your local emonPi. See the guide linked above.

JohnSchols · 2018-03-26T16:01:13.783Z

glyn.hudson:

Yes it will, you will need to enable a duplicity ‘wormhole’ to give you https access to your local emonPi. See the guide linked above.

Thats great, got it to work. Thanks

John

CelsoHenriques · 2018-05-26T08:27:01.073Z

Sorry to bring back the topic but i have a question about dataplicity and emoncms android app.

Using Dataplicity, am i able to use the android emoncms app to see the consumption of my emonpi?

Thanks

pb66 · 2018-05-27T11:33:53.771Z

The emoncms app should work fine using a dataplicity “wormhole”. The app doesn’t use any special api calls.

peter · 2018-05-29T09:27:03.512Z

I’m confused. Why would you use SSH and then hand stuff over to a 3rd party? You may as well use telnet. It’s not clear if dataplicity is a “dyndns” style redirector or a proxy? If the former, then great but if the later then people need to understand that their data is no longer secure.

CelsoHenriques · 2018-05-30T17:20:37.444Z

Thanks Paul! Going to try it soon.

Actually, its one of the things that is keeping me away from using it.

I am trying to find (to be honest) a way to use emoncms outside of my home network and keep away of another montly bill, in this case, on the raspberry pi that i have installed in my brother’s house.
In my emonpi, with all the purchases that i have made and the ones that i will in the future, i will, for sure have 4 or 5 years on emoncms.org, but not my brother. So, i was trying to find a way to avoid negative balance on emoncms.org.

@glyn.hudson:
Did the oficial stable emoncms android app should have the Portuguese translation? I got out from Beta Testing and found that the latest stable version in google play is 2.0.9 with date 14/07/2017 and portuguese is not present.

ian · 2018-05-30T18:29:04.429Z

From Dataplicity web site.

Dataplicity connects using client-initiated HTTPS, so it’s safe, encrypted and you don’t need to make specific firewall exceptions.

peter · 2018-05-31T07:47:07.963Z

Uh huh. So, you trust them with your private data that would otherwise be end-to-end protected between devices you control using SSH then? Cool.