Dataplicity VPN

borpin · 14 December 2016 10:03

Came across this today https://www.dataplicity.com/. Might be useful for the EmonPi?

Bill.Thomson · 14 December 2016 17:00

Nice find Brian.

Now I can make changes to my RPis from my office. Works like a champ!

Dave · 14 December 2016 23:12

Yes very nice, like Bill very handy for accessing a SSH session on the work PC.

Regards
Dave

Jon · 15 December 2016 16:17

is there a cost for dataplicity?

pb66 · 15 December 2016 18:15

Apparently not!

There is no “fair usage” or maximum devices or maximum “online time” or anything along those lines. There is also no forum.

I looked at this not so long ago and whilst it looks fantastic for occasional non-dependent use, my concerns were that I could end up rolling out loads of devices spanning the length of the country, that were dependent on access via this 3rd party that could disappear over night or decide to implement charges or throttling etc.

There is no doubt this is a good idea, and the client/agent software is open source, but the server software isn’t (AFAICT), so the fact I cannot run my own instance and/or find any commitment or reassurance about the future of the service or the cost etc, prevents me using it on anything other than those devices I have easy physical access to unfortunately.

I stumbled across it as I have been working on a similar solution for my own use using reverse ssh tunnels over ssl. I’ve tried it and found it very easy to setup and use. The android app is really useful too. But the routing through a 3rd party with little or no disclosure makes me a little nervous.

One minor annoyance with it is having to enter a password to get access and then a password for elevated privileges (unless you take the risk of adding the dataplicity user to the sudoers group), I have become very attached to using private/public keys to automatically and very securely log in over ssh to many devices without using passwords.

But for the occasional tweak or update etc I think it could be a great addition to the emonpi image for most users, as it saves any messing around with any port forwarding and/or installing putty etc etc.

pb66 · 21 December 2016 17:31

I’ve received an email about future charges from dataplicity. After April 2017 only one free device per new user.

If you haven’t tried it yet, it might be worth a try registering to see if you might just qualify for the “early adopter” 10 free accounts ?

Although it still isn’t clear how many devices you can have on either the standard or the pro accounts, possibly “unlimited connections” means devices since the accounts are single user so the number of open consoles is likely to be low ?

I suppose for many users one connection is enough, once you have sshd’d into that one device you can ssh to any LAN devices behind the firewall that have sshd running and an open port for ssh.

borpin · 21 December 2016 21:40

Signed up before they implemented it so see what happens. Just not had time to do it before!

pb66 · 26 December 2016 13:16

Just a quick note on using this with emonSD, the service will fail to start after a reboot as the log files cannot be found.

This is due to the logs being held in RAM and lost on reboot on a read-only OS.

So aside from the obvious use of “rpi-rw” before using the install command provided by dataplicity, you must also add something like

if [ ! -d /var/log/supervisor ]; then
        mkdir /var/log/supervisor;
        touch /var/log/supervisor/supervisord.log
        chown -R dataplicity:dataplicity /var/log/supervisor;
        service supervisor restart
fi

to the /etc/rc.local file to ensure a logfile is recreated at each new boot, and then the service restarted again.

I have deliberately created a standalone entry in rc.local for dataplicity rather than adding it to the existing list of “officially included” softwares that need the logfiles created at boot because it is not “officially included” and therefore any automated updating of the existing code using regex could be disabled or result in this edit being over-written. Unfortunately if the rc.local file is over-written entirely during an update this edit will need to be re-applied.

barfle · 4 January 2017 13:11

Hi there,

Thanks for pointing out this issue and a workaround. We’ve tagged this to fix it up for you in our standard installation. Will post back when done.

Best,
Elliot
Dataplicity Founder

pb66 · 4 January 2017 13:34

Hi Elliot

This type of issue was fixed it emonHub by adding a line to the init script to create the application specific log directory if not found at start up.

github.com

emonhub/emonhub/blob/development/service/emonhub#L33

    
      
          SYSCONF_PATH=/etc/default/
          
          
# If there's no configuration file, die!!
          [ ! -f $SYSCONF_PATH$NAME ] && echo "Fatal: cannot find $SYSCONF_FILE$NAME config file!" && exit 1
          
          
# Pull in the config settings and echo them out for easy troubleshooting.
          . $SYSCONF_PATH$NAME
          
          
[ ! -f $EMONHUB_CONFIG ] && echo "Fatal: cannot find $EMONHUB_CONFIG" && exit 1
          [ ! -f $EMONHUB_PATH$NAME.py ] && echo "Fatal: cannot find $EMONHUB_PATH$NAME.py" && exit 1
          [ ! -f $EMONHUB_LOG ] && sudo mkdir -p "$(dirname "$EMONHUB_LOG")" && sudo touch "$EMONHUB_LOG" && sudo chown -R $DAEMONUSER "$(dirname "$EMONHUB_LOG")"
          
          
# PATH should only include /usr/* if it runs after the mountnfs.sh script
          PATH=/sbin:/usr/sbin:/bin:/usr/bin
          DESC="OpenEnergyMonitor emonHub"
          # Enter your installation path here
          DAEMON=$EMONHUB_PATH$NAME.py
          
          
# Specify command line arguments here
          DAEMON_ARGS="--logfile $EMONHUB_LOG --config-file $EMONHUB_CONFIG"

this avoids any additional install steps whilst ensuring the logfile can always be found without any impact on a “standard non read-only” OS.

Great to hear you are looking to resolve it, the rc.local workaround isn’t ideal.

glyn.hudson · 16 May 2017 23:22

Nice, I’ve just tested dataplicity. Works amazingly well, so easy to setup. I would be quite happy to pay $2 per month if I needed to use it on more than one device.

I’ve added the bash commands to setup the log files as suggested by @pb66 to the default emonPi rc.local. There is no harm in the log folder being created on all emonPi’s. The changes can be pulled in by running an emonPi update:

Wormhole works great to access local emoncms on port 80:

ian · 29 June 2017 14:01

Hi

I do not have an emonpi and I cannot use emonPi update as it is no longer showing on my administration page.
I have in the past used emonPi update and then altered the settings.php to match my setup.
I just have a local pi hosted emoncms. I have updated with git pull but dataplicity will not connect. Do I have to take specific steps to pull in the changes that Glyn refers to above?

Ian

glyn.hudson · 29 August 2017 15:42

Hi @ian

Are you running a DIY system instead of the standard emonSD build? If so you will need to make the changes manually to /etc/rc.local as the file will not be symlinked as as in emonSD. Here is the current emonPi rc.local

github.com

openenergymonitor/emonpi/blob/master/rc.local_jessieminimal

#!/bin/sh
#
# rc.local

echo" My IP address is":
hostname -I

# WLAN fix on RasPi 3B+ Stretch
rm /var/run/wpa_supplicant/wlan0
ifdown wlan0
sleep 1
ifup wlan0

# Will only run if /var/log is mounted in tmpfs
if ( mount | grep "on /var/log "| grep -q "^tmpfs " )
then
  for i in "redis" "apache2" "mysql" "openhab" "logrotate" "mosquitto" "supervisor" "emonhub" "emoncms"; do mkdir /var/log/"$i"; done
  for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do touch /var/log/"$i"; done
  for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do ""chmod 666"" /var/log/"$i"; done
  chown -R root:adm /var/log/apache2

This file has been truncated. show original

ian · 29 August 2017 16:39

Hi @glyn.hudson

I actually ran emonpi update but still could not connect.
I then manually copied the rc.local file but again no success.

In the end I set up remot3.it which solved my immediate requirement which was to have access while I was away for 10 days.

Regards

Ian

glyn.hudson · 30 August 2017 16:15

Did you reboot after copying rc.local?

After the reboot can you see if /var/log/supervisor/supervisord.log has been created? If so is there any log info?

$ tail /var/log/supervisor/supervisord.log

ian · 31 August 2017 16:15

Hi @glyn.hudson

I went through the process again copying the rc.local your referenced as I would like to use Dataplicity.

I rebooted and I have a supervisord.log created at reboot date and time but it is an empty file.

Emonpi shows up on Dataplicity under my devices but always with a red bar to the left.

My other Pi’s show up with a green bar to the left.

My guess is that Dataplicity is not running on my emonpi for some reason.

borpin · 3 September 2017 18:10

I had trouble with Dataplicity on one of my Pis. There were some permission problems IIRC. Go onto their support - usually pretty quick to respond.

(Edit deteted - should have read the thread more carefully). Issue was the supervisor log for me but just creating it solved it (not a RO FS).

Check the logs and the service to check DataPlicity really has started and not failed quietly.

johnbanks · 6 September 2017 16:44

I’ve had trouble with Dataplicity Wormhole locking up …

I took it up with Dataplicity and got a reply from the Founder …

Re_ Wormhole & Cloud Commander.pdf (16.6 KB)

The reply went over my noobie head, I’m afraid.
It may mean more to some of you guys

Dataplicity is super easy to install and a great resource/capability for working remotely.

ian · 7 September 2017 13:44

Fantastic

This solved my problem.

In addition to this change advised by Elliot from dataplicity

change the logging directory of supervisor to just be
/var/log/ instead of /var/log/supervisor/ (you can do this by modifying
/etc/supervisor/supervisord.conf).

You also have to edit rc.local referred to by @glyn.hudson in an earlier and take out the redundant supervisor/ directories

Regards

Ian

johnbanks · 13 September 2017 18:41

Help please …

My dataplicity connected emonTx + RPi (not an emonPi) is located 40 miles away and difficult for me to get to so I don’t want to screw things up,

Would someone please post a copy of the modified file /etc/supervisor/supervisord.conf

Also I’m quite unclear as to what the procedure is for modifying rc.local.

Thanks