Dataplicity VPN

pb66 · 2016-12-15T18:15:23.493Z

Apparently not!

There is no “fair usage” or maximum devices or maximum “online time” or anything along those lines. There is also no forum.

I looked at this not so long ago and whilst it looks fantastic for occasional non-dependent use, my concerns were that I could end up rolling out loads of devices spanning the length of the country, that were dependent on access via this 3rd party that could disappear over night or decide to implement charges or throttling etc.

There is no doubt this is a good idea, and the client/agent software is open source, but the server software isn’t (AFAICT), so the fact I cannot run my own instance and/or find any commitment or reassurance about the future of the service or the cost etc, prevents me using it on anything other than those devices I have easy physical access to unfortunately.

I stumbled across it as I have been working on a similar solution for my own use using reverse ssh tunnels over ssl. I’ve tried it and found it very easy to setup and use. The android app is really useful too. But the routing through a 3rd party with little or no disclosure makes me a little nervous.

One minor annoyance with it is having to enter a password to get access and then a password for elevated privileges (unless you take the risk of adding the dataplicity user to the sudoers group), I have become very attached to using private/public keys to automatically and very securely log in over ssh to many devices without using passwords.

But for the occasional tweak or update etc I think it could be a great addition to the emonpi image for most users, as it saves any messing around with any port forwarding and/or installing putty etc etc.

pb66 · 2016-12-21T17:31:09.754Z

I’ve received an email about future charges from dataplicity. After April 2017 only one free device per new user.

2016-12-21 (1).png599×950 42.1 KB

If you haven’t tried it yet, it might be worth a try registering to see if you might just qualify for the “early adopter” 10 free accounts ?

Although it still isn’t clear how many devices you can have on either the standard or the pro accounts, possibly “unlimited connections” means devices since the accounts are single user so the number of open consoles is likely to be low ?

I suppose for many users one connection is enough, once you have sshd’d into that one device you can ssh to any LAN devices behind the firewall that have sshd running and an open port for ssh.

borpin · 2016-12-21T21:40:39.481Z

Signed up before they implemented it so see what happens. Just not had time to do it before!

pb66 · 2016-12-26T13:16:50.613Z

Just a quick note on using this with emonSD, the service will fail to start after a reboot as the log files cannot be found.

This is due to the logs being held in RAM and lost on reboot on a read-only OS.

So aside from the obvious use of “rpi-rw” before using the install command provided by dataplicity, you must also add something like

if [ ! -d /var/log/supervisor ]; then
        mkdir /var/log/supervisor;
        touch /var/log/supervisor/supervisord.log
        chown -R dataplicity:dataplicity /var/log/supervisor;
        service supervisor restart
fi

to the /etc/rc.local file to ensure a logfile is recreated at each new boot, and then the service restarted again.

I have deliberately created a standalone entry in rc.local for dataplicity rather than adding it to the existing list of “officially included” softwares that need the logfiles created at boot because it is not “officially included” and therefore any automated updating of the existing code using regex could be disabled or result in this edit being over-written. Unfortunately if the rc.local file is over-written entirely during an update this edit will need to be re-applied.

barfle · 2017-01-04T13:11:51.101Z

Hi there,

Thanks for pointing out this issue and a workaround. We’ve tagged this to fix it up for you in our standard installation. Will post back when done.

Best,
Elliot
Dataplicity Founder

pb66 · 2017-01-04T13:34:05.606Z

Hi Elliot

This type of issue was fixed it emonHub by adding a line to the init script to create the application specific log directory if not found at start up.

github.com

emonhub/emonhub/blob/development/service/emonhub#L33

    
      
          SYSCONF_PATH=/etc/default/
          
          
# If there's no configuration file, die!!
          [ ! -f $SYSCONF_PATH$NAME ] && echo "Fatal: cannot find $SYSCONF_FILE$NAME config file!" && exit 1
          
          
# Pull in the config settings and echo them out for easy troubleshooting.
          . $SYSCONF_PATH$NAME
          
          
[ ! -f $EMONHUB_CONFIG ] && echo "Fatal: cannot find $EMONHUB_CONFIG" && exit 1
          [ ! -f $EMONHUB_PATH$NAME.py ] && echo "Fatal: cannot find $EMONHUB_PATH$NAME.py" && exit 1
          [ ! -f $EMONHUB_LOG ] && sudo mkdir -p "$(dirname "$EMONHUB_LOG")" && sudo touch "$EMONHUB_LOG" && sudo chown -R $DAEMONUSER "$(dirname "$EMONHUB_LOG")"
          
          
# PATH should only include /usr/* if it runs after the mountnfs.sh script
          PATH=/sbin:/usr/sbin:/bin:/usr/bin
          DESC="OpenEnergyMonitor emonHub"
          # Enter your installation path here
          DAEMON=$EMONHUB_PATH$NAME.py
          
          
# Specify command line arguments here
          DAEMON_ARGS="--logfile $EMONHUB_LOG --config-file $EMONHUB_CONFIG"

this avoids any additional install steps whilst ensuring the logfile can always be found without any impact on a “standard non read-only” OS.

Great to hear you are looking to resolve it, the rc.local workaround isn’t ideal.

glyn.hudson · 2017-05-16T23:22:12.678Z

Nice, I’ve just tested dataplicity. Works amazingly well, so easy to setup. I would be quite happy to pay $2 per month if I needed to use it on more than one device.

I’ve added the bash commands to setup the log files as suggested by @pb66 to the default emonPi rc.local. There is no harm in the log folder being created on all emonPi’s. The changes can be pulled in by running an emonPi update:

github.com/openenergymonitor/emonpi

creat logfiles for dataplicity

committed 10:51PM - 16 May 17 UTC

glynhudson

+5 -3

Screenshot 2017-05-17 at 00.09.22.png2323×1700 763 KB

Wormhole works great to access local emoncms on port 80:

Screenshot 2017-05-17 at 00.19.08.jpg1708×835 90.4 KB

ian · 2017-06-29T14:01:56.984Z

Hi

I do not have an emonpi and I cannot use emonPi update as it is no longer showing on my administration page.
I have in the past used emonPi update and then altered the settings.php to match my setup.
I just have a local pi hosted emoncms. I have updated with git pull but dataplicity will not connect. Do I have to take specific steps to pull in the changes that Glyn refers to above?

Ian

glyn.hudson · 2017-08-29T15:42:43.263Z

Hi @ian

Are you running a DIY system instead of the standard emonSD build? If so you will need to make the changes manually to /etc/rc.local as the file will not be symlinked as as in emonSD. Here is the current emonPi rc.local

github.com

openenergymonitor/emonpi/blob/master/rc.local_jessieminimal

#!/bin/sh
#
# rc.local

echo" My IP address is":
hostname -I

# WLAN fix on RasPi 3B+ Stretch
rm /var/run/wpa_supplicant/wlan0
ifdown wlan0
sleep 1
ifup wlan0

# Will only run if /var/log is mounted in tmpfs
if ( mount | grep "on /var/log "| grep -q "^tmpfs " )
then
  for i in "redis" "apache2" "mysql" "openhab" "logrotate" "mosquitto" "supervisor" "emonhub" "emoncms"; do mkdir /var/log/"$i"; done
  for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do touch /var/log/"$i"; done
  for i in "emoncms.log" "mysql.log" "mqtt_input.log" "redis/redis-server.log" "service-runner.log" "mysql/error.log" "apache2/error.log" "supervisor/supervisord.log" "ntp_update.log"; do ""chmod 666"" /var/log/"$i"; done
  chown -R root:adm /var/log/apache2

This file has been truncated. show original

ian · 2017-08-29T16:39:04.603Z

Hi @glyn.hudson

I actually ran emonpi update but still could not connect.
I then manually copied the rc.local file but again no success.

In the end I set up remot3.it which solved my immediate requirement which was to have access while I was away for 10 days.

Regards

Ian

glyn.hudson · 2017-08-30T16:15:05.480Z

ian:

I then manually copied the rc.local file but again no success.

Did you reboot after copying rc.local?

After the reboot can you see if /var/log/supervisor/supervisord.log has been created? If so is there any log info?

$ tail /var/log/supervisor/supervisord.log

ian · 2017-08-31T16:15:21.980Z

Hi @glyn.hudson

I went through the process again copying the rc.local your referenced as I would like to use Dataplicity.

I rebooted and I have a supervisord.log created at reboot date and time but it is an empty file.

Emonpi shows up on Dataplicity under my devices but always with a red bar to the left.

My other Pi’s show up with a green bar to the left.

My guess is that Dataplicity is not running on my emonpi for some reason.

borpin · 2017-09-03T18:10:51.149Z

I had trouble with Dataplicity on one of my Pis. There were some permission problems IIRC. Go onto their support - usually pretty quick to respond.

(Edit deteted - should have read the thread more carefully). Issue was the supervisor log for me but just creating it solved it (not a RO FS).

Check the logs and the service to check DataPlicity really has started and not failed quietly.

johnbanks · 2017-09-06T16:44:49.394Z

I’ve had trouble with Dataplicity Wormhole locking up …

Selection_0991327×524 31.9 KB

I took it up with Dataplicity and got a reply from the Founder …

Re_ Wormhole & Cloud Commander.pdf (16.6 KB)

The reply went over my noobie head, I’m afraid.
It may mean more to some of you guys

Dataplicity is super easy to install and a great resource/capability for working remotely.

ian · 2017-09-07T13:44:55.415Z

Fantastic

This solved my problem.

In addition to this change advised by Elliot from dataplicity

change the logging directory of supervisor to just be
/var/log/ instead of /var/log/supervisor/ (you can do this by modifying
/etc/supervisor/supervisord.conf).

You also have to edit rc.local referred to by @glyn.hudson in an earlier and take out the redundant supervisor/ directories

Regards

Ian

johnbanks · 2017-09-13T18:41:22.829Z

Help please …

My dataplicity connected emonTx + RPi (not an emonPi) is located 40 miles away and difficult for me to get to so I don’t want to screw things up,

Would someone please post a copy of the modified file /etc/supervisor/supervisord.conf

Also I’m quite unclear as to what the procedure is for modifying rc.local.

Thanks

ian · 2017-09-13T18:47:34.420Z

I am not clear. Can you already connect via Dataplicity?

johnbanks · 2017-09-13T20:57:22.144Z

Hi …
I can connect to & use the dataplicity terminal where you enter su pi and the password.
However when I use Wormhole which takes you to the emonCMS login, it locks up - see my post in this thread 3 or 4 up.
Thx for yr response

ian · 2017-09-14T07:59:13.875Z

Hi

I don’t use Wormhole. My problem was that I could not get Dataplicity to connect.

However this is my supervisord.conf

; supervisor config file

[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0700 ; sockef file mode (default 0700)

[supervisord]
logfile=/var/log/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; (‘AUTO’ child log dir, default $TEMP)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket

; The [include] section can just contain the “files” setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files cannot
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf

beaylott · 2017-09-17T22:39:35.427Z

Also check out resin.io … although the use case is slightly different. I have been using it for deploying stripped down emonhub RPI nodes forwarding temperature/humidity data. They are promising to open source a lot of their platfrom in coming months.