I use Dataplicity to access 5 remote instances of RPI/emonTx.
It works well. For example, from my Windows PC, I can set up a Dataplicity Porthole on any of the remote RPi/emonTx’s and then using WinSCP download an emon Export Backup to my Windows PC.
I would like to get a LOCAL RPi to do what my Windows PC does – set up a Dataplicity Porthole on a remote RPi/emonTx and then to an download Export Backup using scp (rather than WinSCP).
Has anyone managed to achieve this? And if so, how?
Not something I have done, but would also be interested if someone has this working
By way of update …
I posed the question to Dataplicity support on 12 Jan but as yet have heard nothing.
In the meanwhile, I’m trying out remote.it which I think can access a remote emonTx/RPi from a local Linux computer (eg: an RPi) rather than only from a local computer running Windows/Mac.
remote.it:
• Has extensive documentation
• From my Windows PC, I can use it to puTTY into a remote emonTx/Rpi and also download files using winSCP
• (have not yet tried this from a local RPi tho’)
• The behaviour when accessing the emoncms web interface is quirky and Support at remote.it are investigating. (Had this problem 2 years ago when first using Dataplicity)
• Gary M at Support is super responsive
• The interface seems not quite as slick/convenient as Dataplicity but that might be because I’m on a learning curve
• The monthly billing from Dataplicity is clear and very prompt
• Looks to be half the cost of Dataplicity for multiple remote instances
Re the quirky behaviour when accessing the emoncms web interface:
On first access I get what’s shown in ScreenShot 193 …
If I then click on the top left icon, I get ScreenShot 198 …
… and can then log into the remote emoncms but note there’s a security warning.
Gary M at Support is on the case.
I’ll keep you posted on progress.
By way of update …
I’ve had the following from Support at remote.it …
Hi John,
I’ve installed the emoncms package and reproduced your issue. I can’t say that ALL of the problems stem from the cause I will describe to you, but certainly some of them do.
#1 the ecomcms program is written similar to some router configuration pages, in that content is assumed to be available relative to the IP address/URL that was used to access the device.
#2 So when the login page wants to go find a file, it references it using:
href=“http://jgnaxrxw.p22.rt3.io/Theme/basic/favicon_emonpi.png”
and in this case, the reverse proxy address we provide is an https endpoint. But the code accesses it using http.
You can edit the URL in your browser and make it “http” instead of “https”. It will take you straight to the login screen, however it thinks that the connection is unencrypted because we are using http. In fact, the connection from our proxy server to the device is still encrypted, however we are just moving the issue from one place to another.
So, you could try changing the code in all the pages that reference local graphics or files (to say “https” instead of “http”), but I’m willing to bet you don’t feel like doing that.
Another thing you can try is to use the desktop application which is currently in Beta testing. The desktop application creates peer to peer connections rather than proxy connections. I’ve tested that with the emoncms running on a Pi and it does not generate that security warning. That’s as far as I went.
See:
Desktop Beta Program - Desktop Beta - Remote.It ForumI suggest you try the Desktop application and let me know if this gets you over that particular hurdle.
Let me know if you have any other issues or questions.
Thanks
My take is - a work around that bypasses/ignores the Firefox Security Warning seems sensible in the knowledge that the communication from remote.it is secure.
I’ll continue my experiments with remote.it. If it leads to being able to download an export from a remote emonTx/RPi to a local RPi then great - as that was the challenge I was looking to find a solution for.
Any comments?
I must be getting old - is this not the right way to do it anymore?
- is this not the right way to do it anymore?
… I’m not qualified to answer that question.
However assuming an objective is to continuously improve the utility/functionality of emoncms then, I think, the ability to seamlessly work with the likes of Dataplicity (which it does) and remote.it would be very useful.
So please excuse the perhaps dumb question – is it possible for emoncms to be agnostic and accept requests, whether http or https, to access the web log-in screen?
Interesting, I wonder how dataplicity manage to make it work fine with https?
My point was that I think that their statement is rubbish, but I could be (old and) wrong.
Currently I run (and pay for) 5 instances of dataplicity running on remote emonTx/RPi’s.
Dataplicity seems to do things differently in that it is necessary to download/install the dataplicity agent program on each remote instance.
This from their website …
https://docs.dataplicity.com/docs/how-it-works
And here’s a link to a 3 page Raspberry Pi Forum posting that was initiated by the dataplicity developer …
https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=117100
It is not too revealing – proprietary? Lots of comments about security. And a ref to zerotier which looks intriguing.
And the support guy at remote.it has suggested that I try their peer-to-peer arrangement which I must do.
The reason for my interest in all this? – having recently lost a lot of data because of a power cut at the remote site, I would like to set up a local RPi that periodically downloads the periodic export backups made by a remote emonTx/RPi. And that cannot be done with dataplicity to the best of my knowledge.
I wonder if you might not be better to pay for a dropbox account or use Google Drive and upload backups to that.
Example Gist (untried just one of the first google results)
By way of update …
I’ve now tried remote.it’s Desktop application (beta).
It’s super - from my Windows 10 PC, I can now:
Connect straight to a remote emoncms web log-in page without any security warnings
Via SSH, access a remote emoncms and ‘do things’
Connect via winSCP and download files from a remote emoncms - peer to peer connection is fast – a 120MB Export file downloaded in seconds
And the connection does not need to be always on
Their Desktop application is also available for Mac, RPi & Ubuntu Linux
The cost is more than reasonable – free for personal use. This is broad in scope as defined by their Fair Use policy – extracted below:
Each Customer may have:
one Free account
You are unlikely to be affected by this Fair Use Policy if:
Your Free Subscription Plan does not exceed 10 Devices and/or 250 daily Connections and there are no connected Devices for Business use
My most recent question to Support at remote.it was replied to in 9 mins.
PS: This Forum discussion is titled ‘Dataplicity’ – perhaps it would be more appropriately titled as ‘Remote Access’?
@johnbanks - great find.
2 immediate thoughts on signing up.
Their website is not compliant with the Planet49 judgement on cookies (just accept or else).
They have used my Google profile pic without my explicit & informed consent.
I am therefore a little wary of their privacy practices if they get such basic things wrong.
[edit]
Third thing, no 2FA offered. Even with a stupidly secure password, I’ll not expose any of my devices via this.
[edit2]
On profile pic, actually it is worse than that, it has picked the pic up from and unrelated social media account.
Question about remote.it – Do you know if it provides a “wormhole” to a Pi running emoncms that will accept HTTP in addition to HTTPS? My emoncms server is in my house and I use the Dataplicity wormhole feature to send it data from outside my LAN using HTTPS requests. But Dataplicity requires HTTPS and refuses HTTP requests. One of my external devices (an IoTaWatt) cannot send HTTPS so Dataplicity doesn’t work for this case. I would love a utility that allows me to post HTTP and HTTP to my emoncms server without having to open port 80 on my home router.
By way of background …
I’ve been using dataplicity for a couple of years to access a remote site with 5 instances of emonTx/RPi running on the remote network. So, as & when needed, I can remotely access the web interface of each emonTx/RPi and also download data files.
I’ve just become aware of remote.it and established that it can do just what dataplicity has been doing for me.
If I’ve correctly understood your situation (and I have no familiarity with IoTaWatt – do they have a Support Forum?), it is quite different to mine. You are posting 10? sec data from a remote site to your local instance of emoncms?
As a suggestion – contact Support at remote.it – I have found them very helpful and extremely prompt in their replies.
A lateral thought occurs – could you install an RPi emoncms at the remote site (where the IoTaWatt is) which collects the 10? sec data. Then it would just be a case of remote emoncms ‘talking’ to your local emoncms. I wonder if that might not simplify things – just a lateral thought and perhaps I’ve got the wrong end of the stick.
I do not use social media and my ‘knowledge is of a meagre and unsatisfactory kind’ to quote Lord Kelvin 
But notwithstanding, here goes …
Nothing is perfect, all things are relative/comparative. The OEM website under Resources – Setup/Chapter 13 – Remote Access suggests dataplicity as the solution. This begs the question – is dataplicity more secure and a greater respecter of privacy than remote.it? I can’t answer that question.
Logic suggests that remote access (reverse proxy, peer to peer or whatever) must involve a man-in-the middle. Is that incorrect?
During remote access, one is more exposed to bad actors. Strong passwords & two factor authorisation are defences. But the ultimate defence is – don’t do remote access … goodbye IoT? To the best of my knowledge, dataplicity is an always on remote connection whereas remote.it can be an only on when needed connection.
Best practice is not fixed but constantly evolving for organisations/businesses. Seems logical therefore that, at any specific point in time, an individual organisation/business may not be at current best practice level. Intent & direction of travel are perhaps important considerations?
Privacy … covers quite a spectrum from the total recluse to the Hollywood star. I don’t use social media and have ad blockers installed but without my email address being publically known, you & I would not be having this conversation, for example.
As to a personal icon/profile pic – I’m a neophyte and don’t have one. This Forum has assigned me one – did the Forum first check whether I had one on another website?
Just my thoughts …
They do. It’s at https://community.iotawatt.com/
To mis-quote Oscar Wilde, Once is unfortunate, twice is careless (third is downright bad management).
3 strikes and you’re out IMHO.
Yes and YMMV, but if I’m going to use an account for a service that allows access to my LAN, 2FA is an absolute minimum now. Writing this caused me to go and look at my Dataplicity account and update the password (I now use Bitwarden and completely random passwords). However, their 2FA is SMS based and while better than nothing, is not secure, so again that is a non-starter for me. The device I had registered has been offline for a while, but I will not be registering another one.
IoT does not have to exit your personal LAN.
Once I get my new Home-Assistant setup running, I’ll install the WireGuard add-on for a self hosted VPN solution.
Yes true, but the account login is secured using 2FA (via GitHub) and it is an explicit and informed decision to share that information.
I do recognise I am getting more paranoid; I’m at a point whereby much of my internal traffic is over HTTPS!
This forum thread has fizzled out into a somewhat detailed consideration of security/privacy.
However if one needs remote access then this will be a security/privacy risk and so it’s a matter of minimising that risk - in my opinion.
Notwithstanding that risk, you’ve expressed an interest in solutions that would enable an Export Backup existing on a remote RPi/emonTx to be downloaded to a local RPi.
Using remote.it and the following script, it is possible to do just that and the script can run in the background as a cronjob not needing to enter usernames or passwords. I’ve garbled my personal info …
#!/bin/sh
## Purpose:
## To use remote.it to connect from RPi A (remote) to another RPi B (local)
## Then to copy a file from RPi A to RPi B
## The task to be completed without the need to input user names or passwords
## Hence - this task can be run as a cronjob on RPi A
## Pre-requisites:
## connectd installed on both RPi A and RPI B
## A provisioning or configuration .txt file must exist on RPi A. This defines the Service ID of the target RPi B. Set autoconnect as 2
## An ssH Service created on both RPi A and RPi B
## id-rsa keys from RPi A must exist on RPi B or be copied to RPi B /home/pi/.ssh/authorized_keys
## Now begin ...
## Define key parameters
DEV_KEY="Q~~~~~~~~~~~~~~~~~~~~5"
USERNAME="j~~~~~~~~~~~~~k"
PASSWORD="~~~~~~~~~~~~~"
DEVICE_ADDRESS="80:~~~~~~~~~~~~~~:1C" ## GUID/ssh Service ID for RPi A
HOSTIP=
## Connect to remote.it and get the response json
foo=$(curl -X POST -H developerkey:"$DEV_KEY" -H Content-Type:application/json \
-H Cache-Control:no-cache -d "{ \"username\":\"$USERNAME\", \
\"password\":\"$PASSWORD\" }" https://api.remot3.it/apv/v27/user/login)
## Extract the response token
foo=${foo##*token'":"'}
foo=${foo%%'","'auth*}
TOKEN=$foo
echo $TOKEN ## This line CAN BE REMOVED after de-bugging
## Get the connection json
foo=$(curl -X POST \
-H "token:$TOKEN" \
-H "developerkey:$DEV_KEY" \
-d "{\"wait\":\"true\",\"deviceaddress\":\"$DEVICE_ADDRESS\", \
\"hostip\":\"$HOSTIP\" }" \
https://api.remot3.it/apv/v27/device/connect)
## Extract the CONNECTION_ID
foo=${foo##*connectionid'":"'}
foo=${foo%%'"}'}
CONNECTION_ID=$foo
echo $CONNECTION_ID ## This line CAN BE REMOVED after de-bugging
##
## NOW DO STUFF ...
## For example - copy a file from RPi A to RPi B
##
date=$(date +"%Y-%m-%d")
scp -i /home/pi/.ssh/id_rsa -P 33300 /var/opt/emoncms/backup/emoncms-backup-$date.tar.gz [email protected]:/home/pi
## Finally close the connection
curl -X POST \
-H "token:$TOKEN" \
-H "developerkey:$DEV_KEY" \
-d "{\"connectionid\":\"$CONNECTION_ID\", \
\"deviceaddress\":\"$DEVICE_ADDRESS\" }" \
https://api.remot3.it/apv/v27/device/connect/stop
exit
The first requirement is to sign up for a remote.it account. This is free and one user with up to 10 devices gets free use under their Fair Use Policy.
Their Guide is comprehensive covering Windows, Mac, Linux and RPi which makes it a bit difficult to follow. Their Support is super responsive.
The first step is to install connectd on each RPi and then using an interactive menu to define each Device (RPi) and the Services you require on each Device. The script above requires the ssH service but also install the web service (you can then access the emoncms Web Interface on the remote RPi ).
I should point out that if REMOTE RPi A is at a remote geographical location then you’ll need to make a ‘service’ visit to perform the above steps. But once done, you’ll have remote access to RPi A and the rest of the setup can be done back at ‘local base’ – in my case from a Windows laptop with an ssH terminal for each RPi open on my desktop.
The next step is to create a provisioning or configuration file on RPi A saved in /home/pi and made executable. The Guide has details. You’ll need the address of the target (ie: RPi B’s –ssH Service). Again follow the Guide. And do use autoconnect 2.
At this stage it’s a good idea to run connectd -f name.of.yr.file.txt & on RPi A from an ssH terminal. You’ll see the connection being made and that things are working to that point.
The script requires 2 bits of data … your Developer Key DEV_KEY so follow the Guide to get that and the DEVICE_ADDRESS which is the ssH Service ID for RPi A. Again follow the Guide.
Finally, in order to avoid entering user names and passwords, do the following on RPi A ssh-keygen Just press ENTER when asked for a passphrase and accept the default location which is … /home/pi/.ssh/id_rsa
Now carefully copy & paste the pi@RPi A key from RPi A /home/pi/.ssh/authorized_keys to RPi B /home/pi/.ssh/authorized_keys
You can now run the script on RPi A (after making it executable) or incorporate it into a cronjob, etc.
In summary – the script opens the remote connection, securely (scp) copies an emoncms-backup export file with today’s date and then closes the remote connection. In my case, a 150MB export file takes 6 mins to copy.
Hope this of interest
Please take my post above with a pinch of salt.
As I work more on this, I’m having problems …
Remote RPi A is connecting with RPi B on my local network but the exchange of rsa keys is failing.
Will update you on this.
In the meanwhile I’m enjoying the functionality ot remote.it - from my Windows laptop I’m accessing remote emonTx/Rpi’s via both the emoncms web interface and ssh terminals.