Following on from the forum thread: Emoncms local vs remote, a key drawback with logging data locally on an EmonPi or EmonBase is that it is harder to access data remotely while away from the home. Setting up remote access the traditional way with port forwarding is a relatively complex and can be insecure, see guide: remote access.
Newer services such as dataplicity are fantastic, providing full SSH access as well as HTTPS web view, we use them ourselves and recommend them in our guide. Dataplicity offer a single free account ideal for home users.
I wanted to see however if it would be possible to implement a minimal remote access solution, where all the heavy emoncms html, css & javascript files are loaded direct from the online remote server for fast load times and the requests for data from specific local emoncms API endpoints are passed to the local emoncms installation with only the relevant emoncms data being returned. I wondered if MQTTS could be used to do this…
With @emrys’s help we have got a concept of this up and running and it works surprisingly well! Here’s how it works.
- The local emoncms installation has an always running python client that connects and subscribes to a MQTT broker on the remote server - waiting for request commands.
- HTTP requests to the online server are translated to MQTT messages that contain the description of the request. These are published to the subscribing users request topic.
- The local client receives the request and translates it into a HTTP request to the local emoncms HTTP api. The response is then sent back to remote server via the users response topic.
- The remote server sends the response as a response to to the original HTTP request. This all happens in short enough time for the process to be synchronous (so far in testing).
- The remote server web view displays the result e.g feed list or graph.
The MQTT communication is all done over TLS for security. The remote server can be accessed over HTTPS.
System diagram
Client source code:
https://github.com/emoncms/remoteaccess-client
Server source code:
https://github.com/emoncms/remoteaccess-server
Future development
It like the idea of integrating this more closely into the emoncms.org service, it could be possible to access local feeds from an emoncms.org account directly and click on specific feeds to record or backup these feeds on the remote server. Feeds mirrored to emoncms.org would have a higher level of availability and data access performance - ideal for public dashboards. Feeds that dont need these additional levels of service could remain local with remote access and therefore not incurr any storage costs on the remote server. Input configuration and processing could all happen locally on the emonPi/emonBase.
This level of integration is however some way off and its not clear yet how the remote access approach we’ve developed here scales, its early days. For the time being this remote access option will remain slighlty partitioned from emoncms.org located at mqtt.emoncms.org but sharing the same account login. Otherwise, they are on different servers.
Trying it out
While this is still a work in progress it is now at a sufficient stage for early testing, and we would welcome feedback. I will post the installation guide in the next post and the user guide in the third.
