Hi, I just learned about this system and am very interested. In my initial evaluation I’m trying to understand the system - with one thing being the protocol used to transmit data from a remote site to a master site. I’ve taken a tour of the source code in several repositories, and in the emoncms organization there’s a repository with documentation. But it wasn’t sure what mechanism is used to send data upstream – unless it’s the “/feed” URL, and it’s therefore JSON data format with a REST API (if I grokked it right).
I was wondering if you’re using MQTT over the Internet for data exchange?
In my case this isn’t just idle curiosity. In my day job we offer a commercial data monitoring system for solar. My current task is related to researching “DER Aggregation” meaning becoming an intermediary between utility companies and individual solar systems, and integrating solar with the rest of the energy grid.
Here in California, state policy is moving towards requiring that all solar inverters support “communication” over the Internet with the goal of building a system to coordinate energy production and energy storage in a way that supports energy grid stability. For that purpose they’ve collaborated with a bunch of solar inverter makers and others to develop a protocol - IEEE 2030.5 - that is very complex and very comprehensive, and has extensive capabilities for controlling inverters.
Data is usually posted via the input api so that it can go through the input processing defined by the user and those processes will then post the data to the feeds. You can post data directly to the feeds using the feed api’s if you choose but that isn’t considered “normal use”.
The documentation linked shows the http(s) api’s but there is also the ability to post using MQTT but the MQTT is not as mature or feature packed as the http inputs. for example you cannot have more than one emoncms user with MQTT, the current implementation is designed with a single home user in mind and is hardcoded as “user 1”.
There is also no timestamp included with the data in the current MQTT implementation so the data is timestamped upon arrival at emoncms, not a big problem for a local install but any network delays etc will impact the timestamps and cause errors in power to energy (or vice versa) calculations and play havoc on the fixed interval feed engines.
If you are shifting a lot of data the bulk input api is recommended to reduce traffic, load and processing times, this is not available via MQTT (yet?)
Anomalies can also creep in with some calculations involving individual MQTT topics, eg publishing to /power1, /power2 and /power3 doesn’t mean they are guaranteed to arrive in that order, so summing all 3 powers in power3’s processing might result in using stale data if power1 or power2 is a little slow being updated, the http input will process each input in turn as a batch so the order is maintained.