I’m definitely guilty of that. I’ve been free-loading on emoncms.org for about 3 years I think and never spent a penny in the shop, so I too would like to add my thanks for keeping it free as long as you have. My abuse wasn’t deliberate, more laziness, so there’s nothing like putting a price on something to make you sit up and take notice. I’m slogging my way through my dashboards and discovering all sorts of redundant feeds that I’m working on deleting. In addition, my system pre-dates emonhub, so I suspect incorporating that in my chain would greatly reduce my burden on the server.
I suspect I’ll probably end up going self-hosted, but a clean-up of my setup is well overdue either way. I do have one set of extremely static feeds that I use to supply a diagnostics page:
Those things are useful when diagnosing problems, but otherwise not at all. There are 21 feeds there, and all the min/max processing is done on the front end (stored in NVRAM on the monitor). After ~3 years each of those feeds is 5MB big. Embarrassingly I push them up every minute. Most could probably be pushed every hour or even every day. Is there a better (cheaper?) way to display very static data about the health of your monitor?
So this is all your fault Dave
That might be closer to the truth than you realise ;-). I’m hoping Trystan doesn’t look too closely at my usage, or I’ll be kicked off well before 1st July. My account balance is starting to compete with the national debt!
For the work you’ve done STM, I’d like to think that you’ll be forgiven + some…
…but I do take your point!
Something that would solve a great amount of problems with resource usage both for emoncms.org and self hosted users alike is a temporary feed that stores information for a limited ammount of time, discarding the old data.
I’d like to have feeds updating every second for the last 24 hours for real-time usage viewing and troubleshooting, and even though I’m self hosted, it’s a large amount of data to keep accumulating forever.
Then I can have historical feed with 10s or less update rate,that can also be time limited (2 years for example) and keep cumulative data such as kWh per day since the beginning.
Your servers hosting emoncms.org must have lots of very old data at 10s rate that maybe meaningless to the users but they got to have that update rate to get an acceptable real-time usage view.
Thanks @dBC @Paul @rjsc
Would the ability to display emoncms inputs in the dashboard be sufficient for many of these items?
Yes I think this is a good idea, certainly worthwhile investigating. There is still the processing overhead to think about but I think we need to move to a storage strategy on emoncms.org that can do this kind of thing as well as the option to split across storage medium: 1st stage: RAM, 2nd stage: SSD, 3rd stage spinning disk archive.
At the same time I think its important to have the option to keep long term high resolution data for certain applications. So I think there needs to be these different options.
Maybe for self hosted too.
Hi Emoncms Team, I do understand that nothing is for free. In the last month I just start use emoncms.org and got much fun. I invest a lot days to understand and create my projects.
For me it’s impossible pay 200 GB because I use a lot of feeds.
So I need to move to a local solution I still have that running.
Is there any easy way to backup my projects for restore local?
Maybe you can do a small video how to migration works?
I am still frustrated with the short time window, and incomplete solution for the next 2 months.
Additional I will mention that many users outside English speaking countries will have some problems to understand this issue.
At last I would like say thank you for the inspiration and great solution with emoncm
Yes, that would be perfect. I’ve no interest in their historic values at all, just their latest value.
Hello @GuidoSo, first you have a little over 3 months I appreciate that might not change things much, do you have an emoncms installation running locally? on a raspberrypi? What is your current setup?
PS great to hear that you’ve found emoncms useful and are making good use of it!
thank you very much for take time for my request.
here the infos about my configuration:
Version9.8.28 | 2018.01.27 ModulesAdministration | App v1.1.0 | Dashboard v1.1.1 | EventProcesses | Feed | Input | CoreProcess | Schedule | Time | User | Visualisation ServerOSLinux 3.2.0-4-amd64 Date2018-04-25 08:57:35 CEST , 0 users, load average: 0.07, 0.03, 0.00 HTTPServerApache/2.2.22 (Debian) HTTP/1.1 CGI/1.1 80 MySQLVersion5.5.60-0+deb7u1 Hostlocalhost (127.0.0.1) Date2018-04-25 08:57:35 (UTC 02:00) StatsUptime: 38783 Threads: 1 Questions: 458292 Slow queries: 0 Opens: 220 Flush tables: 2 Open tables: 48 Queries per second avg: 11.816 MemoryRAM
Total: 496.87 MB Used: 199.78 MB Free: 297.09 MB Swap
Total: 1.21 GB Used: 243.37 MB Free: 991.63 MB DiskMountStats /
Total: 48.98 GB Used: 3.19 GB Free: 43.3 GB PHPVersion5.4.45-0+deb7u13 (Zend Version 2.4.0) Modulesapache2handler | bcmath | bz2 | calendar | Core v5.4.45-0+deb7u13 | ctype | curl | date v5.4.45-0+deb7u13 | dba | dom v20031129 | ereg | exif v1.4 | fileinfo v1.0.5 | filter v0.11.0 | ftp | gettext | hash v1.0 | iconv | json v1.2.1 | libxml | mbstring | mcrypt | mhash | mysql v1.0 | mysqli v0.1 | openssl | pcre | PDO v1.0.4dev | pdo_mysql v1.0.2 | pdo_sqlite v1.0.1 | Phar v2.0.1 | posix | redis v4.0.1 | Reflection | session | shmop | SimpleXML v0.1 | soap | sockets | SPL v0.2 | sqlite3 v0.7 | standard v5.4.45-0+deb7u13 | sysvmsg | sysvsem | sysvshm | tokenizer v0.1 | wddx | xml | xmlreader v0.1 | xmlwriter v0.1 | zip v1.11.0 | zlib v2.0 |
Thanks Trystan for providing this service. I’ve been using it for a few years now and have been meaning to shift it to a local instance but don’t have much linux experience. I guess this will ensure I pick up my game!
The charging mechanism is probably fine - I understand the need to keep it simple otherwise it gets too hard for a lot of people. I’d just rather put the money into buying an Iotawatt from your shop, so I think that’s what I’ll do.
Having said that, I’m happy to give a donation for all the years I’ve been using emoncms.org for free. How do I do that?
Looking forward to any updates to the sync script as I’ll be needing that…
Ok the sync module is now available for automatic download as part of the EmonPi/EmonBase update process. For those with an EmonPi, EmonBase or raspberrypi base station running the standard emonSD image you only need to run EmonPi or EmonBase update from the administration page, refresh the page and the Sync module will appear under the Setup menu.
Here’s a screenshot of what it looks like:
I’ve documented the steps for non emonSD setups on the sync module repository here: GitHub - emoncms/sync: In development: Download or upload emoncms feed data between local and remote emoncms server's. The Sync module does require redis and a Linux OS. If your unfamiliar with linux and command line I would definitely recommend using our pre-built emonSD image and a raspberrypi, theres more details on the image and how to download and write to an SD card here: emonSD pre built SD card Download & Change Log · openenergymonitor/emonpi Wiki · GitHub
This tool does not yet download the inputs, dashboards and other configuration. I will be working on this next.
Thanks @Brad_Schultz for the kind offer If you would like to buy a little emoncms credit when it becomes available in the shop that would be most welcome, it would always give you the option to use it in the future… but dont worry if that’s a bit complicated!
@GuidoSo let me know if I can expand on the instructions here GitHub - emoncms/sync: In development: Download or upload emoncms feed data between local and remote emoncms server's for installing the sync module
Hi Trystan, thats a good solution. Thanks a lot for this oportunity. I allready start to rebuild my monitoring. On one side i found out that with this way i do have a lot of to do`s , but the new building is also a oportunity to optimize all. So Thanks a lot for all, i think i have found my way and wil take that time for do it manually
Great job of you!
Trystan, thanks, I’ve had three solid years of use of Emoncms without paying a penny apart from the kit I’ve bought. I’d be quite happy to pay to continue service. Living a strange life across two countries, it’s great to have one database for both houses.
I’m another one who has created feeds “just for interest” so it will certainly make me do a tidy-up.
The pay-for-service option made me think of something else. I currently subscribe to a webcam host provider and a MQTT broker service, basic service only on both. Have you considered extending the service to cover such options? I have no idea of the scale or economics of it, but thought I’d raise the point.
Thankyou @derekb much appreciated. I see emoncms.org as sticking to the time-series storage and visualisation primary focus but then extended with apps and other tools that help make sense of that data. We would definitely like to add a MQTT broker to give the option to use MQTT to post data to emoncms.org, thats on our list of things to add.
Thanks again, Trystan
The introduction of billing on emoncms.org brings with it some interesting questions about monitoring system design (or IOT in a wider sense) which I think it would be good to discuss openly. We don’t want to push people to use and pay for emoncms.org where it’s not suitable, we only want people to use it if it is valuable. There are I think advantages and disadvantages to both remote and local data storage and visualisation and it’s likely a matter of choosing the right tool for the application, or a combination of both. Luckily there are many options with emoncms: emonpi/emonbase, diy local, self hosted remote, emoncms.org.
I started a dedicated thread on this topic here: Forum post: Emoncms local vs remote, thought I better link it in here so that it doesn’t get lost.
I came to emoncms in December 2016 after smart energy groups implemented a pricing strategy. I think requiring people to pay for stuff is fine. I really liked smart energy groups, but I found the pricing strategy somewhat complicated and it seemed to be more expensive than it was worth to me. You can see their prices here: https://smartenergygroups.com/pricing
It appears to be higher than what you are proposing and more complicated with different prices based on frequency of updates and calculations or not. It also is a billed service not like a cell phone top up plan.
I really didn’t want to use emoncms.org (because it felt a little wrong as I hadn’t bought any hardware, thought that changed last year), but I was having difficulty getting it to work locally. I eventually figured out how to make it work there but never got around to making it work locally. I guess now, I will have to decide how much access is worth and if I want to run it locally or use something else. I am also looking at Grafana and Influx, since I have a lot of sensors on lots of things and it might be best if I self host.
I fired a docker image of Grafana and it really looks great! Thanks for sharing!