Hooking and processing measured data before publishing to EmonCMS

Emoncms
, ,
1

Hi,

I am new to EmonCMS and EmonHUB, got my first EmonPi box last week and have been experimenting with pulse counter, which seems to be working perfectly. Later on, I will be adding a few EmonTH temp/humidity sensors, plus a few separate, Raspberry Pi’s gathering temperature data with 18b20’s. My intention is to inject their data to EmonCMS as well.

I have a few questions about the right way to do some data processing:

The pulse count on EmonPi is working OK, and the data is published to emonCMS. No problemo.

However, in addition to publishing raw data to EmonCMS, I would like to simultaneously trigger a process for storing these measurement data pairs of timestamp/pulse count locally, and to calculate an approximation of current power usage in Watts based on the few latest measurements (say, 5 or 10 most current). This figure would then be published as “power” measurement to EmonCMS to be used in dashboards. I cannot use the clip-on current sensors.

Should this be done in EmonHub or in EmonCMS - and where and how in those?

I suppose that injecting the temperature readings from other RPi’s will be a case of simple scripting, and there are probably many examples to follow. Are there any complications to be expected?

I have also a few additional ideas about calculating the cost of electricity for dashboards. There are a few components in our electricity price. First of all, the price consists of

  1. price of energy per kWh, either fixed, day/night, or hourly ELSPOT market price + margin, depending on your contract. There are even more options, since the price may vary by winter/summer and/or weekday/weekend. To make it manageable, one could (and should) make an hourly energy price table with all this taken into account, and use that for calculations.

ELSPOT prices are published in advance on the web, so they must be downloaded and processed in order to get the right figures for the table. Current VAT must be added to the ELSPOT price.

  1. price of energy transfer / network maintenance per kWh, again, similarly either fixed or day/night, winter/summer/weekend, etc. Another hourly price table should be constructed to determine transfer cost per kWh. VAT is included.

  2. a special energy tax per kWh, fixed number

  3. monthly fee for the company selling energy, VAT included. Used to calculate hourly or daily price.

  4. monthly fee for the company owning the distribution network, VAT included. Used to calculate hourly or daily price.

I have made some Excel (actually Openoffice/Libreoffice Calc) magic to work this out, and it could be transferred to make rather accurate price display for dashboard. But where to do all these calculations? What is the interface to make your own scripting for data processing?

I guess that is essentially my question - where and how to process and customize data feeds? How to hook and catch pulse count measurement at EmonPi/EmonHub?

BR,

Ari-Matti

2

To answer myself:

I ended up reading emonhub.log with a script run once per minute as a cron job, and searching for the latest published timestamp/pulsecount value pairs. I then take a few of them and calculate a running average for “power” value calculation.

To my disappointment, the timestamps seem not to be the actual pulse times but rather the count at pre-set intervals, which hurts the granularity of the derived power value quite significantly. The shorter the data sample period, the worse it gets.

Is there a simple - or even relatively simple - way to get hold of the EXACT time of the pulses? Millisecond precision instead of rounded one-second timestamp values (as published to emoncms) would help tremendously as well.