Pulse Monitoring Help

marcoslater · 2021-08-11T14:41:19.743Z

Hi,

I bought an EmonPi kit with the pulse monitor, the sales docs and features made it sound like it was a relatively easy setup to get going on graphing energy use, unfortunately my experience has been rather different and I am in need of some help to figure out what to do to get this to work.

I’ve spent about two hours trying to figure out what to do with the input to get it to work, but no luck yet.

I have a Carlo Gavazzi EM210 meter which has a red pulse that pulses every 0.01 kWh, under the current usage readout (~7KW, as it’s an office environment) it pulses between 3 to 5 seconds.

I’ve yet to figure out how to get this value into a usable kWh graph, current Watts usage and the rest. So far I have only managed to get a semi-reliable counter that increases, though even that was having some trouble due to the pulse happening so often.

I’m very new to this software, could someone give me a pointer as to what sources I would need to create to get the graphs I want?

Thank you.

Robert.Wall · 2021-08-11T16:41:01.982Z

Welcome, @marcoslater to OEM

marcoslater:

I have a Carlo Gavazzi EM210 meter which has a red pulse that pulses every 0.01 kWh, under the current usage readout (~7KW, as it’s an office environment) it pulses between 3 to 5 seconds.
… that was having some trouble due to the pulse happening so often.

That should be fine - it is well within the capabilities of the emonPi.

marcoslater:

So far I have only managed to get a semi-reliable counter that increases, though even that was having some trouble

I think your problem might be more to do with ambient light than pulse rate. Is the optical pulse sensor the one from the OEM Shop, and is it quite close to and accurately positioned over the LED on the meter, and is the meter in a relatively dark place, or if not, can you shade it in any way?

marcoslater:

I’ve yet to figure out how to get this value into a usable kWh graph, current Watts usage and the rest.

I don’t know what you have in mind for “the rest”, so I can’t help you with that. A pulse count isn’t the best source of data for power, because you must differentiate the count, i.e. derive the rate. That’s OK if you have a large increment in the count every 10 s (the interval at which the total count is reported), but if (say) you have a single pulse every 20 seconds, you will record zero power if you miss the pulse, but you’ll count 10 Wh for the 10 s in which the pulse arrived - that’s 3.6 kW. We always advise a c.t. in conjunction with a pulse counter if you are interested in the power consumption (as distinct from energy).

What I think you want, is this:
Go to Setup → Inputs.
On the Inputs page that shows the pulse count, click the spanner icon against the pulse count and you’ll see “emonpi process list setup”. You want to add processes.

Choose “Log to feed” from the drop-down list (it’s the default).
Choose “Create new” from the feed drop-down (also the default) and give it a name - the first part is emonpi (that’s fine) and for the second part it’s probably offering you “pulsecount”, but you can have something more appropriate if you wish. The final choice is the feed engine - this is the database where the data is stored. PHPFINA Fixed Interval No Averaging at an interval of 10 s is fine - it means it records the count every 10 s on the 10 s (by internet time).
Click “Add”. That gives you a Feed called (e.g.) “pulsecount”.
If you want power, you need to add another process.
Choose “kWh to power” from the feed drop-down and give it a name - probably “OfficePower” or something like that. All the rest can be the same as the first feed, so Add that feed.
That’s all you need initially. Make sure you click “Changed, press to save” at the bottom before leaving the page.
You can then go to Inputs → Feeds and you should see your two feeds. The count should steadily increase by 2 - 3 counts every 10 s, but expect the power to jump about by quite a lot each time, as the number of pulses counted in each 10 s period changes.
(If you wanted to record the pulse count and the average power over the last minute, you could make the interval for both feeds 60 s. That would give a much steadier average power reading - but with less detail.)

The final step is on the Inputs → Graphs page. You should see your Feeds listed under “emonpi”, the two columns of tick boxes refer to left and right Y-axes. Put the energy on one and power on the other, and after some minutes, you should see a graph start to appear.

I have not included any scaling, because I don’t know exactly what you want, but you have a starting point there. Don’t be afraid to try things out - that’s the best way to learn. (You can always delete the Inputs and the Feeds if you make a mess of it and want a fresh start - the Inputs will reappear as soon as new data comes in.)

marcoslater · 2021-08-16T12:47:41.263Z

Hi Robert,

Thanks for taking the time to write up a guide for us to use, my main goal for the EmonPi is to provide a day/week/month overview of our power usage in the form of a graph, both in watts used and Kwh (daily/monthly).

My understanding is that even though pulse monitoring is not ideal, it should work for this use case, as the granularity I seek is not high, 60s for instance is perfectly acceptable, as this is primarily to see trends and patterns within our power usage, not for instantaneous feedback.

I followed your guide, but was unable to get the right results, the graph swaps between 0W and 720000W of power usage, in a binary fashion, there’s no in-between.

My guess is that the power input is not quite designed for the pulse to happen every 0.01kWh but rather every Wh or such? And as a result provides some mangled data?

I’m still confused as to how you’re meant to take a “pulse every x” and convert it into the more appropriate values, I see in the guides you can multiply it by a specific amount, but that still doesn’t seem to work, no matter the direction we go.

The pulse graph has been running on it’s own for a little while now, and it’s definitely showing the correct patterns, i.e. where the graph shoots up significantly faster during the day than the night, which is correct for the office environment it’s deployed in.

It’s worth noting that the EM210 power meter it’s connected to does also show real-time Kw usage on it’s built in display, so we have a reference baseline to compare to, say if we had a minutely average, I could cross check it whilst physically looking at the readout and make an informed assessment of wether we’re in the right spot with our calculations.

Further, there’s no issues with the actual readout, I can confirm the EmonPi is getting every single pulse, the probe was purchased with the EmonPi and is the original one, taped directly to the light output on the reader, there appear to be no difficulties in it reading the pulses, as I can see the Kwh readout on the reader increase by 0.01 each time the LED on the probe pulses.
My original suggestion of it missing some was misguided as I thought that perhaps the 10s measurement would only read one pulse within those 10s or similar, I had not understood that it was simply an aggregate function of pulses within those 10s.

Let me know if you have any other suggestions on how we could tackle this readout, so far no luck.

Robert.Wall · 2021-08-16T15:07:34.820Z

marcoslater:

the graph swaps between 0W and 720000W of power usage, in a binary fashion, there’s no in-between.

As I tried to emphasise, that’s inherent in trying to get a rate from a value that’s reported every 10 s regardless. If one pulse represents 10 Wh, you will record 0, 3.6 kW, 7.2 kW, 10.8 kW, 14.4 kW and so on for 0, 1, 2, 3, 4 pulses per 10 s. There’s nothing you can do about that as far as the data recorded in the initial feed is concerned.

The algorithm for kWh to Power is: Take the difference between the present value and the last recorded value in the feed you will be using, multiply by 3,600,000 and then divide by the difference between the time now and the time recorded for the last value in the feed. You clearly need to scale that to get back to watts.

What feed interval do you have?

marcoslater · 2021-08-16T15:14:04.350Z

Hi Robert,

Currently I am trying it with an interval of 10 seconds, I am happy to go to 60 seconds though.

If we did 60 seconds, would that increase the accuracy of the average measurement over that minute, vs the accuracy of the 10s?

Accuracy is more important to me than available interval increments, so I’m happy to go with 60s if that’s the case.

Let me know what you’d like me to try.

Bill.Thomson · 2021-08-16T15:22:22.788Z

marcoslater:

EM210 power meter

The datasheet for that meter says RS-485 communication is an option.
If the second to the last character of the part number is an S, then the RS-485 option is present.

Ref: em210ds.pdf (959.5 KB)

If the option is present, it could be an alternate method of reading the meter.

Robert.Wall · 2021-08-16T16:36:46.724Z

I’ve been playing with various processes, and I can’t see an immediate solution to your problem. It appears that irrespective of the feed interval, everything is dictated by the time of arrival of the data, which is 10 s. So if you have a 60 seconds feed, it will be updated every 10 s but only the last value to arrive in the 1 minute slot will be recorded - which is not what you want.

I can see a solution - a new family of processes “If elapsed time <, skip next” etc, which would allow processing the next step only occasionally - but that does not exist. If it did, what you’d do is accumulate the pulse count as I presume you are doing now, then every minute, or half hour (or whatever) run a “kWh to power” which would write the average power over that interval into its feed.

How practical is Bill’s suggestion? The fundamental problem is emonCMS only has access to pulse data every 10 s, whereas for the meter itself, that’s down to somewhere in the tens of microseconds area. So power isn’t a problem - in the same way as it isn’t for your emonPi if you were measuring current at about 60 samples per cycle.
If you can get the data directly from the meter, then that will give you an accurate value for the power.

Or you could add a c.t. to the wire the meter is on.

@TrystanLea
Is there a way of doing what @marcoslater wants within emonCMS?

[Edit]
Looking at the code for kWh to power, it averages over the interval since the previous data arrived. If the destination feed interval is the same as the incoming data rate, the feed gets what @marcoslater is seeing now. If it’s greater than the incoming data rate, the feed gets a snapshot of the last average over the data rate - which is largely meaningless.

Is the obvious solution to make it only write to the feed if $timeelapsed is equal or greater than the feed interval for the power?
If the feed interval is the same as the incoming data rate, nothing changes. If it’s made greater, then power will be the average over that interval. Thus by choosing a suitably long feed interval for the power, the average can be made arbitrarily long up to the maximum feed interval of 1 day. I’m not sure what the value passed on would be in this case - presumably the last power value in the feed or the new value if it’s been updated. Anything else breaks backwards compatibility.

TrystanLea · 2021-08-16T20:00:58.756Z

Hello @Robert.Wall @marcoslater Yes you are right that the kWh to power process does not work well (or at all) for the pulse count coming from the emonPi. I did actually remove this process from emoncms.org and should remove it from the version of emoncms running on the emonPi/base as well as it probably leads to more confusion than its worth!

For pulse counts I would just record as an ever accumulating Wh or kWh feed. Using the Wh accumulator or kWh accumulator process in emoncms.

Then use delta mode in the emoncms visualisations and graph page to work out the daily consumption Calculating Daily kWh - Guide | OpenEnergyMonitor

To get an idea for power from a pulse count I would then use the graph module like this:

Select the cumulative pulse count feed
Set the interval to say 60s, put a tick in the Fix checkbox next to the interval.
Put a tick in the delta checkbox to calculate the difference between each interval
Next we need to enter a scaling factor to convert the pulse count in each 60 seconds into a power value. In my example below my pulse count is in watt hours. The scaling factor is equal to 3600 / time period = 60 in this case.

I’ve included a comparison with an actual power feed as measured with a CT here, which shows generally quite good agreement:

image1123×802 84.5 KB

Robert.Wall · 2021-08-16T21:01:56.919Z

So you’re not considering my suggestion then?

TrystanLea · 2021-08-17T07:48:59.340Z

Sorry @Robert.Wall I replied in haste last night. I see what you mean now, I can see that that would be a good solution as well. A complicating factor is how to deal with feed engines that dont have set intervals e.g PHPTimeSeries and MySQL. Perhaps a parameter could be passed to the process that would effectively do the same thing averaging over a set time period but being user defined from the input processing UI rather than derived from the feed…?

TrystanLea · 2021-08-17T07:53:08.444Z

Question is is it worth continuing with the inclusion of this process (and try to improve it) or should I just deprecate it? With my suggestion to use the Graph module you avoid the need to record a separate feed for power data and instead calculate the power on the fly, it has a bit more flexibility as you can change the interval as required to make the most sense of the data…

marcoslater · 2021-08-17T11:05:29.353Z

Hi,

Thanks again to both for contributing and trying to help me out here.

I’m a complete newbie when it comes to emoncms so I’m afraid the response here went over my head a bit, but it sounds like what I want to do is definitely possible!

So lets say I have said pulse counter active in an emonpi, I cleared all my prior attempts and am starting from scratch. I know the pulse happens every 10Wh of usage on the meter.

Given these, what steps would I need to take to get to a graph of power usage “W” and kWh, to then feed into the “MyElectric” app to fulfil my original goal with this EmonPi?

Many of the other guides I followed didn’t seem to work as I expected so far, so I would like to reach a conclusion that works via these posts to help the next guy who may have a meter like mine or similar misunderstandings about how to configure it properly.

TrystanLea · 2021-08-20T21:14:50.228Z

A post was split to a new topic: kWh to power input process design

SimonD · 2021-08-19T13:33:17.437Z

Hi @marcoslater - I’m coming late to this discussion, and as an utter newbie so far as emoncms is concerned.
But I share your interest in seeing those graphs of W and kWh vs time. Especially cumulative kWh, a.k.a. meter reading (the bar chart of daily kWh not so much - if desperate, I’d label the power axis kWh/day). I would have an idea how to get from pulse sensor output to such graphs were I doing the processing off-line, but my ignorance of how emoncms does its stuff makes me hold my tongue (for now?).
Cheers!

TrystanLea · 2021-08-20T20:30:42.387Z

6 posts were split to a new topic: Emoncms graph timing

marcoslater · 2021-08-20T16:55:58.282Z

Hey @TrystanLea, any chance you may have a moment to further explain your process?

Really hoping we can get this EmonPi set up to monitor our power usage, but am at the last straws here of trying to get it to work, and considering sending the hardware back as I just can’t get it to work.

I feel the answer is here, as you have shown it does work when set up correctly, but I can’t seem to get to that stage.

Robert.Wall · 2021-08-20T19:11:09.932Z

Sorry this thread got hijacked, I’ll try to add to Trystan’s reply. The parts I’m adding are in italics like this.

For pulse counts I would just record as an ever accumulating Wh or kWh feed. Using the Wh accumulator or kWh accumulator process in emoncms.

So forget about converting to power in the input processes. All you should have on the inputs page for the pulse count is (if you want it) a Log to feed followed by a × 10 and then a Wh Accumulator. (I’ve just tried that, and to my total amazement, it accumulates my pulses in tens of Wh - so your scaling problem with the 10 Wh per pulse is solved. Then, being an inquisitive so-and-so, I changed it to ×0.1, and that worked too. )

Then use delta mode in the emoncms visualisations and graph page to work out the daily consumption Calculating Daily kWh - Guide | OpenEnergyMonitor

To get an idea for power from a pulse count I would then use the graph module like this:

Select the cumulative pulse count feed
For this step, you need to go to the Graphs page and create a graph of the feed you created with the Wh Accumulator.
Set the interval to say 60s, put a tick in the Fix checkbox next to the interval.
This is below the graph. The longer the time, the better the resolution of the power value. You trade time accuracy vs power accuracy.
Put a tick in the delta checkbox to calculate the difference between each interval
Further down, possibly off the screen
Next we need to enter a scaling factor to convert the pulse count in each 60 seconds into a power value. In my example below my pulse count is in watt hours. The scaling factor is equal to 3600 / time period = 60 in this case.

That should be correct for you too - because we’ve already scaled the pulse counts to one count per 1 Wh (or more accurately, 10 counts per 10 Wh).

I’ve included a comparison with an actual power feed as measured with a CT here, which shows generally quite good agreement:

image1123×802 84.5 KB

TrystanLea · 2021-08-20T19:29:43.338Z

Thanks @Robert.Wall that’s much clearer!

Robert.Wall · 2021-08-20T19:34:09.057Z

It takes quite a bit of practice to ignore all the underlying knowledge you’ve accumulated over years and get to the state where you see it the way somebody who’s never seen it before does.