A plea for better gas monitoring options in a future release

MikeSims · 2016-05-26T11:37:57.561Z

I’m a relative newbie here, and still trying to understand much of the detail of how the various systems and subsystems hang together in emoncms/emonpi/firmware, but there is one addition I would like to recommend be available in a future release, given the OpenSource ethos, and the committment to energy monitoring (and monitoring of resources in general), and that is to include some functions that are relevant to pulse-based metering systems, and to gas metering in particular.

I was a bit surprised after I had my electrical monitoring up and running, and was eyeing up my gas meter, when I realised that, with all the attention to detail being paid to the measurement, analysis and presentation of electrical energy, that the equivalent for gas-based energy was not even basic. I know that stems largely from the fact that virtually every gas meter uses pulses of some sort, but you can derive more information from a stream of pulses than is currently done in emoncms.

This is important, at least to me, because in my house, I typically consume around three times the energy in kWhours in the form of gas as I do in electricity, over the course of a full year. This is largely to do with the use of gas-fired central heating and hot water in a house in Scotland which can see a lot of cold days, obviously especially so in Winter. My house is by no means untypical, so it appears that, for an energy monitoring organisation, it is pretty largely ignoring a major energy component consumed by a large fraction of premises across the developed world. I could make a similar argument for oil-based energy usage, too, but I’ll leave that for others to make their own cases.

So sticking to gas as context, here is my take on it. Do feel free to critique this, I am entirely capable of making mistakes, or misunderstanding the capabilities of emoncms/emonPi, or being a total idiot, or even living on the wrong planet. I am happy to be corrected, but gently! (Fingers crossed.)

On the detection of a pulse, you know from the context of its source what it represents, in my case the consumption of one cubic foot of gas. You can timestamp it and record it in a feed, and that can be displayed in a graph. If it was recorded at fixed intervals (phpFINA), you will see a series of isolated thin bars all of the same height in a histogram, with empty gaps of varying width between them, essentially a stream of ones and zeros.

Once you have more than one timestamped pulse, you can derive the duration between adjacent pairs, and by dividing the Joules in one cuFt (1146053J) by the duration preceding a given pulse you can derive a rate of energy usage in Watts for the consumption over that period. Those resulting Watts are relevant for every datapoint between the relevant pulses.

A. Emoncms currently doesn’t let you derive that duration using the standard Input processing, and neither does the standard firmware in the emonpi nor emontx, because they do not send pulsegap data, only pulsecount data. If you want a duration, you have to modify either the firmware to track it and send it alongside the pulsecount (which is information that is as pertinent to pulses as counting them), or modify or create suitable Input process functions to derive the durations from the accumulating pulsecount, and for that you need access to timestamps, or at least the current time, to do so, and I submit that that capability should be made available to the user in the processlists.

B. Even if you manage to derive the Watts usage as above, emoncms doesn’t currently let you store that data properly in a feed, because the Watts data for a period can only be calculated at the end of that period, but are relevant for every datapoint between the current pulse and the previous one, but emoncms doesn’t let you write that data back into those intermediate datapoints, so again you need a custom feed function that can do so, and it needs access to a duration to let it determine how far back in the feed it needs to go to amend otherwise empty datapoints. Without this, the gas Watts data in a feed is very unrepresentative of the measured reality.

C. None of the existing apps (Myelectric, mysolarpv, myenergy, etc), cater for gas. They do cater for Watts, so can handle gas Watts, but the labelling assumes all Watts are electric, either grid electric, solar-electric, wind-electric, etc. So there is a need for a mygas, myoil, or even a mydualfuel (with oil/gas labelling choice).

D. A consequence of having a possibly frequent rewriting of gas Watts history is that a moving line graph such as that in mysolarpv can get rapidly out of date if the 5 second refresh doesn’t also carry out its own rewrite at the righthand end of the graph. If a fresh pulse arrives, the Watts value spike is put up immediately, and the previous npoints in the graph should be updated, too where relevant, without having to force a reloading of the data from the feed every 5 seconds.

E. More generally, in a Low-write context, the use of feeds to hold temporary information for an intermediate calculation should be discouraged, when a memory-based alternative can be made available, and a wider range of mathematical operations, or variations permitting different choices of the values or constants they operate on, would improve flexibility. These would reduce or eliminate the need for a user to get stuck in creating her own functions when the standard ones don’t quite let her carry out the operation or calculation she needs.

F. Erm, that’s me done, over to you.

glyn.hudson · 2016-05-26T21:49:25.814Z

We would love to support gas metering, however here are the main reasons why it’s taken a backseat:

Both @TrystanLea and I live in houses with no mains gas. A lot of rural properties in North Wales don’t have mains gas. The presents testing and dev challenges
There is a lot of variety between gas meters which makes developing a common solution difficult
There are laws against tampering with gas meters due to the explosive nature of the beast, this includes strapping on sensors. It’s possible to obtain certification but it’s difficult and prohibitively expensive (I’ve talked to people with experience in this field)

Some efforts have been documented in the Building Blocks.

The simplest solution if you have a modern gas meter with an optical pulse output is to use an optical pulse sensor with an emonTH.

Robert.Wall · 2016-05-27T00:17:18.065Z

glyn.hudson:

A lot of rural properties in North Wales don’t have mains gas.

I think it would be more accurate to say that only properties in the cities and larger towns do have mains gas.

The optical pulse sensor is unlikely to be useful for either a gas meter or a water meter, as these tend not to have an energy supply and hence do not have LEDs that can be detected. Usually, if any provision is made, it will be either a volt-free pair of (mechanical) contacts or (I think more usually) a moving magnet that is intended to operate a reed switch; or a reflective patch on a register wheel.

Reed switches are notoriously prone to contact bounce. Something that might be helpful - for both gas and water meters - would be a module incorporating the MAX 6816 or a similar switch debouncing i.c. that could be attached to the digital input or interrupt input of the sensor node.

Using a reflective patch implies a light source, which in turn implies a certain amount of power, which tends to preclude its use with a battery-powered emonTH (for example), apart from it often being necessary to exclude ambient light.

derekb · 2016-05-27T08:46:34.808Z

Mike,

thanks for the comprehensive post on gas metering. I was running a Txshield for almost a year and tried a number of different methods to analyse gas use in the Arduino firmware, mostly using an interrupt to time the pulses. My gas meter has a “bulge” at the end of the meter shaft with a rotating magnet, and I strapped a tiny A3144 Hall effect sensor on with a cable tie. This picks up the rotations and converts them to 5V levels very reliably. Unfortunately, like you I was never able to get what I wanted, and since changing over to an EmonPi I simply log gas pulses per hour and per day (1 pulse = 10 deci-cubic-metres of gas for me). I’m in a well-insulated bungalow on the coast in NW England and gas:electric ratio here is about 1:1. I’d welcome input/suggestions from anyone else!

Robert.Wall · 2016-05-27T15:18:52.113Z

3 posts were split to a new topic: Distribution of mains gas in the UK

MikeSims · 2016-05-28T08:15:37.678Z

Thanks for the responses so far!

Glyn, I appreciate your position, and it is perfectly legitimate, but that doesn’t have to mean that the additional software facilities I suggested ought to be ignored, and I am not demanding them now. You and Trystan don’t have to do all the development and testing by yourself, this entire energy monitoring venture is nothing if not a collaborative one, and has been for years.

I did not ask for nor expect complete common solutions for all gas meters, I suggested that some simple additional flexibility in the software could support a significant enhancement for those gas users who do monitor their meters, and because they are basically generic extensions for pulse-based metering, they may well help with other pulse-based metering, of oil, and water for example. It seems that the primary differential between such systems is the frequency of pulse arrivals, which really determines the resolution of time detail, which might mean that pulse-counting is entirely adequate for high-rate pulsing systems, but with slower rates of pulses, the durations between them could add some useful information. Let the user choose what to do with their pulses.

Also, I am not asking for these enhancements for myself (unless somebody thinks of something I haven’t ), as I already have working solutions in my system to resolve what I saw as the limitations I listed in my OP. I am not talking about major pieces of work here. Almost all were simple cut-and-pastes of existing functions with some extremely trivial editing, and the only remotely fiddly one was the retrowriting of new Watts to old datapoints, and even that took only about a dozen extra lines of code. The conversion of html and js of the mysolarpv app to turn it into a mydualfuel app again were merely trivial hacks, essentially just re-labelling some displayed titles for the most part.

The real problem I faced was my own ignorance of the coding contexts at almost every level, and I had to do a lot of reading and re-reading, as well as a lot of experiments. You can learn a lot from failure, and I learned a LOT!

What I am saying here, then, is that you already did all of the real work before I even started, and I just added a few very simple but useful extensions to an already very solid and admirable base. I could not have done what I did without that base, and I am not trying to criticise it.

Robert, actually, I really had no idea how extensive or not gas usage is anywhere, but it certainly isn’t a small sector. And given what Glyn was saying, the percentage who do monitor their gas meters might actually be quite small compared to those who monitor their electricity usage. But those who do may find some aspects of the software a little unhelpful, compared to their electric cousins. Also, I had no issues with how to detect my pulses, there is already plenty of information out there and in here for solving that, and it was not one of the issues I raised. It was what to do with them once I had them that was exercising my mind.

Derek, as I was just saying in my response here, I already have working solutions to my gas monitoring issues as I saw them. They may not be the best examples of coding, but since they were all pretty simple hacks, apart from the retrowriting feed, they do work, and do offer the extra flexibility I wanted. If your gas pulses arrive at similar rates to mine, (highest rate I see is one pulse per 32 seconds, and as long as several hours between pulses, if the CH goes off at night) then a means of deriving the durations between them may give you some extra visibility into your usage.

If you’d like to pursue this, perhaps a fresh post, rather than here?

Paul · 2016-05-28T20:55:08.998Z

glyn.hudson:

Both @TrystanLea and I live in houses with no mains gas. A lot of rural properties in North Wales don’t have mains gas.

Is that why you’re selling the gas cooker?

Paul

Robert.Wall · 2016-05-28T22:35:01.742Z

The National Grid says “Our UK distribution networks deliver gas to around 10.9 million consumers”. If we take that to mean domestic customers, and the number of UK households is 26.7 million, then roughly 2 in 5 households have a mains gas supply - and it is probably somewhat less because “consumers” could well include commercial and industrial users as well as domestic.

So Glyn & Trystan are in the majority who don’t have mains gas.

glyn.hudson · 2016-05-28T22:59:03.799Z

Haha yes indeed, well spotted! I inherited it. Do you know anyone who could make use of it?!

MikeSims · 2016-05-29T06:46:45.878Z

Robert.Wall:

So Glyn & Trystan are in the majority who don’t have mains gas.

I demand a recount!

Paul · 2016-05-29T08:32:58.489Z

OK Mike, here is the recount, and a much different result!

Paul