Heat meter accuracy testing

Context: How accurate do we need heat metering to be?

The importance of accuracy varies depending on what the goal of the monitoring exercise is. For basic diagnostics and even heat pump control high accuracy is not that important. It doesn’t matter for rough evaluation of system temperatures, or detection of cycling etc if a temperature sensor is out by 0.5°C. As soon as we start comparing the performance of different heat pumps, discussing COP’s and cost of heat delivered, accuracy becomes much more important.

If we take a look at the following performance comparison chart from HeatpumpMonitor.org, that plots average system temperature vs SPF the majority of the variation above and below the line of best fit is within ±0.3 SPF of the line. The furthest outliers are about ±0.7 SPF:

For a heat pump showing a SPF of 4, the actual SPF could vary considerably depending on the accuracy level, to give a few rough examples:

  • Accuracy: 3%, SPF ±0.1 (3.9 - 4.1)
  • Accuracy: 5%, SPF ±0.2 (3.8 - 4.2)
  • Accuracy: 7%, SPF ±0.3 (3.7 - 4.3)
  • Accuracy: 10%, SPF ±0.4 (3.6 - 4.4)
  • Accuracy: 15%, SPF ±0.6 (3.4 - 4.6)
  • Accuracy: 20%, SPF ±0.8 (3.2 - 4.8)

Some of the variation between systems on HeatpumpMonitor will result from inaccuracy in metering, but the question is how much?

Core to the approach that we have taken with HeatpumpMonitor.org and the Level 3 heat pump monitoring bundle in particular is to use MID certified, class 1 electric metering and class 2 heat metering in order to target a higher accuracy level. We also use the same metering accuracy classes that have been used in previous heat pump monitoring trials such as the Energy Saving Trust and Electrification of Heat trials.

It is however not quite as simple as a class 2 heat meter is ±2% and class 1 electric metering is ±1%. In 2016, BEIS funded an insightful research report into the accuracy of heat meters which can be found here BEIS Heat Meter Accuracy report, I will try to summarise it in the next post.

We have also built our own test bench setup to verify and gain more familiarity with this question ourselves. The results and data from these tests are shared below.

1 Like

BEIS Heat Meter Accuracy report

From: https://assets.publishing.service.gov.uk/media/5a8050cd40f0b62302692c81/Heat_Meter_Accuracy_Testing_Final_Report_16_Jun_incAnxG_for_publication.pdf

Heat meter accuracy is commonly specified using the class notation E.g Class 1, 2 & 3. All of the heat meters that we have used as part of the Level 3 heat pump bundle are Class 2 heat meters as this is the most common type.

The European standard EN1434 (harmonised with MID) describes the accuracy to which the flow meter, temperature sensors and calculator must achieve. The accuracy class of the heat meter is associated with the flow measurement only.

Maximum Permissible Error (MPE)

Class 2 Flow measurement:

Ei = ± (2 + 0.02 qp/q) but not more than ± 5%
Ei is the uncertainty in flow measurement expressed as a percentage
qp is the highest flow rate at which the meter can function continuously
q is the actual operating flow rate

Uncertainties due to the measurement of temperature difference (same for all classes):

Et = ± (0.5 + 3Δθmin/Δθ)
Et is the uncertainty in temperature difference measurement expressed as a percentage;
Δθmin is the minimum temperature difference for which the system is rated, and;
Δθ is the actual operating temperature difference

Uncertainty contributed by the calculator (same for all classes):

Ec = ± (0.5 + Δθmin/Δθ)
Ec is the uncertainty contributed by the calculator expressed as a percentage;

Example: What is the MPE of a Axioma Qalcosonic E4 heat meter QP2.5 m3/hr ( Δθmin = 3K ). At DT 5K and flow rate of 0.87 m3/hr (5kW)?

Ei = ± (2 + 0.02 qp/q) = 2+0.02×(2.5÷0.87) = ± 2.06%
Et = ± (0.5 + 3Δθmin/Δθ) = 0.5+3×3÷5 = ±2.3%
Ec = ± (0.5 + Δθmin/Δθ) = 0.5+3÷5 = ±1.1%

The EN1434 standard requires that to determine the overall error, the absolute values of these
errors must be combined as an arithmetic sum. The total MPE error for these conditions is therefore 2.06 + 2.3 + 1.1 = ±5.46%. If the actual temperature difference is closer to 3K which is quite normal for heat pumps during mild weather the MPE would rise to 2.06 + 3.5 + 1.5 = ±7.06%.

On the face of it a Maximum Permissible Error of ±7% under fairly standard conditions seems quite high given that the Class notation suggests that the accuracy is closer to ±2%. The MPE assumes that all errors are at their maximum extent and of the same sign, we would expect variation on both of these in the real world.

BEIS Report test results

The BEIS Heat meter accuracy test report results for Energy error across a range of different heat meter types where are follows, showing quite a wide range of results at DT 3K and slightly narrower at 5K. These results are within the MPE limit’s defined by the standard but it’s clear that errors of 4-6% are not inconceivable:

The errors in flow rate measurement alone for DN25 meters where outside the maximum permitted total error, with a mean error is +2.36%. DN40 meters fared better with a mean error of +1.8%.

The report explores a range of further factors that can affect heat measurement accuracy including:

  • Incorrect orientation of meter (meter oriented vertically up or down).
  • Air entrainment within flow
  • Disturbances to flow (bends in close proximity)
  • Installation in close proximity to pump
  • Testing of effect of changes in fluid (glycol antifreeze protection)
  • Calibration testing - temperature measurement
  • Testing of different and incorrect temperature sensor mountings
  • Transient operation
  • Effect of dirt
  • Used meters

I will come back to summarising the findings from the report on these factors in a follow up post.

1 Like

OpenEnergyMonitor test bench

We wanted to get a better understanding of the accuracy of heat meters in practice ourselves and so decided to build a test bench in our office. Here are a few pictures of the test bench, it consists of an inline electric element, a circulation pump and expansion vessel, two heat meters and a couple of 1200x600 K2 radiators.

We used radiators as a load rather than a big volume of water so that we could run the system at steady state conditions in order to avoid calculation complexity with heating of the water volume and system components themselves.

The inline electric element is driven by an MyEnergi Eddie PV diverter that we can use to set different power levels. We monitor both electrical input to the diverter and electrical output using both an SDM630 and EmonPi2, we also monitor the power used by the circulation pump.

Insulated in order to minimise heat loss between return and flow temperature sensor on the inline meter side of the heat meters: Bubble wrap used for part of this for quick configuration changes.

A later configuration with longer straight sections before the meters, please excuse the presentation :sweat_smile:

The data coming out of this typically looks like this.

  1. We can see the flow and return temperatures rise at the beginning of this test period as we heat up the system volume. The electrical element (black line) was ran at full power at the start in order to get up to a reasonably high system temperature.

    Note: We can see a relatively large error here between the electrical element input and the heat registered on the heat meters, most of this missing heat is heat used to heat up the primary volume of water and copper pipes, electrical element etc before the heat meter - this is why tests at constant temperature are so important.

  2. The heat input is then reduced to a level that will keep the system at steady state using the Eddie programmable power levels.

  3. We then select a period where the flow and return temperatures are as stable and flat as possible and take a set of averaged measurements across the period. In this particular example we took a set of measurements between 05/11/2024 12:04:20 and 05/11/2024 12:42:50 an average of about 230 datapoints per feed.

  4. Zooming in on this section we see the following:

  • Heat input from the electrical element and central heating pump (~14W) in black. Measured heat output by the Sontex 789 under test in yellow. Measured heat output by the Axioma in orange.

  • The Sontex is on the face of it under-reading by 0.6%, the Axioma is a bit lower, under-reading by 2.6%, but we have not accounted for heat lost from the pipework, electric heater body and pump body prior to the heat meter yet.


Estimating heat loss prior to heat meter

Estimating heat loss to the room between flow and return temperature sensor’s, heat lost through insulated pipe sections.

There is about 2.8m of insulated pipework including 10mm insulation surrounding 15mm pipework, and a range of thickness around the pump and immersion heater ranging from 5mm to 15mm. Insulation work is not perfect, there are small sections of exposed pipework around some of the bends and clips.

If we assumed a continuous 2.8m length of 15mm pipework with 10mm insulation.

Q = 2 π * k * (T_fluid - T_room) / ln(ro / ri)
k = 0.035W/m2.K
T_fluid = 43.5 C
T_room = 21.0 C
ro = 0.0075m (outer radius of pipe 15mm / 2)
ri = 0.0175m (outer radius of insulation ro + 10mm)
Q =  5.84 W/m
Heat loss = ~2.8m x 5.8 W/m = ~16W

A section at 5mm insulation would be 9.7W/m and a section at 15mm insulation about 4.5 W/m.
There about 1m between the first heat meter and the second heat meter but insulation is thicker here. There’s also about 0.8m out of the 2.8m in 22mm pipe with 13mm insulation, heat lost from this section could be 6.3 W/m. I have assumed an average of about 5.8 W/m or DT x 0.26 W/m.K as a temperature dependent simplification.


Subtracting heat lost before the heat meter

For the example test period above the mean water temperature of the pipe was 46.7C and the room was 22C. The heat lost before the heat meter may therefore have been about 18W.

Subtracting this heat loss from the heat input 1595W - 18W = 1577W. The Sontex would now be over-reading by 0.5% and the Axioma would be under-reading by 1.4%. That’s a very respectable margin of error. 1.4% is equivalent to ±0.05 COP.

The next post provides results from a wider range of tests that we have performed.

2 Likes

Heat meter accuracy test results

The spreadsheet is a bit wide and so I have split the screenshots below into two parts. columns A-V and then L-AH.

heat_meter_accuracy_testing.ods (45.6 KB)

  • On average the Axioma (and one Kamstrup test) under-read by 1.67% and the Sontex over-read by 0.57%. The average difference between these meters was therefore 2.24%.

  • The tests show a range of results for the Axioma and Sontex meters, with a minimum under-read of 0.5% all the way through to a maximum under-read of 3.24% for the Axioma (A range of 2.74%). The Sontex results show a range from a 2.16% over-read through to a 1.34% under read (a range of 3.5%)

  • The maximum difference between the readings of both meters was 4.14% again with the Axioma under-reading and Sontex over-reading.

  • The Sontex slightly over-reading and the Axioma slightly under-reading seems to be a consistent result across these tests.

1 Like

Comparing the results from an Axioma and Sontex connected to the same heat pump system

To provide another comparison I installed an Axioma on my heat pump system at home alongside the existing Sontex Multical 531 that’s been monitoring the system for the last 4+ years. I’ve installed the Axioma slightly closer to the location where the primary pipework goes through the external wall to the outside unit, so in theory this should record slight more heat as there’s a few meters of uninsulated pipework between them.

The results are very close, with the Axioma based data reporting a COP of 4.02 over the last month vs 4.00 from the Sontex. Perhaps a slight under-read by the axioma might be getting canceled out here by the additional heat lost on the way to the Sontex.

Axioma:

Sontex:

1 Like

Conclusions so far

The above gives us a number of initial measurements indicating expected accuracy of the heat meters that we use. In general I would expect from these results a tolerance of ±0.1 COP or better on a SPF of 4.0 or around ~3% Accuracy. Technically the MPE (maximum permissible error) of the heat meter together with an electric meter could push this to ±0.3 or 7%, but our measurements to date do not indicate that this is common, our maximum measured error was an under-read of 3.24% on the axioma (average under-read of 1.67%). This said the number of tests we have performed is still relatively small and it seems that the BEIS testing did produce larger errors that were close to the MPE.

As time allows we will continue with further tests on our test bench here and also use it for other interesting experiments such as looking at the effect of adding radiator fans for increasing radiator outputs.

Any thoughts for other test to do and reflections on the above are very welcome!

5 Likes

Thoughts

(1) bloody excellent work as always

(2) heat loss between heater and meter could be established by bypassing after the heat meter and heating the pipework to a steady temperature using the electric element

(3) using two temperature probe pockets (or ball valves) directly either side of a well insulated immersion heater block with the pump jammed right up underneath it would eliminate most heat loss error (flow sensing element can safely be remote - it can be part of the heat loss) and is easier to make permanent than the bubble wrap.

(4) High flowrates are easy for the meter. It’s the 3 degC delta that’s most difficult. You’re right on the limit of the design/MID approval range with that.

(5) Is this using instantaneous measurements or the cumulative energy register for deciding on total heat or the instantaneous values for temperature/power/flowrate? The former are fiscal accurate according to MID. The latter are unregulated. Many meters will round and indeed round down for these values. I don’t think sontex or axioma do; but kamstruo certainly used to round power down to the nearest 0.1 kW.

1 Like

Good work, @TrystanLea, but could you clarify how you verified the measured flow rate?

In industry, if we didn’t have a pre-calibrated (ISO 9001) meter in series, we used a “drop test” to check flow meters (measured the liquid level fall in a large upstream vessel of accurately known diameter over a period of time).

1 Like

He’s calibrating overall energy I think (compounding the errors on flow rate and temperature)

1 Like

Another idea: crack the case and flip the axioma into “test mode” (see manual) - then count the pulses for cumulative energy/volume.

This is high resolution (low quantisation error) and how they’re calibrated at the factory.

You can the split the error between flowrate measurement and temperature measurement (assuming electricity input is spot on) to see which needs more love and attention.

Pumping electricity…where does this end up as heat? That energy is dissipated but…is it dissipated in the heater side or in the balance of the system or from the pump body?

At factory the flowrate elements get calibrated with a positive displacement piston/plunger type rig; and the temperature probes get water bathed.

I would be curious if there’s any difference between those ball valves and the tees; between both probes mounted in tees Vs one in a tee and one in the meter body.

1 Like

We tried to insulate the pump body as best we could so all the pumping energy should end up in the system.

Yes, but where I’m the system?

Immersion dumps all electricity as heat on the supply side.

Pump dumps some energy as heat and some energy as motion of water. That energy turns to heat throughout the system (not all on the supply side of the meter)

Splitting hairs at this point, but you’re into hair splitting in the accuracy here so I’d try account for it.

2 Likes

And some of the energy is disapated outside of the system as noise?

3 Likes

Probably absolutely minimal, but yes!

Thanks @Marko_Cosic @SarahH!

Correct, We are using the instantaneous power measurements rather than energy specifically but yes compounding the errors on flow rate and temperature. I use power to give higher resolution than we typically read on the energy registers, though is there a way to get the energy registers to report Wh instead of kWh?

I did spot the low resolution of the power values on the Kamstrup specifically so used the mass flow rate equation to calculate power based on a fixed specific heat for that one and the error was much less as a result.

Good point, what is the hydraulic efficiency on a wilo pico pump ? It’s using 14W and the flow rate is ~6-7L/min. I will make a note of the pressure drop head next time I power it up. Perhaps it’s only actually adding ~50% of 14W to the heat input or would it be a bit more than that (lower overall hydraulic efficiency)…

Yes we could test that.

Axioma can do 0.1 kWh (use the E3 configurator software) BUT I would suggest using the TEST pulse outputs instead as that’s much smaller / higher resolution.

The question is where is the heat appearing. Your ideal heat source would be a “point” heat source with zero loss to ambient / perfect water mixing / a temperature probe either side of it.

You can get close to this by moving temperature probes literally either side of a flow-through heater with absurd insulation levels OR by operating the rig at near ambient conditions to minimise heat loss (heat a 220 litre drum of water using a 1.5 kW / 3 kW / 6 kW immersion heater, and place the temperature probes either side of it)

I think I still have a 6 kW back up heater that’s flow-through and has 1" / 28 mm end fittings. 1.5 / 3 / 4.5 kW. Pop a tee or temperature probe ball valve on either side. Heat up a 220 litre drum of water. Keep the drum at ambient temperature by blending in cold tap water at a rate of 1.5/3/6 kW.

Flowrate sensor can be remote. Pump from the OUTLET of the immersion heater (after the temperature proe) and then through the flowrate sensor (after a long enough bit of pipe). Have a short fat run of 28 mm pipe from the barrel to the inlet of the immersion heater.

This way the pump is essentially out of the equation as far as measuring the heat added by the immersion is concerned. It’s “after” the probes so the leccy used by the pump isn’t adding heat inside the immersion heater between the two probes.

Then add two electricity meters. I’ve got an ABB I can loan you. Compare with Chinesium unit. We shouldn’t assume that it’s perfection although it should be consistent with a resistive load?

1 Like

We could certainly re-arrange the setup, to remove the pump from between the flow and return temperature sensors and shorten the heat loosing pipework. Perhaps that’s a good next step test before going to more complicated setups like a large drum etc?

Agreed another meter on the input wouldn’t go amiss!

Thinking more:

You have a “heat input” - this is an inline heater, controlled by an Eddi device, and metered by one electricity meter.

Two electricity meters ought to be better than one. A switchable heater (1.5 / 3 / 4.5 kW) ought to be easier to meter the electricity for than an Eddi-modulated heater though I can’t imagine that’s a major source of measurement error.

(I’ll ship an ABB meter and a heater if I still have these)

You want to measure “heat that was added to water flowing through this device” so put the temperature probes virtually immediately either end of it.

You want to check what impact having one probe in a tee and one in the meter body has (vs both in tees). You want to check what impact tee vs ball valve has. You want to see what insulating over the tee / ball valve and temperature bung might have. (best results should be with symmetrically mounted sensors) You also want the heat to be mixed into the water thoroughly.

You could put the probes either side of the heater. They then only see heat from the heater (pump is an irrelevance / just appears as a lower than expected heat loss from the system?)

You could also put the heat meter directly after the heater; then the supply temperature probe directly after the heat meter. This will increase heat loss (from heat meter flow tube) but it will ensure good mixing and make the two supply temperature measurement points as close to each other as they can be?

(I’ll ship a couple of ball valves and a “bung” for temperature probe pockets)

Insulate the inline heater and the heat meter flow sensing element.

Pump, expansion vessel, heat dump etc all go outside this “core measurement” area. You want no heat input to this / no heat output from it.

Meter goes “test mode” counting the volume pulses and the energy pulses rather than interpolating. Perhaps the leccy meter gets pulse counted too so that the setup is easy to build?

Run a parallel test of the same thing read by M-Bus and using interpolation rather than calibrated energy pulses. See what impact this has on accuracy?

Steady state for temperature is not too important in my opinion. You can afford to allow it to vary as it’s only the delta between sensors that matters. So scrap the barrel idea.