EmonTX missing data on CT4 input at regular intervals

Is it a network problem - either on your LAN or between your router and emoncms.org? (i.e, your ISP?) I don’t know how “quality” is arrived at; if it’s documented anywhere, I’ve no idea where to look. If it’s purely the number of received packets versus the expected number over the same period, then a transmission problem - or even a variable delay which causes packets to fall into the wrong ‘slot’ - would explain it.
[What I mean by that is, if you’re using PHPFINA at 10 s rate, the data is simply dropped into the “now” time slot as it arrives. So let’s say a packet arrives 9.9 s into the time slot consistently, all is well. but if one packet gets delayed by 150 ms, it misses its slot and goes into the next one. If the next packet isn’t delayed, it goes into its proper slot and overwrites the late one.]

Why ct4 should be different, I’ve no idea. I don’t know what goes on inside the ESP8266, nor how the data is sent over the Internet (i.e. how many packets, of what size). But there’s no law saying that the data needs to be sent in that order, so you could shuffle ct4 up into 2nd or 3rd place, if you wished.

You don’t need to send empty values for anything you’re not using, so you could comment out ct3, maybe vrms, and pulse.

If its a network issue I’m relatively relaxed as that can be checked easily enough when it happens again. The ‘quality’ dropping off does not appear to affect the ‘good’ inputs anyway. Its back at 100% now.
I’m ultimately looking to use all 4 inputs and vrms so pulse could go.
Its the absolute regularity of the dropping out/coming back that puzzles me. Looks to be the same period on as off - ie 50% - suspiciously the ‘quality’ is also around 50% of the good inputs.
For now the workaround is to use inputs 1-3 but I want to be sure I’ve not got a faulty unit so have asked openenergymonitor support for a view.

In case this happens to someone else I’ll detail the solution.
OpenEnergyMonitor kindly checked out the hardware they had sold me and pronounced it OK.
When I reinstalled I had the same problem but on a different input.
I then started a new node on Emoncms.org and moved each input over to it, using identical Process Lists to create new feeds. Have had no problems since.
Can only conclude that the problem was within the original node on the Emoncms.org site.

I must say I find that rather surprising. Is @TrystanLea aware of your conclusion?

Sorry to bump an old thread, but Im bumping into this issue too I think.

Have emonTX with wifi.

Installed and connected solar yesterday, so today was the first full day. and the solar feed (on CT2) seems to freeze updating.

Nighttime now and the power usage is tracking but the solar feed and the "house consumption, ie solar gen + CT1) is cutting out for 20- 30 minutes.

I added a second log to feed (a new one) and that has been running fine, It may not be the emontx all along.

Going to nuke it and set up again and see if it changes anything. Could it actually be a bug in emon cms? trying to narrow it down will be difficult, esp if i cant re create it

Edit: nuked it and set it up again it is running fine and steady after an hour. Now how on earth could i break a feed. Will keep an eye out for it in the future.

You’re going to be using a totally different sketch in your emonTx - assuming it’s a new one (bought since the end of last year).

That’s the emonESP, or an emonTx with the ESP8266 module attached?

Just to say that since I changed node, as I described, I’ve had no further problems for nearly a year

Yep Emon tx with the emonesp.

It’s been perfect since I reset it too( the cms feed)

@TrystanLea
This sounds weird. Could it be a funny in emoncms.org, even though you looked a while ago EmonTX missing data on CT4 input at regular intervals - #7 by DickiePhitt

I cant see anything that might cause it on emoncms.org. @graememk if you want to send me your account name in a private message, I can see if there is anything associated with your account if you like?

It was on a raspberry pi 2 that I have Emon cms setup on and doing the recording.

Only thing I might have changed was that I originally set the update interval to 5 seconds rather than the default 10.

I’ll setup another feed on an irrelevant input and see if I can replicate.

Today it has worked perfectly.

I was able to replicate it, Yep it was setting a feed to 5 seconds. (this is on a emontx +wifi, currently just plugged in to the AC, havent got around to installing it yet)

Currently its an input over mqtt, but had the same behaviour when posting over the http to a emoncms.

I can post to emoncms.org if you like and see if I can replicate the bug there.

Looks like I’ve got the same issue. There is 20 min worth of missing data on a regular basis, and i only ever 50% data quality in the feed summary. All other feeds maintain ~100%.
This is the data from my newly added ‘solar diverted’ feed. This is CT on the live wire of a cable running from an mk2router to the immersion tank. Below is a night time data set (so no current to measure).


Below is day time data set:

The CT is connected to a TX, running a modified sketch from Robert Wall,
https://community.openenergymonitor.org/t/missing-temperature-feeds-on-emontx-and-emoncms-setup/17264/6?u=dr_w. The TX has 6 temperature sensors plugged in via a breakout board using an RJ45 cable.

Has your custom sketch run for the last 18 months, or have you only just put it to work?

Hi robert,
It’s been running since march 2021. No other problems that i’ve noticed, just this missing data when i added this new CT a few weeks ago.
I’m now wondering if i setup the feed wrong and didn’t set a time interval in the process list setup. I’ve added a new process duplicating the ‘solar divert’ feed but this time made sure i set a 10 s interval.

Is it possible to edit existing feeds to change the interval? Doesn’t seem to be an editable option in the feed setup.

No. The PHPFINA feed works by storing the start time, and calculating the time for each data point from the interval and the record number. You’d need to rewrite the file interpolating or skipping values to “change” the interval of historic data.

You can hover your mouse over the name in Feeds to see what the properties are.

1 Like

Check emonhub is set to log DEBUG and check in the window that you see the data coming in.

Then SSH in and open the log and look to see what emonhub is receiving as data.

Then check the MQTT Publish (if it is using MQTT) and check the data is published every time.

If the data is coming in OK and going out OK in emonhub, the issue will be emoncms.

Personally, I have seen the input MQTT service get overloaded if sent too many topics to process too quickly.

It is odd it is just that one feed, though.

[edit]
To follow the emonhub log use a command like this - the 2 -e create an OR in the grep.

Yours will probably look different!

pi@emonpi:~ $ tail -f /var/log/emonhub/emonhub.log | grep -e "emon/emonpi" -e "NEW FRAME : OK 5"
2022-10-28 09:54:07,437 DEBUG    RFM2Pi     24368 NEW FRAME : OK 5 140 0 0 0 140 0 240 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2022-10-28 09:54:07,534 DEBUG    MQTT       Publishing: emon/emonpi {"power1": 140, "power2": 0, "power1pluspower2": 140, "vrms": 235.36, "t1": 0, "t2": 0, "t3": 0, "t4": 0, "t5": 0, "t6": 0, "pulsecount": 0, "time": 1666947247.437131}
2022-10-28 09:54:12,492 DEBUG    RFM2Pi     24369 NEW FRAME : OK 5 139 0 0 0 139 0 1 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2022-10-28 09:54:12,666 DEBUG    MQTT       Publishing: emon/emonpi {"power1": 139, "power2": 0, "power1pluspower2": 139, "vrms": 235.53, "t1": 0, "t2": 0, "t3": 0, "t4": 0, "t5": 0, "t6": 0, "pulsecount": 0, "time": 1666947252.491709}
2022-10-28 09:54:17,443 DEBUG    RFM2Pi     24371 NEW FRAME : OK 5 139 0 0 0 139 0 244 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2022-10-28 09:54:17,715 DEBUG    MQTT       Publishing: emon/emonpi {"power1": 139, "power2": 0, "power1pluspower2": 139, "vrms": 235.4, "t1": 0, "t2": 0, "t3": 0, "t4": 0, "t5": 0, "t6": 0, "pulsecount": 0, "time": 1666947257.4434764}
2022-10-28 09:54:22,495 DEBUG    RFM2Pi     24372 NEW FRAME : OK 5 135 0 0 0 135 0 21 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (-0)
2022-10-28 09:54:22,644 DEBUG    MQTT       Publishing: emon/emonpi {"power1": 135, "power2": 0, "power1pluspower2": 135, "vrms": 235.73000000000002, "t1": 0, "t2": 0, "t3": 0, "t4": 0, "t5": 0, "t6": 0, "pulsecount": 0, "time": 1666947262.495364}
1 Like

Thanks Robert and Brian.
I’ve managed to track down the issue to the way i set up the feed. I duplicated the feed but this time set the interval to 10 s. The original feed was set to 5 s.
Below is a graph showing the original and new feed (with a 400 offset to separate the two lines on the graph).


You can see there is still a little bit of missing data on the new feed but its 97% complete, which is the same i get for all the other feeds coming from that emonTX.

1 Like

I think you need to look at this, where I explained how the Timeseries Feeds work:

1 Like

thanks. makes more sense now.

1 Like