Hi,
Recently the data quality posted to Emoncms.org fell from 100% to 91%.
All i have changed is that the board now has M-BUS working @ 2400 baud to a Kamstrup 403 heat meter.
Also, the EmonTxV3 is no longer transmitting data to the heatpump board.
Any ideas please?
Can you see a regular missing datapoint in the graph, or specific period in the day?
Here’s mine:
you can see some missing packets around 3:30pm, my heatpump board has an RSSI of -80 its on the edge of WiFi range… I also rebooted my Pi several times yesterday
Hi @TrystanLea
Interesting, so i took a look and my RSSI is -51.
Similar to my EmonTxV3 @ -48, which is working fine.
I just noticed, when logged into the EmonESP, it suggests packet success is 99.9%.
Could that mean that the board is not generating the messages when required?
Could the newly added comms over M-Bus be interfering with the process in any way?
Update:
Looks like the 93% quality is infact the Emon comms. Reset and:
What is the data period? If where you did the upgrades, you will have of course lost some packets.
The graph will show you where there are missing data as Trystan said.
Hi, thanks for helping.
Feeds are set at 10s, if that was your question?
The data loss is regular.
Avery few minutes it looses data for 15 seconds exactly (is this a clue?).
Is that the sending end or the receiving end, or both?
For a while, it’s been considered a “good idea” to send the data a little more frequently than the database is set to receive it. If the rates are exactly matched, and the data arrives in the middle of its time slot (for the PHPFINA database), all is well. But if it sent at a slightly slower rate - because the timebase isn’t perfect - it will slowly creep towards the trailing end of its slot and then fall into the next one, leaving an empty slot. On the other hand, if it’s sent slightly faster, it will creep forwards and eventually fall into a slot that still contains the previous value. That gets overwritten, you lose it but there aren’t any gaps in the data. I suspect what you’re seeing could be explained in this way.
I was just looking at that exact principle.
Looks like the code is set to 9.8 seconds. Too close maybe?
if ((now-last)>=9800 || firstrun)
If it’s the repetition rate, it’s probably OK. If it’s the time between doing its thing (therefore you add the time for that), it may well not be, depending on what it’s doing. I don’t know that software, I’d need to find it and check.
Thanks, i will try reducing it just incase. But just thinking, that data quality issue only started a few days ago.
Since then all i have done is:
Wire in the M-Bus and change the baud from 4800 to 2400 to get it talking.
Moved the Heatpump board from inside the heatpump casing to outside…
And it looks like its EmonESP thats not getting the data through? (Screen shot of EmonESP page)
That could be enough.
Thanks @Robert.Wall
Quick update.
Over the last hour, EmonESP reported 100% success rate
EmonCMS plotted 96%.
So im still not sure where the problem lies.
Looking a bit closer, I can see something similar here, its a higher success rate than yours but still something periodic in there:
I have a sontext 531 which requires reading two pages of MBUS data so the feed interval for the mbus data is 20s rather than 10s for the kamstrup 403.
I also realise I have a few delays in the firmware timing that may well be pushing the timing out e.g: https://github.com/openenergymonitor/HeatpumpMonitor/blob/master/Firmware/Arduino/HeatpumpMonitor_AutoTemp_MultiPage/HeatpumpMonitor_AutoTemp_MultiPage.ino#L406
You could try reducing these. Perhaps try reducing them all to 50ms.
Hello @TrystanLea
Got around to testing today.
Reduced all delay() functions you noted to 50ms.
Seems to be doing that same.
Every 5 / 6 mins it will miss 1 or 2 data transmissions.
Check the emonhub logs and see if it is actually sending the data within the time slots you expect it to. I noted an issue where by my EmonTH was sending data just (less than half a second) before the EmonTX but that was enough for the data to miss it’s time slot.
I use
.
tail -f /var/log/emonhub/emonhub.log | grep -e WARNING -e " NEW FRAME : "
to monitor when the new frame is coming in (and check for warnings).
Hi @borpin
I use Emoncms.org.
I logged into the EmonESP on the board and looked at the log. Message success rate at 99.7%. Suggesting they are all getting through to Emoncms.org?
When i saw a gap in the data i checked the log and this is the result:
Calculating the interval and noting the gap in the data on Emoncms:
So i believe the heatpump board is sometimes taking > 10seconds to send a message and that’s causing the problem…?
Data gap on Emoncms: 21:04:00 to 21:04:15.
Just looked at the interval on the log for another 2 periods.
This period plotted a data gap:
This period didn’t plot a data gap:
Is it expected that the interval on the log on Emoncms is random. I expected it to be more regular?
Ah, I just noticed you use an EmonESP directly from the heatpump to EmonCMS. Limited scope to debug that I fear.
Is that an EmonESP log or Emoncms log (I do not use either)?
If EmonESP, it shows that the data is not being generated at a regular interval. @glyn.hudson or @TrystanLea will be your best help.
That is the success of the sent messages. If the EmonESP is not generating the message, it will not be measured as a failure.
No it should be regular. My personal suspicion is the ESP module. I have had countless problems with ESPs and Wi-Fi over that last few weeks. My conclusion is that the current Wi-Fi implementation for ESPs is unreliable. You can find a fair bit of chat on this out there.
