Null Data and Wireless

Hardware emonTx
1

Hi everyone, I am using the emonTXv4 - WiFi version and collecting data every 10 seconds. Over a 24-hour period I am seeing anywhere from 0.3% to 1.0% of my data come in as Null. See snippet below.

Column headings are:
Date/Time, Amperage, Temperature (F), and RSSi
2023-11-13 10:27:40, 6, 52.3, -46
2023-11-13 10:27:50, 6, 52.3, -46
2023-11-13 10:28:00, null, null, null
2023-11-13 10:28:10, 6, 52.5, -47
2023-11-13 10:28:20, 6, 52.3, -47
2023-11-13 10:28:30, 6, 52.3, -47

I am attributing the nulls to momentary loss of wifi connection as all data values are null. Is this correct and to be expected? (< 1% of data as null isn’t the end of the world, but I am trying to understand what could cause this).

Also, is this to be expected if I use 433Mhz instead of WiFi?

I appreciate any insight or experience others may have.

Gordon

2

I have no experience of emonTx4 and Wi-Fi, all I can add is in the last 24 hours - and this is a 12-channel emonTx4 using two Nodes to send the data - the first node to transmit recorded 8635/8642 successful messages and the second one 8638/8642, using LPL protocol at 433 MHz ISM band to an emonPi.

3

Thanks, Robert. If I interpret this correctly, you are having a success rate of 99.91% and 99.95% respectively. Pretty darn reliable!

I know this is getting into the minutia, but would you attribute the unsuccessful messages to “wireless” or would you expect similar results if this was a hardwired connection? Just trying to learn and get expectations set correctly.

Thanks,

Gordon

4

I prefer to look at it as 4 and 7 out of 8642 going missing.
With a direct serial wired connection, I would expect exactly 100% success. Anything less I would attribute to a failure at the receiving end to receive and process the data properly. As for Wi-Fi, as I wrote I have no experience. However, I was a little surprised by your figure of up to 1% failures - I was under the impression that Wi-Fi should be better than this. It doesn’t necessarily mean that the data didn’t arrive - it might have arrived late (because of other traffic on your LAN) and missed its slot, so a NULL value was recorded. And then of course if the next packet of data arrived on time, it would overwrite the late packet.

One way you might be able to investigate this would be to set up an EmonCMS Variable Interval Timeseries feed - probably recording the message count. This records the actual time of arrival in emonCMS of the message, and it would be apparent just where the NULL values were in relation to the time. You probably need to find one of my many explanations (in the forum) of how the Fixed Interval Timeseries works to understand what’s going on.

. For the record, which Wi-Fi add-on board do you have?

5

Hi Robert,

Even with 11 packets out of 8642, that is still 0.1% - very good results for wireless.

I am not surprised with the wifi results as it has been very challenging to get a good RSSI reading. Part of the challenge is the emonTX is in an outdoor box (albeit plastic) and there is no way to extend the wifi antenna as it is part of the wifi module. This is the main reason I am going to try the 433Mhz solution.

I will look into the Variable Interval Timeseries feed as you suggest. One question I did have is regarding messages. If I have a sensor sending data every 10 seconds (6 per minute) I would expect to get 8,640 messages per day (24 hours). So is the message count in emonCMS actual messages sent, or is it messages anticipated (based on the timestamp)? I guess I could test this by disconnecting the wifi for a few minutes to see if this impacts the count.

The wifi module I am using is the one supplied by the Open Energy store - Huzzah ESP8266 wifi adapter.

6

The message count depends I think on where you look! The one on the emonCMS Input page comes from the sketch in the emonTx4, and it’s the number transmitted (every 9.96 s from memory). If there’s one in the ESP8266 page, it’s probably the same. “missed” and “missedprc” are, according to Trystan, missed messages - when I asked, he didn’t specify whether they are detected in emonHub or emonCMS nor say the criterea for including in the count, so I don’t really know what they signify. The statistics on the graph is the number of data points plotted v.s. the number expected over the period (so for my 24 hours it includes both ends - and I think it’s one too many, it should be 8641).

I only suggested the Variable Interval Timeseries feed as a diagnostic tool - it uses about double the memory of the fixed interval (because it stores Unix time for each data point) - in addition to the Variable Interval for data you will retain.

7

I doubt it is your data coming in as NULL, simply that emoncms has not had an input value during the 10s window.

@Robert.Wall - what is the emonTX data rate, just under 10s IIRC. If so there could easily be delays in processing that data occasionally.

I am also not sure if the Wi-Fi module is sending the data via MQTT (and single topic JSON or multiple topics) or the HTTP API. Again, this could introduce delays enough to miss the window.

For other systems/data sources, I generate a timestamp and send it with the data so it doesn’t matter when it arrives, it is put into the right window. This can be useful if you have a number of readings from a single device so they all end up in the same window.

8

I wrote the library - and that accepts the data rate the sketch gives it. Trystan wrote the sketch and it’s somewhere in Github - I’m never sure which the “factory” default is. The possible catch with all this is the library reports are tied to mains cycles (nothing new there, even the discrete sample emonLib did that), so the report might be 20 ms late. I’d need to check the code but I don’t think it’s cumulative.

This was the point of setting up an additional VITS Feed Null Data and Wireless - #4 by Robert.Wall to see exactly when the packets were arriving.

It looks as if he’s set the rate to 9.8 s - so there should be 200 ms of “creep”, or to put it the other way, if the previous data packet only just made it before it’s slot closed, the following incoming packet could be delayed up to 200 ms before it missed its slot, and a NULL got recorded instead.

9

Yes, exactly. For whatever reason, I suspect the data is missing then slot.

10

A VITS feed would show this is the case - or not. It would also show the scatter in the logged arrival times, which might give a lead to the cause.

1 Like
11

Gents, I reviewed the null entries over a 36-hour period and found that some entries are grouped over multiple timeslots. This seems to indicate more than a late message (eg 20 ms late) but rather that something interrupted the transmission of data packets. Wouldn’t this support a theory of problems in the wireless signal?

image
image520Ă—1261 17.8 KB

I intend to experiment with the VITS feed later this week and will report back.

Thanks,

Gordon

12

Don’t you mean the Wi-Fi signal? Or have you swapped and you are now using what we call wireless - the built-in UHF ISM band 433 MHz transmitter in your emonTx4 and the RFM69Pi add-on board in your emonBase? (Which is how I got > 99% success.)

13

My comment is wireless in general as both wifi 2.4 and 433 are wireless technologies. The table I show is wifi 2.4GHz related, so I should have been clear about that. Any wireless technology is subject to transmission hiccups. Cell phones are a perfect example. All this to say, I would expect less than 100% success when using any wireless technology vs a wired connection.

What I am trying to determine is how much of my null data issue is related to message timing as you and others have referred to vs poor wireless signal (in my case wifi).

I am trying to address the poor wireless signal first by trying the 433Mhz radio. Once I determine if it is possible to improve the wireless signal, then I can look at other areas such as message delay.

My hope is that 433Hhz will be much better (I have not received the emonBase yet in order to test).

Thanks for all the help and support.

14

The EmonTx4 LowPowerLabs firmware when used with the emonBase will retry sending the message multiple times if it does not receive an acknowledgment, this has in my experience reduced the number of missed packets considerably - at least in the cases where there was an issue before. (The LowPowerLabs radio format is the default radio format, unless that you have chosen or uploaded JeeLib Native firmware).

15

I think the OP only has the Wi-Fi TX so no receiver for 433 data.

16

Brian, you are partially correct. My current installation is WiFi but I have just received the emonTXv4 433 and the emonBase - looking forward to getting both set up for comparison.

Regards, Gordon

2 Likes