Community
OpenEnergyMonitor

OpenEnergyMonitor Community

Irregular tx data receipt

Tags: #<Tag:0x00007f1d4c91e248>

Hi.

I’ve recently installed 3 emonTX units on node id’s 15,16 and 17. At the time of installation I was running emonSD from Sept19 I think.

Anyway after installation all appeared to be ok after a little assistance from this forum with adjusting to the newer sketch values. At this point I wasn’t recording any data to feeds just checking that the inputs were being received.

Today I wanted to start recording this data in new feeds as I have adjusted what I am monitoring. I decided that I would back up the data, flash to the latest emonSD image and then restore the existing data. All this appears to have gone without hiccup.

However, I now notice that the inputs for the 3 emonTX units are missing some of the data posts. All three of them update intermittently, sometimes in the expected 10 seconds, but sometimes they go yellow or red as they take a longer time before the next reading is received.

I can’t be 100% sure this wasn’t happening before as I didn’t spend much time looking at it.

I’ve looked at the log and there are regular errors with:

2021-01-20 19:36:27.040|WARN|index.php|406 Not Acceptable|emon/input/post

I presume these are the data posts that are being lost. Is there any logical reason for this as part of the upgrade?

@TrystanLea - any suggestions?

How did you end up setting up the communication? Direct, RFM or WiFi?

Hi,

They are all on RFM. They are installed next to each other, is there a problem with this do you think?

There is the potential for RFM clashes if they all send at the same time. Try resetting each one (press the reset button) at 1s intervals and see if that helps.

Nice neat setup.

Ahh ok, I will try that when I get back later tonight.

Thanks for the comment on the setup. I’ve been through the “chuck it together” phase in my first iteration when determining the usefulness of this stuff and now I’m sold I wanted to do it properly! Still needs a bit more cable management, but Im waiting for my heat pump feed to be added to this board, so I have an extra CT to be added to each phase for that once it’s installed, hopefully next week.

Also, that middle aerial is bugging me, but it just won’t sit perfectly vertical, so I might have to swap it with another one! :smiley:

Where is the base station located?

I think you may have more than one issue perhaps. The emoncms log warning suggests the whole api payload was rejected (406) but if there was a perminant fault you wouldn’t have any data at all, also if the data is incomplete (missing frames now and again) that would suggest the data gets through sometimes.

The way emonhub works it buffers data for quite sometime so if there was a deformed frame within the bulk api request it would simply keep trying and getting rejected until the bad frame was dropped on a first in first out basis when the buffer hits a limit (or that’s how it did work previously). Therefore a bad frame would result in no data for a significant amount of time about 4.5hrs with 3 Tx’s. So, I suspect the interfacer might be crashing and restarting to be able to clear the offending frame, at a guess!

It’s the emonhub logs you need to look closer at, post some here for help debugging.

Have you updated (emonhub) in the interim perhaps?

Hi,

the base station is located in the room behind the wall in the photo (single skin brick). It’s strange when you are watching the input screens as they all update but not consistently and it changes from unit to unit as to which 1 (or 2 sometimes) does not update.

I looked through the emonhub log for any “BAD FRAME” type messages as this is what I was seeing previously when I had the inputs configured based on old emonTX sketches.

I did run the Full Update in the Admin section before the SD card upgrade (and backup) and also after the SD card upgrade so it should be running latest versions.

I’m not at home currently so will have to check it again later.

Try each emonTx in turn by stopping the other 2, if each emonTx gives perfect results then it’s likely they are clashing, try powering them up at 2 second intervals (or reseting as Brian suggests). If any/all of the emonTx’s are posting erratic try moving the base closer to see if that helps.

Personally I would use serial connections to run 3 emonTx’s, even if that involves another RPi to just send the data to your other base if you specifically don’t want it near the Tx’s.

None of these RF issues would cause a 406 at emoncms so that must be unconnected, either another less frequent issue (or from another source) If you are using a emonsd image all on one Pi then http isn’t used to post locally and the 406 is not from emonhub unless you have altered the emonhub.conf to use http to local emoncms.

1 Like

Evening gents. Thanks for all your replies and help.

I tried moving the emonPi closer, no real change. I tried re-orienting the 3 ariels, no real change.

I have now turned off all units and turned them on at 2 second intervals and have verified the red led pulse from the units when transmitting are separated and the data is regularly updating on all 3 units in the emonPi, so it does look like RFM clash.

I hadn’t considered this as an issue. Obviously if there is any power cut, then they are likely to be clashing again as they will all boot at the same time.

Paul, you mentioned running them over serial. I was planning to re-locate the emonPi to the workshop soon anyway as I will put the other emonTx units on ESP8266 wifi adapters as it’s not possible for me to pick them all up using RFM in the range of one emonPi. Is there a guide for doing this with multiple emonTX units being connected via serial to a single Pi?

Thanks again.

Not a formal one but a few here.

With 3, I’d be inclined to get a TTL to USB converter and plug them into the USB port of the Pi.

Not really as there are so many different ways it’s been done.

This thread shows how I’ve done many installs, the key element is using FTDI usb adapters with an eeprom, ie genuine (or perhaps really good fake’s?) ftdi ft232rl chip.

The issue you may face with using default FW with “emonesp compatible” key:value serial output is lack of node id to anchor the serial output of anyone device to a certain set of emoncms inputs, that’s why I use the ftdi adapters. However @Robert.Wall may have included a way to send a nodeid with key:values in the new CM FW? I do believe he has included a way to send space separated values with leading nodeid (by setting a fag at compile time?), which is my preferred format.

I would serious consider using Pi Zero’s over emonesp’s where you can. The advantage of the emonesp is is fit’s inside the later emonTx’s, but there is a huge difference between an emonESP and a Pi Zero running emonhub for the same money(ish). You could probably manage to get a PiZ in the emonTx case if you really wanted to (I’ve not tried) but I’m not entirely convinced the aluminium case won’t impact wifi to some degree despite the perpex ends. Just some thoughts for you to consider, there is rarely just one right answer.

Just use a Node Name in the interfacer definition to solve that unless on reboot, different EmonTX could come up to different USB devices. Can you lock those to specific USB device?

I think we could do more on providing options on the data output format from the EmonTX. An option to add in an ID and output as a JSON format would be really good.

I’ll second that. WiFi is far more stable on the Pi.

Yes that is the problem and that’s why I do it the way I explained unless you only have 1 usb device you must either have a node id passed within the payload or be able to identify the different adapters, by using unique ids (need ftdi eeproms) or unique devices eg 3 different brands (usb vendor ids to be accurate).

Unfortunately no - it has copied exactly the “standard” (however right or wrong that might be) format.
The code looks like

#ifdef EMONHUBTX3EINTERFACER
      // ---------------------------------------------------------------------
      // Key:Value format, used by EmonESP & emonhub EmonHubTx3eInterfacer
      // ---------------------------------------------------------------------
      Serial.print(F("MSG:")); Serial.print(emontx.Msg);
      if (CT1) { Serial.print(F(",P1:")); Serial.print(emontx.P1); }
 
[etc]

#else
      Serial.print(EEProm.nodeID); Serial.print(F(" "));
      Serial.print(emontx.Msg); Serial.print(F(" "));
      Serial.print(emontx.P1); Serial.print(F(" "));

[and so on]

If the consensus is that the NodeID should be included for the emonHubTx3eInterfacer, then somebody had better tell me what’s required and have it approved before the emonPiCM and the supporting monitor nodes software is released, because I won’t be happy if I have to re-issue a handful of sketches and all the supporting documentation a week or two later.

However, what I have included in the ‘new’ sketches to accompany the emonPiCM is a wait at startup that is a function of the Node ID. After the factory test is

   delay(EEProm.nodeID * 20);                   // try to avoid r.f. collisions at start-up

which should alleviate the problem of all three emonTx’s transmitting at exactly the same time (to begin with - it won’t prevent them from drifting together over time).
But for some time now, the transmitter code in rfm.ino has included a ‘hold-off’ whereby the transmitter listens on the channel before transmitting - if it detects a signal, it will wait until that has stopped before switching round and transmitting. It’s not a guaranteed solution, because the radio is half-duplex and takes a finite time to switch round from listening to transmitting, so two can be listening at the same time and decide it’s safe to transmit at almost exactly the same time, but it does seem to offer a significant improvement. If the sketch is relatively recent, it should be using that.

The problem is not sending the data, it’s the ‘option’ bit and finding a way to switch or select those when the presumption is that the emon** must be ‘plug-and-play’ and the user doesn’t have the capability of communicating with it (i.e. no programmer or other connection that will enable a human input).

1 Like

I think this argument doesn’t really stand else it would be true for all of the ‘options’ available. It is likely to only be needed for a select few customers, but to those, it would be exceptionally valuable (such as this one).

Perhaps this is a discussion for another thread but I would propose:

  1. An option to enable the inclusion of a NodeID in the serial data message.
  2. The ability to input and save the required NodeID
  3. The option to enable a Valid JSON output via the serial interface

By default all are disabled so the stock output remains the same as currently.

I’m happy to look at the EmonHub side of things for the Tx3 Interfacer.

1 Like

Bummer. I purchased a couple of these with the extra Tx’s I purchased recently. I didn’t realise there would be issues identifying the node from the Esp’s. oh well you live and learn.

Looks like some good discussion here on the possible solutions. Obviously I would love to see some of these options implemented but understand I am in the minority.

I’ll leave it to you guys to debate, but if there is anything I can do to help please let me know. Would be happy to test anything you need me to.

The truth is, there are no options unless (a) one of the two DIP switches is used, or (b) there is some other detectable physical change, or (c) a programmer or equivalent is available, either to reload a new or edited sketch or at run-time to configure the loaded sketch.

I don’t have an ESP8266, so I know nothing about them other than what’s available generally that I’ve had the time to look at. I thought that could be set within the ESP, as the presumption is the FTDI/serial interface is wired and that uniquely defines the source to the ESP (or wherever the wires go).

I think part of @borpin’s argument for the Pi Zero W over the ESP8266 is the ability to (more easily?) define this.

Makes sense, I will look at the PiZero option when I get to that part.