Weird problem; powercuts over night, now EmonCMS not recording

Hi,

We had a couple of short power cuts over night. Now my EmonCMS is not recording any more.

Digging does show that maybe the feeds are not receiving data since the last power cut :

And, indeed, the input I am pretty sure used to work is shown as dead :

But if I scroll alllll the way to the bottom of the inputs :

It is there working OK. It’s almost like it’s renamed itself for no obvious reason?

I can find the feed that this new input maps itself to but that does not seem to appear, and also the processing would be wrong that was done in the former setup :

The former looks like this:

Also the actual inputs seem to changed name somehow?

What I cannot seem to find is a way to reassign the old “8” input to what is being called “emontx3_discreet_8”. There just doesn’t seem to be any way to force it back. And I’m fairly confident it would be a mistake to try and copy all the stuff from the old “8” over to the “new” “emontx3_discreet_8”. Plus all my old data would then be broken off and lost.

Any ideas? Firstly about how to fix this mess, and secondly why a couple of short power cuts would trash it like this?

I think you’ve stumbled across a newish “feature” which appears to automatically create a new input if it sees a problem with the incoming data, rather than junking the corrupted data.

Definitely one for @TrystanLea to solve.

1 Like

One interesting thing which I’m not sure I’d noticed before is…

I have literally loads of inputs that I’ve no idea what they are!

(wondering if it’s seeing stray RF from other similar devices, but… IDK, seems a bit improbable).

If it is RF, I kind of wish there was a way of - just via a cable - connecting my sensors (CTs/flash counter) directly to my Pi. I have the EmonTX box and a receiver GPIO board on my Pi, but the two are literally 1m apart and could be easily cabled!

I took this up privately with Trystan a week ago. This is what he wrote:
"Good question, clearly the auto node feature is seeing a lot of node data that matches the data length of the template decoders but extends over a much wider / invalid node range. There is no filter on valid node id’s for this feature.

Question is why is there so many apparently valid messages coming through to emonhub… the code on the emonPi should be filtering some of this, though spurious nodes has always and continues to be a problem that probably deserves more attention."

I was running an endurance test on emonLibDB, which precluded updating the emonPi. I stopped the test (at just over 1¼ million reports - at 10 s intervals - without stopping or resetting) and updated emonCMS, whereupon it hasn’t given a problem - but this was only a week ago, so by no means conclusive. I cited “I have 517 spurious node definitions in my emonhub.conf, 509 being unwanted emonTx4 and one an emontxshield.” (And I’d cleared all the spurious ones out a few weeks before.)

But one thing that was different for me - it continued using the original Node name.

Is the “new” input actually writing to the original Feed with the original processing steps?

1 Like

What is your full set-up there?
How many sensors and what sort are connected where?
Do you envisage expanding it at some point in the future?

It’s just one EmonTX and one RPI connected by RF.

The EmonTX has a CT on main incomer, a CT on solar inverter, and a CT on the SOLIC.

And there’s a pulse counter thingy on the EmonTX too. And also the PSU passes a known fraction mains voltage too.

There is. You need the wired connection and the correct Interfacer in emonHub.
https://docs.openenergymonitor.org/emontx3/direct.html#direct-raspberry-pi-gpio
and maybe this might add some useful information:
https://docs.openenergymonitor.org/emonhub/emonhub-interfacers.html

You still need your power supply to the RPi, plus the a.c. adapter to measure the voltage.

Hello @alexbloor

is this on emoncms.org or on a local emonbase?

@TrystanLea
Here’s your answer:

I was thinking it wasn’t quite clear enough, if the screenshots above are from emoncms.org this could be an issue caused by the relatively recent update to emonhub that sends input names via the emoncms http interfacer. If this is local then the issue may instead be caused by the new autoconf feature.

Either way I would recommend moving the input processing to the new input names as these are more descriptive and deleting the old inputs.

Hi there, I don’t think I’ve done any updates to the emonhub unless it has done them automatically (without my say-so)?

I have several years back data, is there some way to consolidate this? I’m also not sure what it means to “move input processing to the new input names”? I cannot see a way of doing this.

I think my main concern is almost anything I think might be the right answer is likely to trash something else! Or prevent me importing/consolidating the old and “new” data. And I’m still not really any the wiser as to why a powercut caused the input names to all change without my say-so?

Does this help answer anything?

Ok so looking around and thinking about this a bit more.

I think I need to create a new feed, with all the same settings as the original feed, but talking to the input that is the same as the old input but with all different names.

I need to ‘post process’ the data that the “new” input stored (if it exists?).

And finally I need to consolidate, maybe by post processing the old data into the new data feed?

The problem is I really have exactly zero confidence in my ability to do any of these steps, or what the right order is…

It’d be really helpful if anyone could advise me on whether what I say above is at least vaguely sensible sounding?

Also, would anyone be able to tell me if I’m losing data at the moment? To reiterate, my “feeds” are 4.6 days out of date, and not updating, but I do have “up to date” input data. Is the input data being recorded in such a way that I can re-create the feed post-hoc? Or is it gone for ever?

I may yet need to dump OpenEnergyMonitor/EmonCMS/EmonPi/EmonTX altogether and change to something else as I am finding myself overwhelmed by the admin side of things; literally no idea what to do or where to go with this. All I can really see is it was working, then we had some power cuts, then for no really good reason I can see it broke itself and no longer really works, and in spite of going through the documentation, and asking on the forums, nothing really makes that much sense or has any easy answers :frowning:

Part of my problem is that because I’ve only implemented this once, and not over and over again in lots of situations, I really don’t feel I know and understand it that well. But I am sure there has to be a product that is simpler out there. Failing all the above, does anyone have a suggestion for something with CT clamps that maybe has a lot less features but basically draws graphs and/or has a cloud service that does all the pretty stuff? I need power monitoring and I don’t have any at the moment… so I need to look at other ideas to solve this :frowning:

Update: Has anyone tried Emporia stuff? It looks a lot less fiddly.

I thought I’d give one last try, but before I did so, do a full backup.

Even that doesn’t work :cry:

Looks great, right?

Nope. The backup is a 1 KB file.

So I can’t even back it up before I (risk) breaking it.

Scratch that. It hadn’t finished. But the fact it offered a link to an extant file made me think (reasonably) it had finished. I refreshed the page, and now the file is more like 150MB. Good. Ok so now I can probably semi-safely risk breaking everything, at least.

If I knew how to raise an suggestion, I’d point out that the UI ought to perhaps say what message is displayed in the logging once it is finished, and maybe suggest not displaying the .tar.gz link UNTIL that had appeared in the log.

Ok some progress : I manually copied over all the “8” input entries to the new ones with new names.

And, now, my feeds are updating again.

So that is cool. And a bit of a start, in that my realtime stuff; dashboards etc, are back.