I don’t know why emoncms does this, since my old emoncms installation didn’t do this, I’m basically using node red to read data from a shelly EM to then send it to emoncms, but the problem is that sometimes when the raspberry pi gets a reboot it just makes a new input with the same exact name so the data stops logging completely which means every time I have to configure the logging for the new input, is there any solution to this, idk if it’s emoncms or node-red’s emocms
Hello @Anonimo_LLopi which version of emoncms are you running? Im pretty sure we implemented a number of fixes for this, but could certainly be that we’ve missed something…
It’s version 10.8.5 “low write”, it came preinstalled with ICC program ICM and had most of the modules installed disabled by having a – on those modules’ folders I removed the – and they appeared. Idk if the issue could be the low write thingy, I’m not sure how to remove the low write mode and I don’t care about writings since I have it on a 256gb SSD. Cheers
I’ve still had this a couple of times, with the latest version of EmonCMS (updated via GIT).
It’s annoying but as the OP mentioned it seems to happen after a restart.
Top Tip: You don’t need to recreate the feed processing, you just need to delete the new (blank) input. Everything will then return to normal.
@TrystanLea The previous Emoncms version I had was 10.1.6 if it helps, as I said before I didn’t have this problem in that version, my raspberry pi gets a reboot on it’s own up to several times a day so this bug is quite annoying
@Vster Thanks for that workarround, it does work
Ok interesting is that on version 11.0.5? Any idea why your system is rebooting? Could it be a power supply issue?
Nope, I’ve updated to 11.0.5 and it’s still happening, it doesn’t happen in every reboot, infact it took 3 reboots for the bug to happen.
And no, it’s actually rebooting on purpose, ICM tends to freeze and I did a simple program with node red that detects when a input on emoncms didn’t get a input update (sent by ICM, if ICM is frozen no input will be sent to emoncms) in 3 minutes so then it just resets the raspberry pi, it works pretty well except for this input thing
Is the Node-Red instance on the Pi as well?
Hard rebooting the device Emoncms is on is a bad idea as data in REDIS may get lost. If you can break the ICM bit onto a PiZero or similar so Emoncms does not get restarted will help.
[edit]
Are you using stock firmware on the Shelly?
1 Like
Well, first off ICM (from ICC software in case you are thinking about some other ICM) cannot be run in anything less than a raspberry 2 or 3 and is already bought for my raspberry pi 4.
Also I’m not sure that the auto reboot I’m doing is considered a hard reboot or not since it’s not like pressing the power switch on the raspberry power supply meanwhile the os is running, it is a node red palette thing that tells the os to reboot and I never had data loss afaik and even if I did a few minutes of data is not that important since I get inverter data every day
A few minutes of data is ok but Lossing a whole day of data from my Shelly em (grid W, microinverter W etc) because of this input bug is definitely annoying
Cheers
Edit: Shelly is stock, there is no problem with it afaik
OK, I’m not really clear what the ICM is, how it is connected and what you are doing with it. You won’t lose a day but probably a few minutes at most.
Any reboot has the potential bad effect on emoncms.
A PiZ is as powerful as a Pi2/3. It will happily run Emoncms :).
Hello @Anonimo_LLopi do you know how the ICC software is communicating with the emoncms installation? is it via MQTT or the HTTP post api or bulk api?
My guess at the moment as to what is happening is that after the reboot somehow the redis cache already has some previous but incomplete data present in it (it should be blank after a reboot). That previous but incomplete cache is causing the redis cache of the input list to keep adding replicas of existing inputs, it looks like you are getting cloned inputs and cloned input processing, so it’s not quite the same as the previous issues that we saw with just the input name but not the processing being duplicated…
I cant replicate the issue here yet unfortunately
I’m not sure but here are the posting settings if that tells you which one it is, but ICM data doesn’t get duplicated inputs, probably since ICM takes a minute to start up from the Raspbian os starting up, ICM doesn’t have anything to do with node red or the Shelly
The issue is then probably caused by node red starting up right when the os starts and sending the Shelly data inputs too fast before things got reset/initialized properly (I do have a 256gb sata SSD)
Unfortunately I have currently disconnected my Shelly temporarily because I had my deye inverter set temporarily (everything working but with cables in the air) and I started doing the cables properly today, the next day I have time to finish the cable management it will be connected again so I can’t currently test it
Cheers
Here’s a update, I’ve put the shelly back and I changed a thing in node red so the program that reads the shelly and sends it to emoncms instead of starting as fast as node red starts when raspberry os boots up I changed so it starts 30 seconds after node red starts
You would think that would have maybe fixed it because I’m giving it 30 seconds for the redis cache thing to start properly, but NO it made it worse because not only the did the shelly inputs sent by node red got duplicated, the whole 60 inputs from the ICM program also got duplicated (first time it happened) and I had to select every one of them and delete them. ICM takes less than 30 seconds to start
Anyways, I’ve had 1 idea to might fix (more like evade) this duplication thing, basically I made a sacrificial test input from node red that gets sent when node red starts up, this way if my idea works redis will start up with this test node and if only this test input gets duplicated it won’t matter, and when the duplication has happened the other inputs that get sent later (30s from start nodered and ~30s ICM inputs) wont get duplicated, this idea has worked with no issues for now
It’s been happening for ages here, especially following a reboot (which happens every night after a backup otherwise emoncms doesn’t restart properly after stopping and restarting various services). My workaround is to monitor the feeds and reset all the inputs if any feeds stop updating (check the inputs api for how to do this). This had been discussed previously.
Well since I made that last post (the whole sending that blank/useless/test input first 30s before the important inputs) I haven’t had a single duplicated input (not even the test input), that or I’m being lucky right now
Cheers
So, duplicated input happened, the test input only, nice, the test input that gets sent first does get duplicated, so this is very nice workarround to this bug so the important inputs don’t get duplicated:
@TrystanLea - interesting - could it be an initialization issue?
However, I have seen it on a running system, but not for a while.
Probably one for a different thread, but is it the backup script that fails or your own version?