Delete multiple input nodes (100s)

borpin · 2018-11-02T20:18:21.591Z

I have discovered (the hard way) that EmonCMS does not like MQTT topics with ‘:’ in them. Issue #1105.

I now have 100s (it has been running a while) of inputs. How can I delete them in one go…

Paul · 2018-11-02T22:07:09.166Z

That should keep you out of mischief for a while!

borpin · 2018-11-02T22:09:08.334Z

Paul:

That should keep you out of mischief for a while!

That I doubt…

So no easy way, clicking takes about a minute each

pb66 · 2018-11-02T23:16:15.759Z

The input/clean api call will delete all inputs that do not have a processlist.

Beware of any valid nodes with inputs that have no processes getting reordered e.g. a node with inputs a,b and c. If b has no processes it will get deleted and then recreated when that node posts again so inputs will then get listed inthe order 1,3,2. To preserve the order of valid nodes, just add a dummy process before using the api eg “x1” or “reset to 0” etc.

borpin · 2018-11-02T23:28:54.663Z

Ah brilliant!!

Deleted 5803 inputs

icenov · 2019-10-05T01:06:40.223Z

Similar problem here, except the input/clean api call is not removing the last 250 odd empty nodes (input/clean took out a few thousand!). I’m not sure how/why (but suspect MQTT is involved as it happened sometime after adding in a few extra sensors via MQTT). I have been manually deleting the empty nodes, but looking for a faster solution! There is no processing on these nodes.

Screenshot%20from%202019-10-05%2010-52-171242×654 19.6 KB

the last few before real inputs start showing:

Screenshot%20from%202019-10-05%2010-52-561253×564 10.1 KB

Any thoughts on how to remove them efficiently?
I’m running the latest version:

Server Information

Services

emonhub :- Active Running
emoncms_mqtt :- Active Running
feedwriter :- Active Running - sleep 60s 153 feed points pending write
service-runner :- Active Running
emonPiLCD :- Active Exited
redis-server :- Active Running
mosquitto :- Active Running

Emoncms

Version :- low-write 10.1.8
Modules :- Administration | App v2.0.6 | Backup v2.1.3 | EmonHub Config v2.0.4 | Dashboard v2.0.4 | Device v2.0.1 | EventProcesses | Feed | Graph v2.0.4 | Input | Postprocess v2.1.0 | CoreProcess | Schedule | Network Setup v1.0.0 | sync | Time | User | Visualisation | WiFi v2.0.2
Git :-
- URL :- GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data
- Branch :- * stable
- Describe :- 10.1.8-4-gfc53af03

Server

OS :- Linux 4.14.71-v7+
Host :- emonpi | emonpi | (192.168.20.24)
Date :- 2019-10-05 11:03:15 AEST
Uptime :- 11:03:15 up 13 days, 18:50, 1 user, load average: 0.02, 0.18, 0.23

Memory

RAM :- Used: 30.10%
- Total :- 976.74 MB
- Used :- 293.97 MB
- Free :- 682.77 MB
Swap :- Used: 0.00%
- Total :- 100 MB
- Used :- 0 B
- Free :- 100 MB

Disk

/ :- Used: 45.62%
- Total :- 3.81 GB
- Used :- 1.74 GB
- Free :- 1.89 GB
/boot :- Used: 51.69%
- Total :- 42.52 MB
- Used :- 21.98 MB
- Free :- 20.54 MB
/home/pi/data :- Used: 3.97%
- Total :- 25.31 GB
- Used :- 1.01 GB
- Free :- 23.02 GB

HTTP

Server :- Apache/2.4.25 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

Version :- 5.5.5-10.1.23-MariaDB-9+deb9u1
Host :- localhost:6379 (127.0.0.1)
Date :- 2019-10-05 11:03:14 (UTC 10:00‌)
Stats :- Uptime: 1197964 Threads: 3 Questions: 2069494 Slow queries: 0 Opens: 36 Flush tables: 1 Open tables: 25 Queries per second avg: 1.727

Redis

Version :- 3.2.6
Host :- localhost:6379
Size :- 3283 keys (1.10M)
Uptime :- 13 days

MQTT Server

Version :- Mosquitto 1.4.10
Host :- localhost:1883 (127.0.0.1)

PHP

Version :- 7.0.30-0+deb9u1 (Zend Version 3.0.0)
Modules :- apache2handler | calendar v7.0.30-0+deb9u1 | Core v7.0.30-0+deb9u1 | ctype v7.0.30-0+deb9u1 | curl v7.0.30-0+deb9u1 | date v7.0.30-0+deb9u1 | dom v20031129 | exif v7.0.30-0+deb9u1 | fileinfo v1.0.5 | filter v7.0.30-0+deb9u1 | ftp v7.0.30-0+deb9u1 | gd v7.0.30-0+deb9u1 | gettext v7.0.30-0+deb9u1 | hash v1.0 | iconv v7.0.30-0+deb9u1 | igbinary v2.0.1 | json v1.4.0 | libxml v7.0.30-0+deb9u1 | mbstring v7.0.30-0+deb9u1 | mcrypt v7.0.30-0+deb9u1 | mosquitto v0.4.0 | mysqli v7.0.30-0+deb9u1 | mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $ | openssl v7.0.30-0+deb9u1 | pcre v7.0.30-0+deb9u1 | PDO v7.0.30-0+deb9u1 | pdo_mysql v7.0.30-0+deb9u1 | Phar v2.0.2 | posix v7.0.30-0+deb9u1 | readline v7.0.30-0+deb9u1 | redis v4.1.1 | Reflection v7.0.30-0+deb9u1 | session v7.0.30-0+deb9u1 | shmop v7.0.30-0+deb9u1 | SimpleXML v7.0.30-0+deb9u1 | sockets v7.0.30-0+deb9u1 | SPL v7.0.30-0+deb9u1 | standard v7.0.30-0+deb9u1 | sysvmsg v7.0.30-0+deb9u1 | sysvsem v7.0.30-0+deb9u1 | sysvshm v7.0.30-0+deb9u1 | tokenizer v7.0.30-0+deb9u1 | wddx v7.0.30-0+deb9u1 | xml v7.0.30-0+deb9u1 | xmlreader v7.0.30-0+deb9u1 | xmlwriter v7.0.30-0+deb9u1 | xsl v7.0.30-0+deb9u1 | Zend OPcache v7.0.30-0+deb9u1 | zlib v7.0.30-0+deb9u1

Pi

Model :- Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)
Serial num. :- 5F51CEB5
Temperature :- 48.85°C - 48.9°C
emonpiRelease :- emonSD-30Oct18
File-system :- read-write

Client Information

HTTP

Browser :- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Language :- en-US,en;q=0.9,nb;q=0.8,zu;q=0.7

Window

Size :- 1378 x 945

Screen

Resolution :- 1600 x 1200

TrystanLea · 2019-10-05T12:40:22.259Z

Hello @icenov there isnt unfortunately an easy way to delete the nodes at the moment via a similar API.

If your happy to sort them via the database, here are the steps:

mysql -u emoncms -p emoncms (password: emonpiemoncmsmysql2016)
DELETE FROM device WHERE userid = 1;
exit
redis-cli flushall

then refresh the interface. This assumes the account you wish to delete devices from is userid=1

borpin · 2019-10-05T16:53:09.453Z

icenov:

I’m not sure how/why (but suspect MQTT

I had 2 machines connecting with the same ClientID when it happened to me. I’d done an EmonCMS export / import so had 2 machines talking to the MQTT broker with the same ID.

icenov · 2019-10-05T22:49:58.162Z

Thanks @TrystanLea - will this delete all inputs, and presumably legitimate inputs will be recreated?

TrystanLea · 2019-10-07T10:11:15.693Z

It will delete the device definitions but not the inputs, the correct devices will then get recreated from the inputs that you have left in place.

icenov · 2019-10-08T05:56:19.297Z

That’s fixed it - perfect! Many thanks

borpin · 2019-10-08T08:25:40.745Z

I suggested a while ago (Phantom Nodes in Inputs) that the Input list should be non-permanent so issues like this could just be flushed out of the list.

Still think this is not a bad idea as a means to eradicate random and unwanted Inputs appearing.

Idea is simply that you need to specifically nominate the Nodes, that EmonCMS has found, that are to appear in the Inputs page for processing.