Delete multiple input nodes (100s)

I have discovered (the hard way) that EmonCMS does not like MQTT topics with ‘:’ in them. Issue #1105.

I now have 100s (it has been running a while) of inputs. How can I delete them in one go…

image

That should keep you out of mischief for a while!
:sunglasses::sunglasses::sunglasses:

That I doubt…

So no easy way, clicking takes about a minute each :hot_face:

The input/clean api call will delete all inputs that do not have a processlist.

Beware of any valid nodes with inputs that have no processes getting reordered e.g. a node with inputs a,b and c. If b has no processes it will get deleted and then recreated when that node posts again so inputs will then get listed inthe order 1,3,2. To preserve the order of valid nodes, just add a dummy process before using the api eg “x1” or “reset to 0” etc.

1 Like

Ah brilliant!!

Deleted 5803 inputs

:rofl::rofl::rofl::rofl:

1 Like

Similar problem here, except the input/clean api call is not removing the last 250 odd empty nodes (input/clean took out a few thousand!). I’m not sure how/why (but suspect MQTT is involved as it happened sometime after adding in a few extra sensors via MQTT). I have been manually deleting the empty nodes, but looking for a faster solution! There is no processing on these nodes.


the last few before real inputs start showing:

Any thoughts on how to remove them efficiently?
I’m running the latest version:

Server Information

Server Information

Services

  • emonhub :- Active Running
  • emoncms_mqtt :- Active Running
  • feedwriter :- Active Running - sleep 60s 153 feed points pending write
  • service-runner :- Active Running
  • emonPiLCD :- Active Exited
  • redis-server :- Active Running
  • mosquitto :- Active Running

Emoncms

Server

  • OS :- Linux 4.14.71-v7+
  • Host :- emonpi | emonpi | (192.168.20.24)
  • Date :- 2019-10-05 11:03:15 AEST
  • Uptime :- 11:03:15 up 13 days, 18:50, 1 user, load average: 0.02, 0.18, 0.23

Memory

  • RAM :- Used: 30.10%
    • Total :- 976.74 MB
    • Used :- 293.97 MB
    • Free :- 682.77 MB
  • Swap :- Used: 0.00%
    • Total :- 100 MB
    • Used :- 0 B
    • Free :- 100 MB

Disk

  • / :- Used: 45.62%
    • Total :- 3.81 GB
    • Used :- 1.74 GB
    • Free :- 1.89 GB
  • /boot :- Used: 51.69%
    • Total :- 42.52 MB
    • Used :- 21.98 MB
    • Free :- 20.54 MB
  • /home/pi/data :- Used: 3.97%
    • Total :- 25.31 GB
    • Used :- 1.01 GB
    • Free :- 23.02 GB

HTTP

  • Server :- Apache/2.4.25 (Raspbian) HTTP/1.1 CGI/1.1 80

MySQL

  • Version :- 5.5.5-10.1.23-MariaDB-9+deb9u1
  • Host :- localhost:6379 (127.0.0.1)
  • Date :- 2019-10-05 11:03:14 (UTC 10:00‌​)
  • Stats :- Uptime: 1197964 Threads: 3 Questions: 2069494 Slow queries: 0 Opens: 36 Flush tables: 1 Open tables: 25 Queries per second avg: 1.727

Redis

  • Version :- 3.2.6
  • Host :- localhost:6379
  • Size :- 3283 keys (1.10M)
  • Uptime :- 13 days

MQTT Server

  • Version :- Mosquitto 1.4.10
  • Host :- localhost:1883 (127.0.0.1)

PHP

  • Version :- 7.0.30-0+deb9u1 (Zend Version 3.0.0)
  • Modules :- apache2handler | calendar v7.0.30-0+deb9u1 | Core v7.0.30-0+deb9u1 | ctype v7.0.30-0+deb9u1 | curl v7.0.30-0+deb9u1 | date v7.0.30-0+deb9u1 | dom v20031129 | exif v7.0.30-0+deb9u1 | fileinfo v1.0.5 | filter v7.0.30-0+deb9u1 | ftp v7.0.30-0+deb9u1 | gd v7.0.30-0+deb9u1 | gettext v7.0.30-0+deb9u1 | hash v1.0 | iconv v7.0.30-0+deb9u1 | igbinary v2.0.1 | json v1.4.0 | libxml v7.0.30-0+deb9u1 | mbstring v7.0.30-0+deb9u1 | mcrypt v7.0.30-0+deb9u1 | mosquitto v0.4.0 | mysqli v7.0.30-0+deb9u1 | mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $ | openssl v7.0.30-0+deb9u1 | pcre v7.0.30-0+deb9u1 | PDO v7.0.30-0+deb9u1 | pdo_mysql v7.0.30-0+deb9u1 | Phar v2.0.2 | posix v7.0.30-0+deb9u1 | readline v7.0.30-0+deb9u1 | redis v4.1.1 | Reflection v7.0.30-0+deb9u1 | session v7.0.30-0+deb9u1 | shmop v7.0.30-0+deb9u1 | SimpleXML v7.0.30-0+deb9u1 | sockets v7.0.30-0+deb9u1 | SPL v7.0.30-0+deb9u1 | standard v7.0.30-0+deb9u1 | sysvmsg v7.0.30-0+deb9u1 | sysvsem v7.0.30-0+deb9u1 | sysvshm v7.0.30-0+deb9u1 | tokenizer v7.0.30-0+deb9u1 | wddx v7.0.30-0+deb9u1 | xml v7.0.30-0+deb9u1 | xmlreader v7.0.30-0+deb9u1 | xmlwriter v7.0.30-0+deb9u1 | xsl v7.0.30-0+deb9u1 | Zend OPcache v7.0.30-0+deb9u1 | zlib v7.0.30-0+deb9u1

Pi

  • Model :- Raspberry Pi 3 Model B Rev 1.2 - 1GB (Sony UK)

  • Serial num. :- 5F51CEB5

  • Temperature :- 48.85°C - 48.9°C

  • emonpiRelease :- emonSD-30Oct18

  • File-system :- read-write

Client Information

Client Information

HTTP

  • Browser :- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
  • Language :- en-US,en;q=0.9,nb;q=0.8,zu;q=0.7

Window

  • Size :- 1378 x 945

Screen

  • Resolution :- 1600 x 1200

Hello @icenov there isnt unfortunately an easy way to delete the nodes at the moment via a similar API.

If your happy to sort them via the database, here are the steps:

mysql -u emoncms -p emoncms (password: emonpiemoncmsmysql2016)
DELETE FROM device WHERE userid = 1;
exit
redis-cli flushall

then refresh the interface. This assumes the account you wish to delete devices from is userid=1

I had 2 machines connecting with the same ClientID when it happened to me. I’d done an EmonCMS export / import so had 2 machines talking to the MQTT broker with the same ID.

Thanks @TrystanLea - will this delete all inputs, and presumably legitimate inputs will be recreated?

It will delete the device definitions but not the inputs, the correct devices will then get recreated from the inputs that you have left in place.

That’s fixed it - perfect! Many thanks

I suggested a while ago (Phantom Nodes in Inputs) that the Input list should be non-permanent so issues like this could just be flushed out of the list.

Still think this is not a bad idea as a means to eradicate random and unwanted Inputs appearing.

Idea is simply that you need to specifically nominate the Nodes, that EmonCMS has found, that are to appear in the Inputs page for processing.

2 Likes