It looks like data is pasing through emonhub ok, next thing to check as Brian suggests it the mqtt_input service
sudo systemctl status mqtt_input
if it’s not running try
sudo systemctl restart mqtt_input
sudo systemctl status mqtt_input
It looks like data is pasing through emonhub ok, next thing to check as Brian suggests it the mqtt_input service
sudo systemctl status mqtt_input
if it’s not running try
sudo systemctl restart mqtt_input
sudo systemctl status mqtt_input
Brian,
Many thanks for answering my cry for help. I’m not much of a programmer so I’m easily confused when something goes wrong.
To answer your first question. My weewx software has the following info in weewx.conf:
# This section is for uploading data to Internet sites
[StdRESTful]
[[EmonCMS]]
url = http://192.168.1.115/emoncms/input/post.json
token = [token from emon]
As for the emoncms, I have the following:
● emonhub.service - LSB: Start/stop emonHub
Loaded: loaded (/etc/init.d/emonhub)
Active: active (exited) since Mon 2018-03-19 14:19:28 EDT; 3h 29min ago
● feedwriter.service - LSB: feedwriter script daemon
Loaded: loaded (/etc/init.d/feedwriter)
Active: active (running) since Mon 2018-03-19 16:37:51 EDT; 1h 11min ago
● mqtt_input.service - Emoncms MQTT Input Script
Loaded: loaded (/etc/systemd/system/mqtt_input.service; enabled)
Active: active (running) since Mon 2018-03-19 14:19:51 EDT; 3h 29min ago
I restarted the emonhub service. Earlier, I got the same “active (exited)” message and restarted it. However,
this didn’t solve the problem.
My “inputs” do show activity but the “feeds” show 30 hours since updated.
As I mentioned, the weather station, on the same local network (Ethernet) has been running and updating all along.
Yesterday I did an apt update/upgrade and an rpi-update on the firmware. I also did an emonpi update which I do several times a week and repeated this morning.
Much appreciated for the help so far.
Bob
I think what Brian was trying to determine when asking about your weatherstation was how the data was routed to emoncms to help debug the issue you have, which it looks like the weatherstation is posting direct to emoncms via http, so that narrows the field a little.
can you provide some emonhub.logs, preferably leading up to when it fails, if you don’t have those you mat need to wait til it occurs again.
Is that “30hrs” reducing? If there was data in the emonhub buffers when you restarted emonhub it will have beeb lost and you will have a hole in your data. But it should resume logging if your inputs are updating.
Paul,
emonCMS has been down for about 30 hours and the number is advancing. An app on my iPad that usually shows the day’s KWHs now shows error, cannot connect to emonCMS.
end of latest emonhub.log:
2018-03-19 18:37:29,268 INFO MQTT Publishing: emonhub/rx/8/values 256.512,331.328,136,81,121.98448,300,300,300,300,300,300,0,-32
2018-03-19 18:37:33,090 DEBUG RFM2Pi 340 NEW FRAME : OK 23 142 0 0 0 4 2 30 0 1 0 0 0 (-68)
2018-03-19 18:37:33,092 DEBUG RFM2Pi 340 Timestamp : 1521499053.09
2018-03-19 18:37:33,093 DEBUG RFM2Pi 340 From Node : 23
2018-03-19 18:37:33,094 DEBUG RFM2Pi 340 Values : [14.200000000000001, 0, 51.6, 3, 1]
2018-03-19 18:37:33,095 DEBUG RFM2Pi 340 RSSI : -68
2018-03-19 18:37:33,095 DEBUG RFM2Pi 340 Sent to channel(start)' : ToEmonCMS
2018-03-19 18:37:33,096 DEBUG RFM2Pi 340 Sent to channel(end)' : ToEmonCMS
2018-03-19 18:37:33,308 DEBUG MQTT Publishing: emon/emonth5/temperature 14.2
2018-03-19 18:37:33,310 DEBUG MQTT Publishing: emon/emonth5/external temperature 0
2018-03-19 18:37:33,312 DEBUG MQTT Publishing: emon/emonth5/humidity 51.6
2018-03-19 18:37:33,314 DEBUG MQTT Publishing: emon/emonth5/battery 3
2018-03-19 18:37:33,316 DEBUG MQTT Publishing: emon/emonth5/pulsecount 1
2018-03-19 18:37:33,318 DEBUG MQTT Publishing: emon/emonth5/rssi -68
2018-03-19 18:37:33,320 INFO MQTT Publishing: emonhub/rx/23/values 14.2,0,51.6,3,1,-68
2018-03-19 18:37:39,066 DEBUG RFM2Pi 341 NEW FRAME : OK 8 121 0 152 0 141 0 81 0 68 43 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-20)
2018-03-19 18:37:39,069 DEBUG RFM2Pi 341 Timestamp : 1521499059.07
2018-03-19 18:37:39,070 DEBUG RFM2Pi 341 From Node : 8
2018-03-19 18:37:39,070 DEBUG RFM2Pi 341 Values : [258.6496, 324.91519999999997, 141, 81, 121.99549440000001, 300, 300, 300, 300, 300, 300, 0]
2018-03-19 18:37:39,071 DEBUG RFM2Pi 341 RSSI : -20
2018-03-19 18:37:39,072 DEBUG RFM2Pi 341 Sent to channel(start)' : ToEmonCMS
2018-03-19 18:37:39,072 DEBUG RFM2Pi 341 Sent to channel(end)' : ToEmonCMS
2018-03-19 18:37:39,364 DEBUG MQTT Publishing: emon/emontx3/power1 258.6496
2018-03-19 18:37:39,367 DEBUG MQTT Publishing: emon/emontx3/power2 324.9152
2018-03-19 18:37:39,369 DEBUG MQTT Publishing: emon/emontx3/power3 141
2018-03-19 18:37:39,371 DEBUG MQTT Publishing: emon/emontx3/power4 81
2018-03-19 18:37:39,374 DEBUG MQTT Publishing: emon/emontx3/vrms 121.9954944
2018-03-19 18:37:39,376 DEBUG MQTT Publishing: emon/emontx3/temp1 300
2018-03-19 18:37:39,379 DEBUG MQTT Publishing: emon/emontx3/temp2 300
2018-03-19 18:37:39,381 DEBUG MQTT Publishing: emon/emontx3/temp3 300
2018-03-19 18:37:39,383 DEBUG MQTT Publishing: emon/emontx3/temp4 300
2018-03-19 18:37:39,386 DEBUG MQTT Publishing: emon/emontx3/temp5 300
2018-03-19 18:37:39,388 DEBUG MQTT Publishing: emon/emontx3/temp6 300
2018-03-19 18:37:39,390 DEBUG MQTT Publishing: emon/emontx3/pulse 0
2018-03-19 18:37:39,392 DEBUG MQTT Publishing: emon/emontx3/rssi -20
2018-03-19 18:37:39,395 INFO MQTT Publishing: emonhub/rx/8/values 258.6496,324.9152,141,81,121.9954944,300,300,300,300,300,300,0,-20
2018-03-19 18:37:49,055 DEBUG RFM2Pi 342 NEW FRAME : OK 8 121 0 156 0 136 0 79 0 89 43 184 11 184 11 184 11 184 11 184 11 184 11 0 0 0 0 (-25)
2018-03-19 18:37:49,058 DEBUG RFM2Pi 342 Timestamp : 1521499069.05
2018-03-19 18:37:49,059 DEBUG RFM2Pi 342 From Node : 8
2018-03-19 18:37:49,060 DEBUG RFM2Pi 342 Values : [258.6496, 333.4656, 136, 79, 122.2267968, 300, 300, 300, 300, 300, 300, 0]
2018-03-19 18:37:49,061 DEBUG RFM2Pi 342 RSSI : -25
2018-03-19 18:37:49,062 DEBUG RFM2Pi 342 Sent to channel(start)' : ToEmonCMS
2018-03-19 18:37:49,062 DEBUG RFM2Pi 342 Sent to channel(end)' : ToEmonCMS
2018-03-19 18:37:49,160 DEBUG MQTT Publishing: emon/emontx3/power1 258.6496
2018-03-19 18:37:49,162 DEBUG MQTT Publishing: emon/emontx3/power2 333.4656
2018-03-19 18:37:49,163 DEBUG MQTT Publishing: emon/emontx3/power3 136
2018-03-19 18:37:49,166 DEBUG MQTT Publishing: emon/emontx3/power4 79
2018-03-19 18:37:49,168 DEBUG MQTT Publishing: emon/emontx3/vrms 122.2267968
2018-03-19 18:37:49,169 DEBUG MQTT Publishing: emon/emontx3/temp1 300
2018-03-19 18:37:49,171 DEBUG MQTT Publishing: emon/emontx3/temp2 300
2018-03-19 18:37:49,173 DEBUG MQTT Publishing: emon/emontx3/temp3 300
2018-03-19 18:37:49,175 DEBUG MQTT Publishing: emon/emontx3/temp4 300
2018-03-19 18:37:49,177 DEBUG MQTT Publishing: emon/emontx3/temp5 300
2018-03-19 18:37:49,178 DEBUG MQTT Publishing: emon/emontx3/temp6 300
2018-03-19 18:37:49,180 DEBUG MQTT Publishing: emon/emontx3/pulse 0
2018-03-19 18:37:49,182 DEBUG MQTT Publishing: emon/emontx3/rssi -25
2018-03-19 18:37:49,184 INFO MQTT Publishing: emonhub/rx/8/values 258.6496,333.4656,136,79,122.2267968,300,300,300,300,300,300,0,-25
Thanks.
Bob
Are you able to log into emoncms? You have said
“My “inputs” do show activity” and “emonCMS has been down for about 30 hours” which conflict.
Assuming you have not changed any input processing, if the inputs are updating the feeds should be logging, if they are not it could be the feed writer
sudo systemctl status feedwriter
but that appeared fine in your earlier info.
● feedwriter.service - LSB: feedwriter script daemon
Loaded: loaded (/etc/init.d/feedwriter)
Active: active (running) since Mon 2018-03-19 16:37:51 EDT; 2h 12min ago
Process: 19726 ExecStop=/etc/init.d/feedwriter stop (code=exited, status=0/SUCCESS)
Process: 19741 ExecStart=/etc/init.d/feedwriter start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/feedwriter.service
└─19758 /usr/bin/php -f /var/www/emoncms/scripts/feedwriter.php
Mar 19 16:37:44 emonpi feedwriter[19741]: Log is turned off
Mar 19 16:37:44 emonpi feedwriter[19741]: Starting RPI
Mar 19 16:37:51 emonpi systemd[1]: Started LSB: feedwriter script daemon.
I can log into emonCMS and the “inputs” screen shows they are getting data on schedule. The “feeds” screen shows 30 hours since anything was received (except for the Node 0 which is the weather station and it shows data every 15 minutes which is correct).
Is there space for data?
df -h
Have the permissions/ownership of the data files been altered? There would normally be some errors in the emoncms.log if this was the case (viewed via admin page).
but you could try this to check them
ls -la /home/pi/data/php*
If the data files are the correct path, ownership and permissions, there is room for the data and the feedwriter is running ok, then it would suggest an issue in the input processing. Have you made any changes?
None of the above explains why your tablet won’t connect when your weatherstation can. Have you tried a browser refresh or tried navigating to another emoncms page?
ls -la /home/pi/data/php*
/home/pi/data/phpfina:
total 122349
drwxr-xr-x 2 www-data root 1024 Nov 24 14:26 .
drwxrwxrwx 10 pi pi 1024 Nov 20 07:58 ..
-rw-r--r-- 1 www-data www-data 7252312 Mar 18 12:38 14.dat
-rw-r--r-- 1 www-data www-data 16 Aug 20 2017 14.meta
…etc
/home/pi/data/phptimeseries:
total 2
drwxr-xr-x 2 www-data root 1024 Nov 7 2016 .
drwxrwxrwx 10 pi pi 1024 Nov 20 07:58 ..
root@emonpi(ro):emonhub# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 3.4G 2.2G 1.1G 69% /
devtmpfs 484M 0 484M 0% /dev
tmpfs 489M 0 489M 0% /dev/shm
tmpfs 489M 13M 476M 3% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 489M 0 489M 0% /sys/fs/cgroup
tmpfs 40M 6.1M 34M 16% /var/lib/openhab
tmpfs 1.0M 8.0K 1016K 1% /var/lib/dhcpcd5
tmpfs 50M 5.5M 45M 11% /var/log
tmpfs 30M 144K 30M 1% /tmp
tmpfs 1.0M 4.0K 1020K 1% /var/lib/dhcp
/dev/mmcblk0p1 60M 23M 38M 38% /boot
/dev/mmcblk0p3 25G 159M 24G 1% /home/pi/data
tmpfs 98M 0 98M 0% /run/user/1000
I haven’t changed anything except for the updates. Nothing in the feeds, configurations, etc…
After the rpi updates, everything ran fine for about 8 hours.
I can log into emonCMS but it just isn’t getting any data in the feeds. Refresh does nothing new.
The system was working very well and I was getting data withing several KWHs of the power company over two months. All of a sudden it stops sending data to the established feeds…
Thanks,
Bob
PS Don’t you guys ever sleep…?_
Of course we do, I’m actually asleep right now! (Well not far off)
I’m afraid I’m running a bit short on idea’s right now. If you have run updates and restarted services I don’t think you will have any data buffered anywhere, it might be worth trying a reboot and keeping an eye on it if it appears ok. It’s not a fix, but it might help get you restarted quicker, and it will tell us if this is a permanent fault or not (if it persists a reboot).
I’ll reboot and get back with you tomorrow. Thanks again for the help.
Bob
After reboot. Feeds not updating. Not sure why there are two sets. Let me know if the page print doesn’t come through and I’ll resend it. Thanks
Bob
I can see what’s happened but I can’t say why at this point. But if you delete all the second set of inputs, essentially delete all the green updating inputs, but be aware as you do the red ones will turn green so be careful not to delete too many. Start at the 2nd “power1” (first green input) and work down the list as each is deleted it will move up a position.
What’s happened is there has been a “glitch” that has resulted in a duplicate set of inputs and as they are the newest, they are getting updated whilst the original ones with your processing are getting stale.
Once deleted all will be well, but you should keep an eye on it in case it reoccurs.
I followed your directions and seem to have everything back to normal. After deleting the duplicate inputs, I had to reconfigure several of the feeds which were more complicated than a simple “log to feed.” Do you have any idea what might have caused the problem? As I mentioned before, the only changes I made were the updates to the rpi firmware/software and to the emon software and things ran fine for about 8 hours.
After adjustments to the AC voltage and CTs (Fluke meter), my measurements were coming within several KWHs of what the power company charges me for (average 1200/month).
Much appreciated for the help.
Bob
That was what my rather wordy explanations were trying to head off.
There was no need to delete any inputs that had processing, the duplicate inputs were void of any processing so it sounds like you got a little keen with the deleting.
In short, no I do not know why it occurs, but I am pleased to say we see it less often these days. I think there is something causing the originals not to be found at the right time, eg redis not populating at the first input, maybe it clashes with something else, I have no idea. But it does seem rare these days. If it does happen again, please try and recall what might be the trigger, now you know what to look for you might notice it nearer the event. But it may never happen again.
@TrystanLea on an EmonPi is there a requirement to check the databases or does that update button do it for the user?
It’s included in the emonpi update routine.
@pb66 one other general thought, I’ve seen myself when posting multiple items to MQTT from a microprocessor, it is not always as quick as you’d think, which can cause problems. As part of the emonhub development perhaps using a JSON format that posts the lot as a single MQTT feed might help.
I totally agree, Have a read of the EmonHub Development thread. Trystan is looking into making the mqtt inputs in emoncms work as per the http bulk upload and changing emoncms so that both indexed (csv) and named (key:value) inputs can at last be used interchangeably rather than having to use one or the other exclusively (per input), emonhub will hopefully be able to post complete payloads via mqtt and in batches too if buffering is needed (eg mqtt across machines or networks).
Yes I’d read it which made me think of it. The JSON format for MQTT or the input API works well for me. Hopefully it will go into the next stable which must be due soon.
I had a similar ‘glitch’ where I had duplicate feeds (rather than inputs as in the screenshot above) after a reboot after a power outage. It occurred originally on v9.8.7, and the issue persisted through to v9.8.27 “low write” when I appear to have fixed it. I was never sure how it occurred, but after every reboot the annoying duplicate feeds names came back. I’m posting in this thread in case it helps anyone else…. the symptoms are the same, the apps and dashboards fail to work because they are linked to feeds that don’t get updated.
With the original issue I deleted the feeds in the GUI, but could not recreate them as the system said they 'already existed’. The solution to this was
After every reboot, the system still failed to work because the old deleted feed-ids seemed to come back (even though they weren’t in the SQL database and the inputs were being updated using names rather than ids). I discovered that on every reboot if I went into the administration page and pressed the ‘flush redis keys’ button, then the correct feed-ids were restored. Even after a ‘redis-cli FLUSHALL’ the issue still reoccurred after a reboot. The eventual solution that worked for me was to delete /var/lib/redis/dump.rdb.
I’ve now got a battery backup system based on a couple of 18650 cells from an old laptop, I hope that will avoid power outages in the future.