Multiple problems after upgrade to 9.9.6

sjthespian · 2019-02-13T16:02:35.250Z

I was running 9.8.6 before the upgrade. The database update went fine with no errors and I’ve even tried running in an incognito window to make sure I have nothing cached on the browser side. I’ve checked a couple of different branches, the 9.9.0 tag is the last time the feeds work – 9.9.3 and on show the same issue that I’m having with 9.9.6.

The real issue seems to be the feeds. I have a hunch that if I can figure out what is causing that I will solve the dashboard and MySolar app issues.

TrystanLea · 2019-02-13T16:10:05.854Z

Hello @sjthespian, could you try going to the api emoncms/feed/list.json directly? Are there any errors in the result?

Do you see any errors in your browsers console log?

sjthespian · 2019-02-13T16:17:24.081Z

Ok, that is useful – it throws a couple of errors.

<b>Warning</b>:  fopen(/var/lib/phptimeseries/feed_90.MYD): failed to open stream: No such file or directory
<br/>
<b>Warning</b>:  filesize(): stat failed for /var/lib/phptimeseries/feed_90.MYD in <b>/srv/www/htdocs-www/emoncms/Modules/feed/engine/PHPTimeSeries.php</b> on line <b>71</b><br />

which is interesting, as I don’t even have a /var/lib/phptimeseries directory! I just have /var/lib/phpfina/ and /var/lib/phpfiwa/. Any idea when that was added? I just created it, but emoncms isn’t adding any data files to it. Is there a way to populate it from phpfina and phpfiwa?

TrystanLea · 2019-02-13T16:29:01.394Z

Glad that found something. Could you repeat your mysql command above but search for phptimeseries feeds:

select * from feeds where engine=2;

Perhaps there’s a feed that did not delete correctly… if its not there then it may be that the redis cache is out of sync with your mysql data, you could clear redis (assuming your not using redis for other applications) with:

redis-cli flushall

emoncms will then reload the content.

sjthespian · 2019-02-13T16:30:35.977Z

That turns up one feed:

mysql> select * from feeds where engine=2;
+----+-----------------+--------+--------+------------+-----------------+----------+--------+------+--------+-------------+--------+------+
| id | name            | userid | tag    | time       | value           | datatype | public | size | engine | processList | server | unit |
+----+-----------------+--------+--------+------------+-----------------+----------+--------+------+--------+-------------+--------+------+
| 90 | Main Power kWhd |      1 | Node 1 | 1469832794 | 6.7553338555542 |        2 |      0 | NULL |      2 | NULL        |      0 |      |
+----+-----------------+--------+--------+------------+-----------------+----------+--------+------+--------+-------------+--------+------+
1 row in set (0.01 sec)

And I just fixed the problem – although I still don’t know what caused it…

touch /var/lib/phptimeseries/feed_90.MYD

That lets the json feed return clean json with no HTML embedded in it, which fixes both my feeds and my dashboards! More importantly, the MySolar app now returns a dashboard (which is more than a bit depressing since it’s raining today and I’m not producing any power. )

Thanks for the pointers in the right direction!

borpin · 2019-02-13T16:34:01.298Z

sjthespian:

switch to the 9.9.6 branch in the base directory

I’d either leave it on stable or master branches.

Can you post the server information by clicking on ‘copy to clipboard’ and then just pasting here (no further formatting required).

TrystanLea · 2019-02-13T16:36:27.534Z

Great to hear that fixed it, not sure either what caused it, will keep an eye out for similar issues.

sjthespian · 2019-02-13T16:39:04.343Z

Server Information


Services
	redis-server	Active Running
Emoncms	Version	9.9.6
	Modules	Administration : App v1.0.0 : Dashboard v1.3.3 : energy : event : EventProcesses : Feed : Input : mqtt : notify : openbem : packetgen : CoreProcess : report : Schedule : Time : User : Visualisation
	Git	URL: GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data : Branch: master : Describe: 9.9.4-206-gacbfca0
Server	OS	Linux 4.4.0-141-generic
	Host	morpheus : morpheus.lapseofthought.com : (172.16.42.16)
	Date	2019-02-13 08:38:37 PST
	Uptime	08:38:37 up 39 days, 12:30, 1 user, load average: 0.43, 0.66, 0.94
HTTP	Server	nginx/1.14.1 HTTP/1.1 CGI/1.1 443
MySQL	Version	5.7.25-0ubuntu0.16.04.2-log
	Host	localhost (127.0.0.1)
	Date	2019-02-13 08:38:36 (UTC -08:00‌)
	Stats	Uptime: 1736004 Threads: 12 Questions: 15936957 Slow queries: 0 Opens: 69486010 Flush tables: 7 Open tables: 416 Queries per second avg: 9.180
Redis	Version	3.0.6
	Host	localhost:6379 (127.0.0.1)
	Size
	Uptime	39 days
Memory	RAM	Used: 77.09% Total: 7.69 GB Used: 5.93 GB Free: 1.76 GB
	Swap	Used: 8.01% Total: 39.89 GB Used: 3.2 GB Free: 36.7 GB
Disk	Mount	Stats
	/	Used: 68.34% Total: 450.08 GB Used: 307.59 GB Free: 119.61 GB
	/home	Used: 91.71% Total: 750 GB Used: 687.83 GB Free: 62.17 GB
	/boot	Used: 23.93% Total: 471.6 MB Used: 112.84 MB Free: 334.41 MB
	/srv/archive	Used: 65.26% Total: 400 GB Used: 261.03 GB Free: 138.97 GB
	/srv/backup	Used: 80.60% Total: 2 TB Used: 1.61 TB Free: 397.33 GB
	/srv/db	Used: 1.09% Total: 688.98 GB Used: 7.5 GB Free: 681.48 GB
	/srv/images	Used: 43.62% Total: 1.18 TB Used: 527.16 GB Free: 681.48 GB
	/srv/timemachine	Used: 44.76% Total: 1.2 TB Used: 549.95 GB Free: 678.85 GB
	/srv/www	Used: 11.87% Total: 773.28 GB Used: 91.79 GB Free: 681.48 GB
	/usbdisk5g	Used: 0.00% Total: 681.48 GB Used: 128 KB Free: 681.48 GB
	/usbdisk5g/containers	Used: 0.00% Total: 681.48 GB Used: 128 KB Free: 681.48 GB
	/usbdisk5g/deleted	Used: 0.00% Total: 681.48 GB Used: 128 KB Free: 681.48 GB
	/usbdisk5g/deleted/images	Used: 0.00% Total: 681.48 GB Used: 128 KB Free: 681.48 GB
	/var/lib/lxd/containers/chef-ubuntu.zfs	Used: 0.55% Total: 685.27 GB Used: 3.79 GB Free: 681.48 GB
	/var/lib/lxd/images/f70ceacb0c15661d8603d041c5c5e94a5de63fbac54803963be700cbed6f721e.zfs	Used: 0.09% Total: 682.11 GB Used: 640.63 MB Free: 681.48 GB
	/srv/media/books	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
	/srv/media/video	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
	/srv/media/mp3	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
PHP	Version	7.0.32-0ubuntu0.16.04.1 (Zend Version 3.0.0)
	Modules	calendar v7.0.32-0ubuntu0.16.04.1 : cgi-fcgi : Core v7.0.32-0ubuntu0.16.04.1 : ctype v7.0.32-0ubuntu0.16.04.1 : curl v7.0.32-0ubuntu0.16.04.1 : date v7.0.32-0ubuntu0.16.04.1 : dom v20031129 : exif v7.0.32-0ubuntu0.16.04.1 : fileinfo v1.0.5 : filter v7.0.32-0ubuntu0.16.04.1 : ftp v7.0.32-0ubuntu0.16.04.1 : gd v7.0.32-0ubuntu0.16.04.1 : gettext v7.0.32-0ubuntu0.16.04.1 : gnupg v1.3.6 : hash v1.0 : iconv v7.0.32-0ubuntu0.16.04.1 : imagick v3.4.0RC6 : imap v7.0.32-0ubuntu0.16.04.1 : intl v1.1.0 : json v1.4.0 : ldap v7.0.32-0ubuntu0.16.04.1 : libxml v7.0.32-0ubuntu0.16.04.1 : mbstring v7.0.32-0ubuntu0.16.04.1 : mcrypt v7.0.32-0ubuntu0.16.04.1 : mysqli v7.0.32-0ubuntu0.16.04.1 : mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $ : openssl v7.0.32-0ubuntu0.16.04.1 : pcre v7.0.32-0ubuntu0.16.04.1 : PDO v7.0.32-0ubuntu0.16.04.1 : pdo_mysql v7.0.32-0ubuntu0.16.04.1 : pdo_sqlite v7.0.32-0ubuntu0.16.04.1 : Phar v2.0.2 : posix v7.0.32-0ubuntu0.16.04.1 : pspell v7.0.32-0ubuntu0.16.04.1 : radius v1.4.0b1 : readline v7.0.32-0ubuntu0.16.04.1 : redis v3.1.1 : Reflection v7.0.32-0ubuntu0.16.04.1 : session v7.0.32-0ubuntu0.16.04.1 : shmop v7.0.32-0ubuntu0.16.04.1 : SimpleXML v7.0.32-0ubuntu0.16.04.1 : snmp v0.1 : soap v7.0.32-0ubuntu0.16.04.1 : sockets v7.0.32-0ubuntu0.16.04.1 : SPL v7.0.32-0ubuntu0.16.04.1 : sqlite3 v7.0.32-0ubuntu0.16.04.1 : standard v7.0.32-0ubuntu0.16.04.1 : sysvmsg v7.0.32-0ubuntu0.16.04.1 : sysvsem v7.0.32-0ubuntu0.16.04.1 : sysvshm v7.0.32-0ubuntu0.16.04.1 : tokenizer v7.0.32-0ubuntu0.16.04.1 : wddx v7.0.32-0ubuntu0.16.04.1 : xml v7.0.32-0ubuntu0.16.04.1 : xmlreader v7.0.32-0ubuntu0.16.04.1 : xmlwriter v7.0.32-0ubuntu0.16.04.1 : xsl v7.0.32-0ubuntu0.16.04.1 : Zend OPcache v7.0.32-0ubuntu0.16.04.1 : zip v1.13.5 : zlib v7.0.32-0ubuntu0.16.04.1

Client Information


HTTP	Browser	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
Screen	Resolution	1920 x 1200
Window	Size	1440 x 797

borpin · 2019-02-13T16:48:27.137Z

Oh *&^%$$% the fix for the formatting of the information does not seem to have worked

sjthespian · 2019-02-13T16:50:32.463Z

It sort-of works – it’s there, just a whole lot wider than the display… My guess it’s the entry with /var/lib/lxd/images/f70ceacb0c15661d8603d041c5c5e94a5de63fbac54803963be700cbed6f721e.zfs throwing off the column width.

borpin · 2019-02-13T16:52:53.365Z

@sjthespian, I just tried it for me in the sandbox and it worked which implies you are not actually correctly up to date.

It should look like this Test formatting of copy to clipboard

Can you please checkout the stable branch and the do git status please.

sjthespian · 2019-02-13T16:55:49.038Z

That cut-n-paste was from 9.9.6. I just switched to stable:

> git co stable
Previous HEAD position was acbfca0... Update version.txt
Branch stable set up to track remote branch stable from origin.
Switched to a new branch 'stable'
> git status
On branch stable
Your branch is up-to-date with 'origin/stable'.
nothing to commit, working directory clean

And here is a new copy of the server info:

Server Information


Services
	redis-server	Active Running
Emoncms	Version	9.9.6
	Modules	Administration : App v1.0.0 : Dashboard v1.3.3 : energy : event : EventProcesses : Feed : Input : mqtt : notify : openbem : packetgen : CoreProcess : report : Schedule : Time : User : Visualisation
	Git	URL: GitHub - emoncms/emoncms: Web-app for processing, logging and visualising energy, temperature and other environmental data : Branch: * stable : Describe: 9.9.4-205-g22d2e6c
Server	OS	Linux 4.4.0-141-generic
	Host	morpheus : morpheus.lapseofthought.com : (172.16.42.16)
	Date	2019-02-13 08:55:34 PST
	Uptime	08:55:34 up 39 days, 12:47, 1 user, load average: 1.82, 1.15, 1.04
HTTP	Server	nginx/1.14.1 HTTP/1.1 CGI/1.1 443
MySQL	Version	5.7.25-0ubuntu0.16.04.2-log
	Host	localhost (127.0.0.1)
	Date	2019-02-13 08:55:34 (UTC -08:00‌)
	Stats	Uptime: 1737021 Threads: 12 Questions: 15944529 Slow queries: 0 Opens: 69526899 Flush tables: 7 Open tables: 416 Queries per second avg: 9.179
Redis	Version	3.0.6
	Host	localhost:6379 (127.0.0.1)
	Size
	Uptime	39 days
Memory	RAM	Used: 80.83% Total: 7.69 GB Used: 6.22 GB Free: 1.47 GB
	Swap	Used: 8.00% Total: 39.89 GB Used: 3.19 GB Free: 36.7 GB
Disk	Mount	Stats
	/	Used: 68.33% Total: 450.08 GB Used: 307.53 GB Free: 119.67 GB
	/home	Used: 91.77% Total: 750 GB Used: 688.25 GB Free: 61.75 GB
	/boot	Used: 23.93% Total: 471.6 MB Used: 112.84 MB Free: 334.41 MB
	/srv/archive	Used: 65.26% Total: 400 GB Used: 261.03 GB Free: 138.97 GB
	/srv/backup	Used: 80.60% Total: 2 TB Used: 1.61 TB Free: 397.33 GB
	/srv/db	Used: 1.09% Total: 688.98 GB Used: 7.5 GB Free: 681.48 GB
	/srv/images	Used: 43.62% Total: 1.18 TB Used: 527.16 GB Free: 681.48 GB
	/srv/timemachine	Used: 44.76% Total: 1.2 TB Used: 549.95 GB Free: 678.85 GB
	/srv/www	Used: 11.82% Total: 772.86 GB Used: 91.37 GB Free: 681.48 GB
	/usbdisk5g	Used: 0.00% Total: 681.48 GB Used: 0 B Free: 681.48 GB
	/usbdisk5g/containers	Used: 0.00% Total: 681.48 GB Used: 0 B Free: 681.48 GB
	/usbdisk5g/deleted	Used: 0.00% Total: 681.48 GB Used: 0 B Free: 681.48 GB
	/usbdisk5g/deleted/images	Used: 0.00% Total: 681.48 GB Used: 0 B Free: 681.48 GB
	/var/lib/lxd/containers/chef-ubuntu.zfs	Used: 0.55% Total: 685.27 GB Used: 3.79 GB Free: 681.48 GB
	/var/lib/lxd/images/f70ceacb0c15661d8603d041c5c5e94a5de63fbac54803963be700cbed6f721e.zfs	Used: 0.09% Total: 682.11 GB Used: 640.63 MB Free: 681.48 GB
	/srv/media/books	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
	/srv/media/video	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
	/srv/media/mp3	Used: 61.44% Total: 20.94 TB Used: 12.87 TB Free: 8.07 TB
PHP	Version	7.0.32-0ubuntu0.16.04.1 (Zend Version 3.0.0)
	Modules	calendar v7.0.32-0ubuntu0.16.04.1 : cgi-fcgi : Core v7.0.32-0ubuntu0.16.04.1 : ctype v7.0.32-0ubuntu0.16.04.1 : curl v7.0.32-0ubuntu0.16.04.1 : date v7.0.32-0ubuntu0.16.04.1 : dom v20031129 : exif v7.0.32-0ubuntu0.16.04.1 : fileinfo v1.0.5 : filter v7.0.32-0ubuntu0.16.04.1 : ftp v7.0.32-0ubuntu0.16.04.1 : gd v7.0.32-0ubuntu0.16.04.1 : gettext v7.0.32-0ubuntu0.16.04.1 : gnupg v1.3.6 : hash v1.0 : iconv v7.0.32-0ubuntu0.16.04.1 : imagick v3.4.0RC6 : imap v7.0.32-0ubuntu0.16.04.1 : intl v1.1.0 : json v1.4.0 : ldap v7.0.32-0ubuntu0.16.04.1 : libxml v7.0.32-0ubuntu0.16.04.1 : mbstring v7.0.32-0ubuntu0.16.04.1 : mcrypt v7.0.32-0ubuntu0.16.04.1 : mysqli v7.0.32-0ubuntu0.16.04.1 : mysqlnd vmysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $ : openssl v7.0.32-0ubuntu0.16.04.1 : pcre v7.0.32-0ubuntu0.16.04.1 : PDO v7.0.32-0ubuntu0.16.04.1 : pdo_mysql v7.0.32-0ubuntu0.16.04.1 : pdo_sqlite v7.0.32-0ubuntu0.16.04.1 : Phar v2.0.2 : posix v7.0.32-0ubuntu0.16.04.1 : pspell v7.0.32-0ubuntu0.16.04.1 : radius v1.4.0b1 : readline v7.0.32-0ubuntu0.16.04.1 : redis v3.1.1 : Reflection v7.0.32-0ubuntu0.16.04.1 : session v7.0.32-0ubuntu0.16.04.1 : shmop v7.0.32-0ubuntu0.16.04.1 : SimpleXML v7.0.32-0ubuntu0.16.04.1 : snmp v0.1 : soap v7.0.32-0ubuntu0.16.04.1 : sockets v7.0.32-0ubuntu0.16.04.1 : SPL v7.0.32-0ubuntu0.16.04.1 : sqlite3 v7.0.32-0ubuntu0.16.04.1 : standard v7.0.32-0ubuntu0.16.04.1 : sysvmsg v7.0.32-0ubuntu0.16.04.1 : sysvsem v7.0.32-0ubuntu0.16.04.1 : sysvshm v7.0.32-0ubuntu0.16.04.1 : tokenizer v7.0.32-0ubuntu0.16.04.1 : wddx v7.0.32-0ubuntu0.16.04.1 : xml v7.0.32-0ubuntu0.16.04.1 : xmlreader v7.0.32-0ubuntu0.16.04.1 : xmlwriter v7.0.32-0ubuntu0.16.04.1 : xsl v7.0.32-0ubuntu0.16.04.1 : Zend OPcache v7.0.32-0ubuntu0.16.04.1 : zip v1.13.5 : zlib v7.0.32-0ubuntu0.16.04.1

Client Information


HTTP	Browser	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
Screen	Resolution	1920 x 1200
Window	Size	1440 x 797

borpin · 2019-02-13T16:58:41.481Z

Have you forced a page refresh of the admin page - clutching at straws. Only other explanation is Safari doing something odd which I cannot test.

sjthespian · 2019-02-13T17:04:02.666Z

Safari doing something odd… isn’t that a given?

I did a full shift-reload of the page before posting the new server info. And on my end I’m using Chrome since my safari beachballs more than it works these days (too many JIRA, Service Now and Splunk tabs open).

borpin · 2019-02-13T17:14:22.359Z

The point of me asking for the info was that the services mqtt_input & service-runner (as well as feedwriter but I doubt you use that) have changed and if you use them you might be interested as you did a manual upgrade.

sjthespian · 2019-02-13T17:18:28.502Z

I’m not using mqtt with emoncms, although I do use it with a couple other automation services (hassio for example). I don’t believe I’m using feedwriter either.

What I really need to do is move this over to a raspberry pi so i can just use the provided image instead of doing this all manually…

sjthespian · 2019-02-13T18:29:41.756Z

BTW, the example you gave is how it looks to me in chrome. The only issue is that the table is incredibly wide due to some unwrappable items in the first column.

borpin · 2019-02-13T18:30:51.526Z

sjthespian:

What I really need to do is move this over to a raspberry pi so i can just use the provided image instead of doing this all manually…

I run mine standalone so it is not usually any more bother. In someways it is less.

borpin · 2019-02-13T19:50:46.514Z

4 posts were split to a new topic: Long Mount points disrupt server information

sjthespian · 2019-02-13T19:24:48.966Z

I just had a thought (always a dangerous thing)… I’m going to look into what it would take to make this a hassio module. That would give me the best of both worlds as it could run on an existing pi and could integrate with the rest of my hassio setup.