Emoncms Graph module developments

TrystanLea · 2016-08-29T11:02:17.216Z

Shall I remove the feed id in the table? Perhaps with tag:name its not needed?

pb66 · 2016-08-29T11:16:41.161Z

IMO you should keep it on the work area, I would certainly like to be able to find the feed id, if you remove it could you provide a tooltip/hoverover? There again should it remain visible for assisting users that provide screendumps? I think keep it!

ID access is something I have been meaning to bring up in a general sense too. The old processlist edit page used to include the input ID at the head of the page, now it seems there is no way to obtain an input id without using a input/list api call or adding a “+input” to a processlist and getting the id from hovering over the shorthand processlist buttons on the input page.

And in contrast the feeds page prominently shows all feedids in the 1st column and also sorts by feedid too, so lists become very confused once you start retrospectively adding feeds at a later date. the default sort order should be based on feed name (within the tag groups) and perhaps the column display order should be Name first.

I would like to see less focus on feedids on the feeds page without losing them altogether like the inputs, and bring back the inputid in a discreet but logical location.

EDIT - Sorry slid a bit off topic there, maybe I should start a new thread about the general use of ids and their visibility (or lack of)

pb66 · 2016-08-29T11:23:21.377Z

Also !!!

Would you consider putting the graph selector & save functions at the top of the feed list pane? the graph selector & save box is a fixed size that will be needed on most visits to the page and some users will only have short feed lists, some of us though have quite long feed lists, look at the size of the scroll bar handle for the left pane in this screendump.

2016-08-29 (2).png1901×1164 63.6 KB

pb66 · 2016-08-29T13:58:49.706Z

Do you see the same issue when creating a multigraph?

This type of issue has been mentioned before with colour picking in some browsers and I do not recall the outcome, If it is not unique to the graphs module/page we should move this discussion another thread to keep on topic.

pb66 · 2016-08-29T14:07:34.060Z

@TrystanLea - Sorry but I found another bug, when using the right hand axis, the legend is not right. the screenshot below is the same 3 feeds but the right boxes are selected rather than the left.

2016-08-29 (5).png1912×355 55.8 KB

bidouilleur · 2016-08-29T20:01:42.462Z

2 posts were split to a new topic: Bugs in new graph module

TrystanLea · 2016-08-31T21:41:33.542Z

Looking into these bugs, thanks for testing, will hopefully update with fixes tomorrow, working through the public dashboard issue at the moment.

TrystanLea · 2016-09-01T14:21:44.573Z

@pb66 in relation to the bug in your last post, does the legend work ok if you reselect those feeds now, after a cache refresh?

TrystanLea · 2016-09-01T14:33:42.884Z

I’ve added the feedid:tag:name to the feed list. Good point about the input ids. The easiest solution would be to add another column perhaps with the id’s in a light gray colour? Or for the configure (wrench) icon to have a hover caption of the form: “Configure inputid:4502” perhaps…

pb66 · 2016-09-01T14:39:37.054Z

No change I’m afraid!

(top tip! do not clear your browser cache whilst logged into the forum, it doesn’t cope well. I couldn’t post, and even though it looked like I was logged in it kept giving me messages like “you need to be logged in to do that” BUT I couldn’t log out either, the “logout” at the foot of the user dropdown (accessed by clicking profile pic) did nothing, it wasn’t until I got a couple of 404 pages that it finally logged me out)

DOH! hang on a sec, I did as I was told and cleared the cache but it would help if I pulled in the changes first, back in few mins !!!

pb66 · 2016-09-01T14:53:45.173Z

Right! lets try again!

There is a change but it’s still not right.

If I freshly select a new feed with a RH checkbox it’s no different, still “undefined undefined”.

However. If I move a feed from the left to the right but just clicking the RH box of a feed that already has it’s LH box ticked, the tag and name persist. (previously it would change to “undefined undefined”)

Do the same test again but deselect the LH box before selecting the RH box and it’s “undefined undefined”.

So the code for the RH box cannot determine new values but it can retain existing data, So for now a work around is to momentarily selct the LH box before the RH box for RH scaled feeds.

Sorry about the false start, multitasking never pays off !!

EDIT - Also to add to that if you select RH first then LH the “undefined undefined” remains, it does not correct unless you deselect and reselect the LH box.

TrystanLea · 2016-09-01T15:13:03.737Z

Yes at the moment the way it works is that it copies over the feed id, name and tag to the graph object when you create the graph which means that if you change the feed name the name is not updated. This saves a feed list api call for every graph load (which could be a large number of api calls on a large dashboard) but perhaps its an optimisation that will lead to more confusion than its worth…

pb66 · 2016-09-01T15:21:49.544Z

TrystanLea:

This saves a feed list api call for every graph load (which could be a large number of api calls on a large dashboard) but perhaps its an optimisation that will lead to more confusion than its worth…

How do the existing dash elements/widgets work? do they fetch or retain the feed names? IMO if I edit the feed name in the Feeds page I would expect it to change everywhere. Obviously graph titles and dashboard names based on feed names wouldn’t change but I think the legend should, especially with the table below parsing the feedid:tag:feedname would that then actually be giving false info?

Also, I’ve just noticed the Min and Max applied to the LH scale are being applied to the RH scale.

This pic shows my usual cpu_temp example with solar_power added to the RH scale (hence the “undefined undefined”) the PV array is currently producing well over 10kW, so the solar_power is 10,000+ but goes of the scale at 70.

2016-09-01 (1).png1752×919 116 KB

TrystanLea · 2016-09-01T15:35:03.126Z

The y-axis min and max applies to both axes at the moment. Il look into what I can do on the legend feed names.
The public accessibility in dashboards is also a little more complicated than I thought, as at present the saved graph object contains all saved graphs for a given user and is requested using the userid which is not stored in the parameters for the graph embedded in the dashboard… which means each saved graph will likely need its own id, in much the same way as the multigraph id to be accessible.

TrystanLea · 2016-09-09T17:25:04.349Z

I’ve now re-implemented the way the graph module saves its graphs to enable use in public dashboards. I’ve updated emoncms.org with the changes including running a migration script to copy over graphs already created on there.

Dashboards however still need to be rebuilt to load the graphs as the parameters saved in the dashboard have changed from graph name to graph id + a full page refresh will likely be needed to reload the javascript for the dashboard.

For anyone who wishes to run the migration script on self-hosted emoncms installs, here is the code, it can be run from the command line $ php graphmigration.php (assuming saved in file called graphmigration.php)

<?php
$server   = "localhost";
$database = "emoncms";
$username = "root";
$password = "pass";
$port     = "3306";

$mysqli = @new mysqli($server,$username,$password,$database,$port);

$allgraphs = array();

$result = $mysqli->query("SELECT * FROM graph");
while ($row = $result->fetch_object()) $allgraphs[] = $row;

$result = $mysqli->query("DROP TABLE graph");
$result = $mysqli->query("CREATE TABLE graph (`id` int(11) not null auto_increment primary key, `userid` int(11), `data` text) ENGINE=MYISAM");

foreach ($allgraphs as $user) { 
    $userid = $user->userid;
    foreach (json_decode($user->savedgraphs) as $graph) {
        print $userid." ".$graph->name." ";
        
        $stmt = $mysqli->prepare("INSERT INTO graph ( userid, data ) VALUES (?,?)");
        $stmt->bind_param("is", $userid, json_encode($graph));
        $stmt->execute();
        $id = $mysqli->insert_id;
        
        print $id."\n";
    }
}

pb66 · 2016-09-09T21:38:42.263Z

TrystanLea:

I’ve now re-implemented the way the graph module saves its graphs to enable use in public dashboards.

That’s good news

Does that mean that viewing a dashboard without logging in, using the read-only apikey will also work too?

bidouilleur · 2016-09-10T07:06:26.961Z

installed the new graph modle and made a new graph, only had one in test, not going to fiddle with the upgrade script.
One little ‘bug’ I see already. When making a graph with ‘feed tag’ unchecked and saved, on the page where you insert the graph, the tag is showed. Sorry for the large image but you see the two screens side by side (realtime screenshot)

Screen Shot 2016-09-10 at 09.02.44.png1259×310 56.4 KB

pb66 · 2016-09-12T17:03:58.630Z

Yes, I’m seeing this too. The embedded graph’s are not respecting the tag show/not show, only when you reopen the graph on the graphs page is the tag setting applied.

Also I’ve noticed the “fill” seems to impose a “zero” minimum Y-axis setting if left to “auto”, in this first pic the Y-axis min is auto and in the second, all I have done is selected “fill” for cpu_min, nothing else. This behavior is overridden by setting a specific Y-axis minimum, as seen in the third pic.

2016-09-12.png1726×1146 149 KB

2016-09-12 (1).png1698×1159 114 KB

2016-09-12 (3).png1702×1155 134 KB

joergbecker32 · 2016-09-20T16:47:55.581Z

I have just found this fantastic new tool for definig graphs! Very nice!

But I am having the same problem that the feed tag is always shown when the graph is used somewhere.

Another thing that is a bit overkill IMO is showing a 14 digit value in the tooltip (but no unit ). On the other hand, it would be nice if seconds could be shown (not only hours:minutes). And the year is truncated to 20 (instead of 2016) for me, not sure if you also see that.

Best regards, Jörg.

pb66 · 2016-09-21T08:57:08.971Z

I have also noted that when accessing the graphs page from the feeds page by clicking the view icon next to a feed, the feed is displayed but the tick box in the lefthand panel is not ticked.

This makes locating the currently displayed feed quite tricky, because of the use of tags and shorter, duplicated feed names, it is really difficult to determine which “Grid power” you are using when there are 15 of them on one site, especially as the list on the left appears quite randomly (un)sorted (I suspect it’s in feedid order but haven’t checked yet).

To add to the confusion, when accessing from the feeds page as described, the tag is always “undefined”, hence the need to hunt out the displayed feed and deselect it and reselect it to get the tag populated correctly.

This unfortunately means the power and features of the graph module are somewhat undermined as I have to choose between trying to correctly select multiple feeds from an unsorted list or access only one feed at a time from the feeds page.

@TrystanLea - I would love to help out by debugging this for you, but as php is not a strong point for me and the code comments are few and far between, I have little or no hope of understanding the code. If you could add more comments we/I could probably figure some of this out and submit fixes for your review. I really would prefer to spend time helping out with debugging than renaming all my feeds, again.