I’ve merged in @Adminde’s development, its really very nice!
and sorry for the delay!
The new tick box, interface looks amazing @Adminde
but I notice that even if I untick ‘create feed use’ it does still create the new feed, although anticking the input process override does work .
Thanks first of all 
I actually thought I tested this more thuroughly and hope this will not result in @pb66 system to completely break again.
Also, I just had some time on my hands and opened another pull request to fix this bug. At least to my understanding and small test right now it should.
Hi Guys,
I have been racking my brains trying to get the device module to work… I have it installed on a standard emoncms setup configured on top of a raspberry pi. Using the GUI everything works, and I can configure a template, initialize it, and it creates the chosen inputs and feeds, however, I have the need to do everything by making API calls.
So, to do this, I ran good 'ol Wireshark to see what calls are being passed when I click the respective buttons. For completeness sake here they are:
Create a new device: http://myserver/emoncms/device/create.json?nodeid=1&name=test&description=&type=emonpi-HEM
List device: http://myserver/emoncms/device/list.json
Prepare device: http://myserver/emoncms/device/template/prepare.json?id=1
Initialize device: http://myserver/emoncms/device/template/init.json?id=1&template={%22feeds%22:[{%22name%22:%22use%22,%22type%22:%22DataType::REALTIME%22,%22engine%22:%22Engine::PHPFINA%22,%22interval%22:%2210%22,%22tag%22:]
All of the above commands work fine when running them directly from a browser and I get the proper results, apart from the last one, which is the one that actually does all the “work”. Every time I run that, I get the following error: |success|false|
|—|—|
|message|“Invalid device template”|
I have tried numerous different other configurations, with the APIkey as well as devicekey, but the one above seems the most correct, as that’s the exact string that is passed when clicking the initialize button in the GUI. The funny thing is that clicking the button in the gui works, but when posting the exact same string in a browser it gives me the “invalid device template” error.
Is there anything else you can possibly add, or something else I an try to initialize the device using the API / Device key, as all I can see in the code seems to be that it wants the device ID and the Template name, but even passing that directly has no effect on the outcome, as it keeps on telling me that the device template is invalid.
Thanks guys, I appreciate any comments or nudges in the right direction
Is the last example url a real example or a shortened example?
How long/complex is your template?
I have just had a look at this and selected one of my test templates at random and watched the request and response in a browser console (chrome) and then copied the full url to the browser bar and it worked just fine.
I then tried it in an incognito browser by adding the main account write apikey and that worked too.
When I tried an Edge browser, I got “{“success”:false,“message”:“Username or password empty”}” even though I knew the apikey was supplied, or so I thought! Closer inspection of the browser bar showed my url had been truncated, I tried several times to get the whole url to stick but it wasn’t happening.
I then moved my “&apikey=abc123” entry to inbetween the “?id=38” and “&template=”, this resulted in a different reply, {“success”:false,“message”:“Invalid device template”}.
A bit of searching and I found out that Edge (and apparently many browsers) have a maximum url length of 2083 chars, when I checked my history, both the url strings in Edge had indeed both been truncated to 2083 chars. When I checked Chrome again I found my original url (plus apikey) is 4134 chars and not truncated in Chrome.
Could it be that your url is being truncated and therefore the incomplete template is indeed “Invalid”.
Hi Paul,
Thanks so much for the info! As a test I have installed chrome quickly and tested the full path of the results I got from wireshark, and this time it worked perfectly, thanks for the insight!
My next challenge is that by default my application spits out anything from 80 to 146 inputs going to emoncms (depending on config), and of those roughly 46 end up being feeds, so if I have to format a template that’s big enough to do that I am afraid I will easily surpass the 2083 character limit, let alone anything bigger than that ;)… But hey, only one way to test! lol!
Thanks again, appreciate the help mate!
Are you testing the device-integration branch @centurionsolar? I hope to get back to the device module soon 
1 Like
Indeed I am :). What I LLLLOOOVVEEE about it at the moment is the fact that I can create ALL the feeds I need in a single API call, making the time to copy accounts only a few seconds as opposed to over 20 minutes if I make the feed api calls one by one. That’s my testing so far at least ;)…
Can’t wait to see what else this little module can do!
I have a slightly off-topic question though: I would like to have the device module api’s available, but I don’t want them showing up in the menu. Is there a way I can “hide” that item in the menu? maybe just a reply with the right file to edit would be lovely, if at all possible?
Thanks so much guys!
1 Like
To just remove it from the navbar whilst retaining full functionality, you could just rename the device/device_menu.php file so it cannot be found eg
pb66@test2:/var/www/test/emoncms/Modules/device$ sudo mv device_menu.php device_menu.php.old
refresh browser and it’s gone.
But when logged in navigating to http://server/emoncms/device/view etc will still work, it’s just not “advertised”.
1 Like
Brilliant, works a charm! Thanks Paul!
Don’t know if this is off-topic or not, but why would it take longer and longer to create feeds as I create more accounts? Surely Mysql can’t get so slow that it’s taking in excess of 50 seconds or more to create a single feed, or is there something happening with the metadata that I don’t know about? Also, do you know if there is anything I can do to speed things up (like a metadata cleanup or something)?
#ScratchingMyHead
That doesn’t sound right.
Does top reveal anything eg a cpu or ram issue? How much ram is redis using?
Any idea’s @TrystanLea?
I am running in the region of 1.2gig memory, although to the best of my knowledge I do not use a redis install. If it’s possible to migrate to a state where I can use redis without losing information it would probably be the better route to take, but at this moment I simply don’t know how blush.
I do see that when creating accounts (specifically feeds), the CPU and memory spikes to 100% for the entire 20 odd minutes it takes to create the feeds, and only then quiets down to normal levels again afterwards.
See what happens if you try to create 80 accounts with an average of 45 feeds each on a pi.
My biggest issue at the moment may be the fact that I do not have root access to my server, as it’s a hosted box, so checking things like top or even restarting apache needs to be done through logging a call with the hoster, but if there’s a way to migrate to a state that I can use redis without losing information, that would be first prize, as I believe that most of the issue is as a result of not using it… But that’s just my opinion… hahaha…
That’s interesting that your seeing slow down issues at 80 accounts and 45 feeds I would have expected it to process creating accounts and feeds of that number in seconds!? Redis is a really good idea to have running, whats your storage medium?
Is it possible to “add” redis to an existing install without breaking it? If yes, is there any article something that can point me in the right direction, and what advantages would I gain over an install that’s not using it?
Regarding the devicce-intergation: I have not found any bugs or things that don’t work as expected, BUT: if you have a massive template (anything more than 20 inputs and 20 feeds), even making a curl or similar call to the api causes the function to fail, because it gets truncated. As Paul rightfully pointed out, there is a limitation to how many characters can be accepted by the browser, and I am guessing that this is true even when purely making calls directly to the API.
The way around it of course is to create two templates for the device in question, whereby one template will configure the first set of inputs and feeds, and the second template will do the rest, as the template adds the new data to an existing account rather than overwrite it by default, so that works well ;).
All taken into account, I say: GO FOR IT! 
1 Like
A key advantage of redis over mysql is that it reduces your disk IO significantly, emoncms will spent much less time waiting on the disk.
Alongside introducing redis, using PHPFina, redis based input queues and switching to SSD’s where the other major performance improvements we made on emoncms.org.
If you can check that redis is running ok, it should be possible to just set $redis_enabled = true; and redis will load anything it needs from the disk. That said its not something I’ve tried on a live server since we first introduced redis quite some time back.
Shall I merge the device-integration branch into the master branch @pb66, @Adminde? I know @glyn.hudson is keen to have the latest device module developments for simplifying the OpenEVSE setup procedure as well as the SolarPV type1 & type2 configurations. Testing it here looks like its working well and thankyou @Adminde for all of your hard work on the latest pull request.
Which repo are you referring to? Both the emoncms and the device module have “device-integration” and “master” branches. Or do you mean both?
Can you please recap on what this entails? This thread started out with some quite significant changes to the way inputs and feeds currently work, I 'm no longer sure what the current status or direction is.
I will try to find time to try the “device-integration” branches later this week, I have not previously tried them as there are still some bugs with the latest (non-integrated) device module. I started to debug at the turn of the year but ran out of time. I’m still using a much earlier version on my live server, so these bugs have not effected my work, but that also means I don’t have recent experience of the bugs is the later version, I will need to run some tests to refresh if you are going to pick up device module dev again.