A Howto install freeboard with MQTT for emontx shield

stephen · 2016-09-14T13:21:18.813Z

How to set up freeboard IoT for use with emontx sheild and other purposes

to get emontx shield to connect you will need to be able to send mqtt to do so,
just install a esp8266 to it and and follow these direction.
Simple daughter board to transmit emontx shield data, web, mqtt etc - #7 by stephen

getting started

I did this on a ubuntu mini laptop but it will apply to
raspberry pi / orangepi or other linux compatible devices with some slight
modification

Setting up apache server and node red

sudo apt update
sudo apt upgrade

sudo apt install apache2 ( not really needded but good to have)

Make sure you have the mod_headers Apache module installed. to do this check
out /etc/apache2/mods-enabled/ and see if there’s a ‘headers.load’ in there. If
there isn’t then just

sudo ln -s /etc/apache2/mods-available/headers.load
/etc/apache2/mods-enabled/headers.load
add line to /etc/apache2/apache2.conf

Header set Access-Control-Allow-Origin “*”

this is required only if you want to send json directly to freeboard

Reload apache server.

sudo /etc/init.d/apache2 reload

sudo apt install mysql-server mysql-client

sudo apt-get install php libapache2-mod-php php-mcrypt php-mysql

sudo apt install nodejs
sudo apt install nodejs-legacy
sudo apt install npm
sudo apt install git
npm install -g grunt-cli

make directory for you Iot server ( use sudo when installing with npm )

cd ~
mkdir IoTServer
cd IoTServer
git clone GitHub - node-red/node-red: Low-code programming for event-driven applications
cd node-red
npm install
grunt build

We can now run the nodered server with the command node red.js -v. I use -v at
the end to see if any error crops up and also to see if any node modules are
missing, and need to be installed.
you can install node modules arduino, rpi-gpio, redisout and mongodb if you like .
If you going to use them, we just need to install them with the npm tool.
. If we want to change the bind address or add authentication, we should change the settings.js file.
Take a look at http://nodered.org/docs/configuration.html for more information, and start
nodered with: node red.js -v -s settings.js
to access node red localhost;1880 in your browser
We can now start designing our nodered workflows that receive data from the
esp8266 via MQTT protocol or even by simple REST based HTTP protocol, process
it and store the data payload, if any.

For storing data into a database, we can use MongoDB, or mysql,
For mysql we can do:

cd ~/IoTServer/node-red
npm install bignumber.js require-all readable-stream
npm install mysql
npm install node-red-node-mysql

restart nodered. The mysql node should be available now in
storage pallet.

to start automatically in ubuntu just use startup application and enter it in there to start on boot

Step 2a: Installing Mosca MQTT broker (Alternative 1 for the MQTT Broker)

We can use several brokers, Mosca or mosquitto here how to install Mosca

cd ~
sudo -s
sudo npm install mosca bunyan -g

And to run it just do: mosca -v | bunyan

For running it with websockets enabled, on TCP port 3000, just do:

mosca -v --http-port 3000 --http-bundle --http-static ./ | bunyan

Step 2b: Mosquitto MQTT broker with Websockets support (Alternative 2 for the
MQTT Broker - )
sudo apt install mosquito
sudo apt install mosquito-client

/etc/mosquitto or some other directory and add the following lines, for example
at the end of the file:

listener 1883

listener 9001 127.0.0.1
protocol websockets

Then we run it with:

mosquitto -c /etc/mosquitto/mosquitto.conf

mosquitto will become an automatic service after install mosca you will need to setup as a service or start with start up application

Step 3:
Installing freeboard two options for install ( the first one is the easiest,
but will require you to make
node red flows - but also is the most compatible)

install node red contrib freeboard

npm

node-red-contrib-freeboard

Freeboard Dashboard Node for Node-RED

Just install this plugin in your Node Red installation root directory by using npm:

npm install node-red-contrib-freeboard

if you get an install error create a directory in node modules folder called
freeboard and try again
restart node red
and to access freeboard
by entering in yourURl of your browser :

localhost:1880/freeboard

now you can send mqtt and other functions,easily to freeboard by connecting
mqtt to the freeboard in node red

second option for freeboard

cd ~/IoTServer
git clone GitHub - Freeboard/freeboard: A damn-sexy, open source real-time dashboard builder for IOT and other web mashups. A free open-source alternative to Geckoboard.

For now we’ll just use apache to serve the freeboard dashboards that is
installed in my server. Because freeboard is not installed on the root web
directory we need to make the freeboard available to Apache. The easiest way to
do this, as long as Apache has the FollowSymLinks option enabled for the
document root, is to create a link to the freeboard directory on the web
document root:

sudo -s
cd /var/www/html
ln -s /home/(your account name)/IoTServer/freeboard iot

And now freeboard is available at the url http://myserveraddres/iot.

now install mqtt plugin for freeboard
download freeboard-mqtt from
GitHub - alsm/freeboard-mqtt: MQTT and IBM IoT Foundation plugins for freeboard.io

To use these plugins within your own copy of freeboard copy the files into a
directory where freeboard will be able to see them, such as plugins/thirdparty
in the freeboard main directory

You will also need a copy of the paho mqtt javascript library (
https://raw.githubusercontent.com/eclipse/paho.mqtt.javascript/master/src/
mqttws31.js) and will need to insert the link to it into the following sections
in the plugin files

“external_scripts” : [
“”
],

To load the plugins in your instance of freeboard you will have to add them to
the list used to initialize freeboard, for example below is a section of the
default index.html that I have extended to load the two plugins.

you can install other freeboard plugin if you wish just follow the howto on
the respective pages:
suggested;
freeboard-jqplot
freeboard-flot-extended
freeboard-dynamic-highcharts

to install plugins in the node red version freeboard. just search node modules folder
for the freeboard directory and and follow the same direction as above

good luck have fun

stephen · 2016-09-16T02:29:34.312Z

well moved everything over to the new orange pi pc plus – went realitively smoothly - installed armbian OS for it ( default was android) - took almost every thing as done above although php was slightly different (apt-get install php5 php-pear php5-mysql ) .

everything runs much much MUCH faster and smoother on this $15 quad core board then the $200 little mini laptop

stephen · 2016-09-22T15:03:02.758Z

well the FreeBoard Iot is completely finished and working well. the new mqttw.lua script seam to solved the problem of stalling mqtt data ( though I increased the wd to 20 to account for slow boot routers). it now just hammers away at until data goes through. - installed all my other modules into free board . reconfigured freeboard to have multiple display pages… end product looks good and easy on the eyes. displays nice on a tablets. tvs and computers. used framing ( iframe did not work that well) and multiple freeboard index pages (index1, index2 …etc) configured differently to handle stuff (I e dimming and controlling lights and thermostats in the dwelling page)

freeboard.jpg1183×867 78.3 KB

stephen · 2016-09-23T18:38:28.756Z

well here a nice video of the of my Iot in action, I find it much more functional and more useful then other Iot. as my data collection can be to the second if i want or need . other iot are limited to 5 minute or more which is kind of useless. I have my data logging set at 2 seconds . I use rrd containers so the data base never gets bigger then a specified size in my case about 1 year at 3 second intervals averaging . for the energy usage and production. so i have I nice crisp graph . when i search out specific time intervals… it also gives an idea on how well my diverter working etc … oh well i like it. perhaps someone else wil too - – i like my data private

https://community.openenergymonitor.org:/uploads/default/original/2X/9/959b558120d3f583b141cfde7bb076d3d77131d3.mp4

Bill.Thomson · 2016-09-23T18:51:06.791Z

stephen:

fictional

Perhaps you meant functional?

stephen · 2016-09-23T18:55:37.022Z

yeah- functional – never noticed with the error correct thanks

Paul · 2016-09-23T19:42:30.615Z

That looks really good Stephen, well done!
Can you explain how to are retrieving historical data. Are you querying the MYSQL database, and if so how.

Paul

stephen · 2016-09-23T19:59:51.211Z

well I have a pl script listening to topics from Mqtt and when something comes in it writes it directly to rrd container (rrdtools) . then I have cron setup to generate to historical data every 5 minutes or so depending on when or how big of sample if it one month sample every 60 minute is good enough … i have example of every thing up loaded here for people to use if they like it in the the iot.zip i believe - example 8hrs in the iot.zip is for creating graph file of 8 hrs of historic data but you will find 10min - 1year scripts in the zip

Simple daughter board to transmit emontx shield data, web, mqtt etc - #7 by stephen

and all the graph building scripts i adapted from my earlier ted project which is included in the same zip all you need to do is edit the sh script to point to your rrd container instead of my old ted containers and adjust cost amount or what have to match your needs ( after extracting do not forget to make then executable)

stephen · 2016-12-12T22:21:01.163Z

notice my previous video is now broken for some reason - just re encoded and uploaded it again

stephen · 2017-01-22T20:46:16.079Z

update: it seams or newer versions of node-red or node.js the installer for
npm install node-red-contrib-freeboard
is broken
you just need to install manually

cd into your node-red directory and

git clone https://github.com/Freeboard/freeboard.git
npm install freeboard
git clone https://github.com/urbiworx/node-red-contrib-freeboard.git
npm install node-red-contrib-freeboard

stephen · 2017-02-06T23:08:55.424Z

small update: for freeboard and MQTT and interactive gauges and freeboard… if installing
actuator , interactive gauges … or via handlebars and custom gauges buttons or sliders

GitHub

GitHub - stufisher/plugins: Plugins for Freeboard.io

Plugins for Freeboard.io. Contribute to stufisher/plugins development by creating an account on GitHub.

or

GitHub

GitHub - onlinux/freeboard-plugins: This is a widget plugin for...

This is a widget plugin for http://freeboard.io. Contribute to onlinux/freeboard-plugins development by creating an account on GitHub.

you need to enable cgi-bin
which for debian base these commands should work

sudo apt-get install apache2     --- if not installed already

sudo apt-get install curl

cd /etc/apache2/mods-enabled

sudo ln -s ../mods-available/cgi.load

sudo service apache2 reload

cgi-bin should be eabled

now just extract this file to /usr/lib/cgi-bin MQTT.pl.zip (523 Bytes)

edit it to match your MQTT server setting

now you can easily send mqtt from freeboad or other webapages

usage is simple when you configure it you set up a default topic but you are not limited to a default topic example:
putting this in a the url of your browser

http://192.168.168.150/cgi-bin/MQTT.pl?1234

will out put this and send and publish data to the default topic

Output:

MQTT Server: 192.168.168.150

MQTT TOPIC: /ID/

MQTT DATA: 1234

entering it like this with a "+ "
allows to change the topic

http://192.168.168.150/cgi-bin/MQTT.pl?/temp+543

outputs this

Output:

MQTT Server: 192.168.168.150

MQTT TOPIC: /temp

MQTT DATA: 543

if you are having issues most likely you did not make the the file executable

 sudo chmod +x /usr/lib/cgi-bin/MQTT.pl

you can also use it to publish MQTT from devices that are not MQTT compatible but are able to do http

Update 2:
here a simple button for handlebar plugin for freeboard or you can use in basic HTML . but you will neend to remove the {{ }} as that just inserts live MQTT data into the page - you can prettify it if you like with CSS or JS

 Turn Heating ON or OFF
</form><form action="http://192.168.168.150/cgi-bin/MQTT.pl?off" method="post">
<input type="submit" formtarget="hidden" name="sub" value="Turn OFF!">  Currently
<    /form>
  </form><form action="http://192.168.168.150/cgi-bin/MQTT.pl?on" method="post"> 
<input type="submit" formtarget="hidden" name="sub" value="Turn ON!">  {{  ID}}
</form>
  <iframe style="display:none"  name="hidden">

or a slider

<div style="margin-top: 1em">
<h2>THERMOSTAT</h2>
<input id="Temp" type="range" min="0" max="254" value="" />

</div>
<p id="result"></p>
<script>
var p = document.getElementById("Temp"),
res = document.getElementById("result");
 p.addEventListener("input", function() {
 $.post("http://192.168.168.150/cgi-bin/MQTT.pl?therm+" + p.value);
  }, false); 
 </script>

here simple thermostat

bin23.jpg935×671 70.1 KB