Using a RRD (Round Robin Database) for local storage

stephen · 2016-12-12T22:01:04.699Z

[Discussion relocated from the Suitable for Bespoke Applications - e.g. Off Grid Wind Turbine? thread]

You can also save your data to RRD container locally, which is how I do mine. The database is self truncated. All you do is generate a graph at whatever interval you want. Seconds, minutes, hours, days, months or years. I find it much simpler and nicer than mySQL. Here is a sample output from today.

24hrsw.png747×193 14.8 KB

…
if you are a bit confused by my output display
KW = 1000 watts
1 mKW = 1 watts
and
100 uKW = 1 watt
My sampling interval is about once a second. For my graphs, which are generated every couple minutes by a cron job, one year of data is about 60 megs per sensor at that interval.

Bill.Thomson · 2016-12-13T01:12:53.166Z

Hello Stephen,

OEM is looked upon as a source of accurate information. That said, there are some issues with your multipliers.

Multipliers greater than one are upper case.
Multipliers less than one are lower case.
Units named after a person are always upper case. This overrides the rule in line 1.
Example: k for kilo, because K is for the degree Kelvin, as per the rule in line 3.
Never mix multiplier and sub-multiplier.

NIce job with the Round Robin Database graph. Looks good.

Regards,

Bill

stephen · 2016-12-13T01:57:04.684Z

ah – I believe the issue of m and u is common to rrd when related to kw if you/l save as watts then it not an issue… I save my data in kiliowatts so I have the small issue of m and u
here a sample of my script to produce the graph it does not matter if you use KW, kw or kW it always inserts the m & u

 #!/bin/bash

 now=$(date +%s)
now_formatted=$(date +%s | awk '{printf "%s\n", strftime("%c",$1)}' | sed -e 's/:/\\:/g')

 /usr/bin/rrdtool graph /var/www/html/24hrsw.png \
-w 650 -h 120 -a PNG \
--slope-mode \
--start -86200 --end now \
--title "Wind: For the Last 24 Hours" \
--vertical-label "KILOWATTS (KW)" \
--color CANVAS#2a2a2a \
--color BACK#2a2a2a \
--color FONT#FFFFFF \
DEF:kw=wind.rrd:kw:AVERAGE \
VDEF:Last=kw,LAST \
VDEF:First=kw,FIRST \
VDEF:Min=kw,MINIMUM \
VDEF:Peak=kw,MAXIMUM \
VDEF:Average=kw,AVERAGE \
CDEF:kWh=kw,10,/,168,* \
CDEF:Cost=kwh,.0665,* \
LINE1:kw#0e9110:"Wind" \
AREA:kw#ffc015:"" \
HRULE:Average#9595FF:"Avg" \
GPRINT:kw:AVERAGE:"%6.4lf%skw" \
HRULE:Peak#ff0000:"Peak" \
GPRINT:kw:MAX:"%6.3lf%skw" \
HRULE:Min#58FAF4:"Min" \
GPRINT:kw:MIN:"%6.4lf%skw" \
GPRINT:kWh:AVERAGE:"  total %6.2lfkw\\n" \

24hrsw2.png747×193 14.8 KB

Bill.Thomson · 2016-12-13T02:13:42.636Z

Suitable for Bespoke Applications - e.g. Off Grid Wind Turbine?

I believe the issue of m and u is common to rrd when related to kw

Here’s a link that explains the preferred method of storing RRD data as SI units to avoid the issue you’re having.

https://lists.oetiker.ch/pipermail/rrd-users/2012-July/018707.html

The relevant line on that page:
You should store them using GAUGE (as kW is already a rate). In fact, multiply by 1000 and store, as then you’re storing as Watt or Joules/sec, the SI unit, which is Best Practice.

stephen · 2016-12-13T03:09:53.082Z

thank you bill
yes i know - i already stated if I saved as watts ( or any base unit) it not an issue I saved as kw in rrd container . so I have the small issue of m and u - i do not mind the m and u. yes i can adjust my bottom display line correctly . but the vertical line would still have the same issue as you can not really control that display output . I left the script as is for a variety of reasons … if I used watts instead kw then everything works out of the box for me or what ever unit I use be it temperature lux wind speed or what have you… i am lazy I do not feel like adjusting all the small details… especially if it just a minor detail of format–
P.S
if you want the above script to generate a RRD graph proper format then save your units in rrd container as a base unit. ie metre, Celsius, watt, joules, litre etc … then above script will display properly . I just used kW because it shorten the display attributes… ie 1500watts to 1.5 kW . as normally it does not shorten the display attribute until it reaches +10Kw. and for me that a alot messier then an incorrect format of mKW as I rarely reach 10kw in usage

Robert.Wall · 2016-12-13T11:58:50.124Z

stephen:

i am lazy I do not feel like adjusting all the small details… especially if it just a minor detail of format–

It is not just a question of format. If you are posting information in a public forum, is it wrong to ensure that your units and abbreviations are correct, comply with internationally agreed practice and, more to the point, don’t confuse somebody who might be struggling to understand the topic and, possibly even more importantly, teach them the correct practice?

stephen · 2016-12-13T14:00:50.877Z

oh my lord

if one saves to rrd container as watts or any base metric unit ( litre, metre, joules, or gram) use this a to generate your graph
the kw listed ie ( DEF:kw=wind.rrd:kw:AVERAGE ) is just the name of the data container within the larger RRD file , you can put multiple data containers with in single RRD file . if using units other then watts just change the watts to match your unit ie grams GPRINT:kw:AVERAGE:"%6.4lf%swatts" to GPRINT:kw:AVERAGE:"%6.4lf%sgrams"

#!/bin/bash

 now=$(date +%s)
now_formatted=$(date +%s | awk '{printf "%s\n", strftime("%c",$1)}' | sed -e 's/:/\\:/g')

 /usr/bin/rrdtool graph /var/www/html/24hrsw.png \
-w 650 -h 120 -a PNG \
--slope-mode \
--start -86200 --end now \
--title "Wind: For the Last 24 Hours" \
--vertical-label "WATTS (W)" \
--color CANVAS#2a2a2a \
--color BACK#2a2a2a \
--color FONT#FFFFFF \
DEF:kw=wind.rrd:kw:AVERAGE \
VDEF:Last=kw,LAST \
VDEF:First=kw,FIRST \
VDEF:Min=kw,MINIMUM \
VDEF:Peak=kw,MAXIMUM \
VDEF:Average=kw,AVERAGE \
CDEF:kWh=kw,10,/,168,* \
CDEF:Cost=kwh,.0665,* \
LINE1:kw#0e9110:"Wind" \
AREA:kw#ffc015:"" \
HRULE:Average#9595FF:"Avg" \
GPRINT:kw:AVERAGE:"%6.4lf%sw" 
HRULE:Peak#ff0000:"Peak" \
GPRINT:kw:MAX:"%6.3lf%sw" \
HRULE:Min#58FAF4:"Min" \
GPRINT:kw:MIN:"%6.4lf%sw" \
GPRINT:kWh:AVERAGE:"  total %6.2lfw\\n" \

if one saves data as kW in the RRD container ( but it will still display incorrectly on the vertical display as any listed data below 1kW will still be listed as m) as you have no control of that…

 #!/bin/bash

  now=$(date +%s)
 now_formatted=$(date +%s | awk '{printf "%s\n", strftime("%c",$1)}' | sed -e 's/:/\\:/g')

/usr/bin/rrdtool graph /var/www/html/24hrsw.png \
 -w 650 -h 120 -a PNG \
 --slope-mode \
 --start -86200 --end now \
 --title "Wind: For the Last 24 Hours" \
 --vertical-label "KILOWATTS (KW)" \
 --color CANVAS#2a2a2a \
 --color BACK#2a2a2a \
 --color FONT#FFFFFF \
  DEF:kw=wind.rrd:kw:AVERAGE \
 VDEF:Last=kw,LAST \
 VDEF:First=kw,FIRST \
VDEF:Min=kw,MINIMUM \
VDEF:Peak=kw,MAXIMUM \
VDEF:Average=kw,AVERAGE \
 CDEF:kWh=kw,10,/,168,* \
 CDEF:Cost=kwh,.0665,* \
LINE1:kw#0e9110:"Wind" \
 AREA:kw#ffc015:"" \
 HRULE:Average#9595FF:"Avg" \
 GPRINT:kw:AVERAGE:"%6.4lfkW" \
 HRULE:Peak#ff0000:"Peak" \
 GPRINT:kw:MAX:"%6.3lfkw" \
 HRULE:Min#58FAF4:"Min" \
 GPRINT:kw:MIN:"%6.4lfkW" \
GPRINT:kWh:AVERAGE:"  total %6.2lfkW\n" \