How can I access raw data from emonpi and what database does the emonpi will use to store the values?
Thank you
See the FAQ entry “Where is my data stored? I cannot see the mySQL tables”
The data engines are explained here: https://github.com/openenergymonitor/documentation/tree/master/BuildingBlocks/TimeSeries
(Note: Phpfiwa is no longer used.)
Hello,
I could see in my arduino that there is an mysql database but it doesnot logs in the energy values. I am not able to find the exact place where the raw data is stored. Please can any one help me to find the access location to see raw data.
Thank you
You do not mean “Arduino”, but you are correct, the energy data is not in MySQL. Did you look at the FAQ? What is the setting in the “settings” file? Are the files where that points to?
hey,
Yeah I have seen those links and seen the file that showed there. But I am unable to figure out what is the database used ?
Thank you
It is a custom database designed here. The structure is explained on Github.
Data can be accessed on the emonPi via:
-
CSV download via Emoncms web interface
-
In real-time via MQTT: MQTT - Guide | OpenEnergyMonitor
-
Via Emoncms HTTP API: Emoncms API - Guide | OpenEnergyMonitor
Hey,
I have found that the data coming from emonpi storing in redis database. So how can I access redis database to fetch those values?
Thank you
This is not the usual way to access Emoncms data, i think you will find MQTT easier. However there is no reason why you cannot connect to the emonPi’s redis server. You will need to open the redis port using UFW
@glyn.hudson I need to know what type of database is emoncms using? I thought that its using redis database.If it is redis how can I access it from the terminal?
Thank you
Redis is used only as a temporary store. The data is saved to disk using a custom PHP time series engine.
https://learn.openenergymonitor.org/electricity-monitoring/timeseries/History
The best way to access the data on an emonPi is via the three methods I mentioned above:
Hey,
@glyn.hudson Thanks for your answer.In my emonpi the power values and the energy values are not similar. What kind of calculations does my emonpi do inorder to get these values in emoncms?
These are the corresponding power values which are taken for timestamp start=1480948358 and end=1480948433
0, 789
1, 789
2, 789
3, 789
4, 789
5, 789
6, 789
7, 798
8, 798
9, 798
10, 798
11, 798
12, 798
13, 798
14, 798
15, 798
16, 798
17, 801
18, 801
19, 801
20, 801
21, 801
22, 801
23, 801
24, 801
25, 801
26, 801
27, 812
28, 812
29, 812
30, 812
31, 812
32, 812
33, 812
34, 812
35, 812
36, 812
37, 791
38, 791
39, 791
40, 791
41, 791
42, 791
43, 791
44, 791
45, 791
46, 791
47, 784
48, 784
49, 784
50, 784
51, 784
52, 784
53, 784
54, 784
55, 784
56, 784
These are the corresponding energy values for the time stamps provided above.
0, 1073368448
1, 1073368448
2, 1073368448
3, 1073368448
4, 1073368448
5, 1073368448
6, 1073368448
7, 1073374848
8, 1073374848
9, 1073374848
10, 1073374848
11, 1073374848
12, 1073374848
13, 1073374848
14, 1073374848
15, 1073374848
16, 1073374848
17, 1073388864
18, 1073388864
19, 1073388864
20, 1073388864
21, 1073388864
22, 1073388864
23, 1073388864
24, 1073388864
25, 1073388864
26, 1073388864
27, 1073402496
28, 1073402496
29, 1073402496
30, 1073402496
31, 1073402496
32, 1073402496
33, 1073402496
34, 1073402496
35, 1073402496
36, 1073402496
37, 1073407808
38, 1073407808
39, 1073407808
40, 1073407808
41, 1073407808
42, 1073407808
43, 1073407808
44, 1073407808
45, 1073407808
46, 1073407808
47, 1073419520
48, 1073419520
49, 1073419520
50, 1073419520
51, 1073419520
52, 1073419520
53, 1073419520
54, 1073419520
55, 1073419520
56, 1073419520
But the total power is not same for both set of data points provided above.Please can you clearify this.
And I want to know where can i find the firmware code of EmonTxV3 in emonpi and how can I get access to it?
Thank you
What sort of input processing have you got setp in Emoncms?
EmonPi firmware can be found in the emonPi github repo:
and emonTx firmware
Hey,
Thanks @glyn.hudson. How can i check what input processing setup is in emoncms? The data that I provided is not similar why? I have already seen the firmwares in the github respositories. I want to know where they can be present to check?
Thank you
You cannot really use a php feed for storing values of that magnitude as they use 32bit single precision floats, so 1073368448 will not increase until the value being stored reaches 1073368481, at which point it will be stored as 1073368512 and won’t increase again until the value reaches 1073368544.
PHPfina (fixed interval no averaging) and PHPtimeseries (variable interval) feed engines are the default options for storing data in emoncms.
Both @Robert.Wall and @glyn.hudson have pointed you to the development notes that explain the structure and the design of these custom storage methods developed specifically for emoncms to provide responsive graphing in a compact and low-write solution for use on a SDcard.
To achieve this, REDIS holds a lot of data in RAM to aid both the responsiveness and “buffered disc writes”. It is not intended to be accessed directly.
The mySQL database is primarily there for the settings and configuration etc. The default configuration does not save data to mySQL. You can add mySQL as a storage option if you desire, but it is not recommended on an SDcard, and is not enabled on a emonPi/emonSD.
This is the list of processes that you have chosen to assign to each of your inputs to manipulate and save your data to “feeds”. What you set up there will determine what and how your data gets retained (or not).
If you can do a screen dump of your inputs page, your feeds page and any “input process lists” that you think are not performing correctly, we maybe able to give you some pointers.
2 Likes
Hey,
Thank you for your answer @pb66 .
So the the values in emoncms will be stored in the phpfeed engines. When we fetching the values from emoncms these will come from the operations done in those scripts right?
I want a clearification in how my Power values are different for the watt-second approach and the watts approach that I have considered.In my watt-second appraoch I will take the start energy and end energy values from the tx_watt-sec feeds and then grab for an interval of time.Then I will calculate the difference between those two and divide the final value with the tx_tsample value at the same start and end energy values. Then I will get one power value with that.
And in watts approach I will take the same time interval and then by usng the url below:
“http://server/emoncms/feed/data.json?interval=1&limitinterval=0&id=33&start=1481022304000&end=1481022469000&apikey=key” I will get the Number of power values within that range and I will calculate power by dividing with number of samples as
Power = (Sumofallpowervalues)/(Number of samples)
Then I will get the one power value but my power value for both above said approaches are showing different power. Is there any reason why they are showing like that?
that depends
or perhaps
Hey,
@pb66 I am attaching my feeds page of my emoncms and I did not understand what “input process lists” means.? can you help me with that.
Thank you
