Hello,
I need to decrease logging interval to 1 reading per second. How can I achieve this, the minimum on the menu is 5 seconds?
Thanks
Are your actually feeding data IN at 1 / Sec or faster?
Bernie Carter
Not sure, if not I would like too? It’s all on the EmonPi
Could you provide me with a little info as to what you have setup to report to the emonpi?
Even a screenshot of the expanded input menu will be helpful.
Sorry for the delay in getting back to you. The aim is to start monitoring equipment and spot patterns in usage of the machine. For example a washing machine or a motor, then use either machine learning or programming logic work out the state of the machine. 10 seconds is not enough, we need at least 1 second and probably more. The picture on this original post has the settings available from the menus for feeding information and the lowest is 5 seconds. I need more. Hope that helps.
Thanks
In order to be of assistance, I would need to know what is feeding data
into EmonCMS. The easiest way would be if you could post a screenshot of
your input menu showing where all your data is being posted from.
Since these units are highly customizable I have no way of telling what
your setup is, thus no way to tell if such a data rate is actually
possible.
Assuming you have an emonPi you would need to edit the firmware sketch so that it reports it’s data each second, once that is compliled and installed you could use a phptimeseries (variable interval) feed instead of a phpfina (fixed interval) feed which will store data with timestamps down to 1sec resolution.
phptimeseries data uses just over twice the disc space of phpfina (9byte v 4byte per datapoint), at 1s resolution that will use disc space at 9 bytes per second per feed.
I still do not know if this will give you the accuracy you need though, due to the timestamps being converted to ints in the emoncms inputs, a swing of 1 sec is only 10% at 10secs, but it is effectively almost +/- 100% when you are using a 1 sec update rate, for example 1519078074.000 to 1519078074.999 would be seen as the same timestamp (0 secs) despite actually being 0.999 secs apart and 1519078074.999 to 1519078076.001 would be seen as 2 full secs despite only being 1.002 seconds apart.
Thanks for getting back to me so quick. I only have one input at the moment which is Services as can be seen below:
We are using EmonPi. Is there any reason I can’t say go to 10 readings a second or more by editing the firmware sketch? Is there anyway to prevent them converting to ints easily?
Would it be easier to amend the emon library to get faster updates
I’ve just found this from looking at the learning section of the website
Another point about the sketch in the emonPi - are you using it as a RFM receiver to handle the data from other nodes (not counting itself, that is)? If you are, you might not be able to go as fast as 10 per second, as the sketch also services the RFM69 receiver, and it needs time to do that. If you’ll never use it to receive other nodes, you will probably be best to remove the RFM parts completely. And if you do try to go to 10 samples per second, you’ll need to cut the measuring period from our standard 200 ms and make it something like 2 or 3 cycles (40 or 60 ms).
Thanks. No we don’t use radio. It’s not very reliable in our environment (lots of steel components in a factory floor). Would i be better using the emon library with network shield than use the existing emon pi. I have all the components and can then send the data using mqtt via the lan.
I doubt that you would see much difference. The emon part uses the serial interface to send the data to the Pi part, and you’d do much the same if you had an Arduino and an Ethernet Shield. But if the Ethernet Shield required any degree of processing power from the Arduino, you might in fact be worse off. I’m not an Arduino expert so I’m only suggesting things you need to check.
Your real problem if you need 10 samples per second is the way emonLib works. I’ve had the upcoming emonLibCM reporting serially at that rate when running on an emonTx, so it should run on the ATMega in the emonPi.
Thanks for the quick response. What im thinking at the moment is to use the emonlib library and basic arduino uno as it strips back to basics and just serial print to screen. Once thats working, then go ahead and starting adding the network shield and mqtt library. That way i know which bit is breaking and which bit isnt. Do you have any comments on what ive suggested? e.g. good idea/bad idea and why 
That’s fine, you can do that and experiment with the timings of both the sampling and of the serial prints as if the payloads are too big the serial port could become the bottle neck if you go too fast.
If you use the standard serial output format of nodeid val1 val2 val3 etc and add a config to the emonhub.conf you can push this data into emonhub via a USB port and it will give you the option to upload to emoncms.
This is where you might introduce more issues, you cannot have accurate data by timestamping it on arrival over a network connection at these sample rates, if you were passing data every 60s, a few seconds either way makes very little difference, so you need to add a RTC and timestamp each packet before sending it over the network.
I do not think you will get reliable sub 1sec interval data if your arduino uno is trying to timestamp and publish each frame as well as sampling.
If you stick to a simple serial connection and use all available resource for sampling you may achieve your goal easier.
That’s beyond my pay grade, I too would like emoncms inputs to use floats for processing timestamps, primarily to make existing posting intervals more accurate, but yes it would allow for shorter sample periods too. I haven’t attempted to work out how this might be implemented as I assume it to be quite involved, hope fully I’m wrong. This is not the first time this has come up.
I’ve been tuned out of this discussion because it didn’t seem like anything IoTaWatt would be appropriate for. Now that the data export handlers are going async, this might be something that could be provided in the upcoming MQTT service for IoTaWatt.
The first issue is quality of data at 1 second intervals. A fully loaded IoTaWatt would only have two samples per channel each second. A partially loaded unit on the otherhand, say 7 channels (8 with voltage) would average about four samples per second (five at 60Hz). So I think the data might be viable.
The second issue is whether the data export is strictly real-time or delivered as in the Emoncms or influxDB server support. If historical integrity isn’t an issue, and I don’t believe it is in these other solutions being discussed, then a real-time async MQTT stream could be produced at 1 second intervals with each data point based on 2-5 samples.
At this point, I’m just coming up to speed on the MQTT services that are in use with Emoncms. Can anyone elaborate on the QoS being used and specifically what circumstances might warrant QoS 1? Right now, I’m thinking that real-time QoS 0 would be pretty easy.
With respect to timing, the packets would probably go out at 1 second intervals +/- a few 10ms (50Hz), provided a connection can be maintained.
As an aside, the time issue for time series storage may be better accomplished with influxDB. It can easily handle ms resolution, and can reduce resolution over time to limit the storage needs over time.
Thanks. Good to know.
Thanks ill stick with a simple serial connection.
Do I need to stamp it on the Arduino? Can’t I just stamp it on either on the way into the pi or if send it to MQTT i can put into influx (a time series database) and that can stamp it? I’m looking for patterns in the electricity usage, a few seconds out wont matter too much providing its consistently out. Hope that makes sense.
Thanks for the post. I’ve not looked at IOTawatt before. Like I said in my previous post, im not sure accuracy (in terms of time) is the most important, just that it is consistent in its posting. For my purpose, im looking for patterns in usage and if its out a few milliseconds that’s fine. I currently send all data from emonpi to influx and display in grafana, i just need more resolution. I also find wireless not the best method in time critical solutions as wireless is more likely to be lost and higher latency ect unless it is time stamped and buffered on the arduino (or esp), and then passed to say influx database with the data and corresponding time.
OK Alex, I can see you have some pretty specific requirements. FYI/ IoTaWatt is currently only 5 second resolution, but it does upload to influxDB with timestamp over WiFi. I suppose a $20 AP right next to it would get you to hardwire pretty reliably - pretty much the opposite of what most people are trying to do, but I can see your reluctance to use WiFi.
Have fun plugging that Arduino stuff together. I’m not willing to do anything custom for your project, but I do think that the IoTaWatt has the potential to do what you want, and if the MQTT feed ends up being a real-time feed, then I believe it will be what you are looking for.
Good luck with your project.
Not if you are just using serial, if you use emonhub it will timestamp on arrival, those timestamps will be to the nearest millisecond, I expect the latency of a constant serial stream to be fairly consistent and pretty short too, but I guess that also depends on how hard you push the MCU too.
It’s only if you use a network connection you should need to timestamp before sending so as to be sure any network delays are not affecting the timestamps. When working at these speeds it won’t take much to distort the picture. I am talking about the frame to frame timing, the actual time/date is of little consequence other than identifying what was being monitored at any given time.
I do not think it would take much to get emonhub to post to a influxdb either. It’s something I keep thinking about but never get the chance to explore. I like influx/grafana and think it is probably the ideal tool for the task at hand.
Thanks for all your comments. I’ve downloaded the emon.lib and just running the code on Arduino Uno to see what throughput I can get (I don’t have the CT clip attached yet) . I am only getting 4 readings a second, ideally I want at least 8 but would be happy with 1 or 2 more readings:
Time in mlsec: 245, Time in sec: 0, 19161.06 83.31
Time in mlsec: 489, Time in sec: 0, 9249.77 40.22
Time in mlsec: 736, Time in sec: 0, 4292.21 18.66
Time in mlsec: 987, Time in sec: 0, 2712.41 11.79
Time in mlsec: 1232, Time in sec: 1, 2394.64 10.41
Time in mlsec: 1481, Time in sec: 1, 2288.04 9.95
Time in mlsec: 1731, Time in sec: 1, 2307.08 10.03
Time in mlsec: 1979, Time in sec: 1, 2561.42 11.14
Time in mlsec: 2228, Time in sec: 2, 2443.51 10.62
Time in mlsec: 2477, Time in sec: 2, 2496.14 10.85
Time in mlsec: 2726, Time in sec: 2, 2494.32 10.84
Time in mlsec: 2976, Time in sec: 2, 2391.45 10.40
Time in mlsec: 3226, Time in sec: 3, 2494.75 10.85
Time in mlsec: 3475, Time in sec: 3, 2420.95 10.53
Time in mlsec: 3725, Time in sec: 3, 2483.36 10.80
Time in mlsec: 3974, Time in sec: 3, 2462.69 10.71
Time in mlsec: 4224, Time in sec: 4, 2483.08 10.80
Time in mlsec: 4472, Time in sec: 4, 2419.62 10.52
Time in mlsec: 4722, Time in sec: 4, 2417.25 10.51
Time in mlsec: 4971, Time in sec: 4, 2505.22 10.89
Here’s the code I am running (I will take the time serial prints out after testing as i know they take processor cycles too):
// EmonLibrary examples openenergymonitor.org, Licence GNU GPL V3
#include "EmonLib.h" // Include Emon Library
EnergyMonitor emon1; // Create an instance
unsigned long time; // to store time
void setup()
{
Serial.begin(115200);
emon1.current(1, 111.1); // Current: input pin, calibration.
}
void loop()
{
double Irms = emon1.calcIrms(1480); // Calculate Irms only#
//double Irms = emon1.calcIrms(1480/2); // Calculate Irms only
time = millis(); // Returns the number of milliseconds since the Arduino board began running the current program
Serial.print("Time in mlsec: ");Serial.print(time);Serial.print(", "); // time in millseconds
Serial.print("Time in sec: ");Serial.print(time/1000);Serial.print(", "); //time in seconds
Serial.print(Irms*230.0); // Apparent power
Serial.print(" ");
Serial.println(Irms); // Irms
}
I’ve removed the delay from emonlib.cpp as it didnt seam to be adding any value my purpose. See below:
void EnergyMonitor::serialprint()
{
Serial.print(realPower);
Serial.print(' ');
Serial.print(apparentPower);
Serial.print(' ');
Serial.print(Vrms);
Serial.print(' ');
Serial.print(Irms);
Serial.print(' ');
Serial.print(powerFactor);
Serial.println(' ');
// delay(100);
}
The only other way i can see to increasing the readings is either :
-
to decrease the sample rate - which is this line -
double Irms = emon1.calcIrms(1480)but that will reduce quality of the readings, right??? can anyone expand on what this means?
p.s. i have halfed IRMS to 740 and i do get twice as many readings. -
or use a quicker processor than the uno, maybe the Duo?
-
or is there another way?
Again, thanks for your time time 
If you’re prepared to wait a little longer, you can have real power (not just current) as I wrote earlier:
No, it does not decrease the sample rate, it is the number of samples in the batch that is averaged and reported. You should choose that number so that you have a whole number of mains cycles. When I last tested the sample rate of emonLib, I got calcIrms( ) reading approx. 5588 current samples per second. So the smallest number you should have is 20 ms worth, or 111.76. You can’t have a fraction. Your nearest is 112 samples, covering just over 1 cycle, so your reading will be wrong. How much wrong depends on where on the cycle you start and finish, and you cannot control that in calcIrms( ). 447 will give you almost exactly 4 cycles, so that’s the smallest number I recommend.