Fast sampling and Emoncms

Hardware emonPi
1

Hi all

We need to collect a data with high sampling rate around 3000 sample per second.
We want to record/save those raw data, no averaging.
And we have emontx+emonESP, emonpi, and IoTaWatt.
We are aware of the transmission limitation of the Emoncms, hence we have our own server.
What is the best way to do so? please.

Many Thanks

2

None of the things you have are designed to do that. While the sample rate to accurately measure power is somewhat less than that, none of the platforms you listed are designed to handle the data at that raw rate.

You need a data acquisition system. Here are some examples.

https://www.omega.com/pptst/OM-LGR-5320_SERIES.html
https://www.picotech.com/products/data-logger

3

In order to see the voltage and current waveforms on the emontx in the past I have written basic firmware to sample the ADC inputs in a block and then print the output to serial, it works well for a few wavelengths:

int samplesize = 300;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial.println("Waveform Sampler");
  delay(1000);

  int voltage[samplesize];
  int current[samplesize];

  for (int i=0; i<samplesize; i++) {
    voltage[i] = analogRead(0);
    current[i] = analogRead(1);
  }

  for (int i=0; i<samplesize; i++) {
    Serial.print(voltage[i]);
    Serial.print(" ");
    Serial.println(current[i]);
  }
}

void loop() {
  // put your main code here, to run repeatedly:

}

A long time ago I wrote a little java tool to extract the samples (and sample a bit faster than the above basic example), it still on the archived forum here Sampler 2.0 | Archived Forum but I haven’t touched that for years, so I cant say Id recommend it…

4

What happens downstream from that? Do you really need to store the data indefinitely for analysis months or years later, or is that just an intermediate step to allow some other process to grind through the data in real time?

Also, do you want to store all of the data all of the time forever, or just a short burst based on some trigger event?

If the storage is really just an intermediate pipe to another process that grinds though the data in real time, then you might get some big gains by moving that process onto the same cpu that’s collecting the data and do the grinding there. The earlier you can reduce the volume of data, the easier the problem becomes.

While it’s only an early stage prototype, something like @TrystanLea’s emonSTM might come close to what you need. It can easily sample at that rate, and has some pretty decent local performance for processing the data (72 MHz Cortex-M4 with FPU and DSP instructions). But it won’t help at all if you truly need to store all that data for future reference. As @Frogmore42 points out, there’s other kit designed to do just that.