Export CSV 1 second interval

Hi, I would like to ask, whether it would be possible to export a CSV file from my temperature history on emoncms in a1 second interval. The problem is, I can find only a 10 second > interval when exporting, but a lot of data is missing in the csv file, because the temperature was messured for example at .2, .5, .11, .16 etc. seconds, but it exports only .10, .20 seconds data. So a lot of temperature data is then missing, because it leaves out everything but these 10 seconds data.

I hope I’m uderstandable. Thanks a lot for the advice! tom

Welcome.

No it isn’t possible as the max rate that EmonCMS records data is 10s resolution. Why do you need 1s resolution? Are you measuring it at that resolution? If not, it makes no sense to export the data at a resolution greater than your recording rate. There is then no ‘missing’ data points.

Well, I need to export my temperature messurements, they are recorded to emoncms for a year or so. And the exact timestamp is not always the same, but it is every ± 1 minute. For example from today:

2019-12-19 23:39:15, 5.6
2019-12-19 23:40:15, 5.6
2019-12-19 23:41:15, 5.6
2019-12-19 23:42:15, 5.6
2019-12-19 23:43:15, 5.6
2019-12-19 23:44:15, 5.7

or a few days ago:

2019-12-14 12:12:50, 1.7
2019-12-14 12:13:50, 1.7
2019-12-14 12:14:50, 1.8
2019-12-14 12:15:50, 1.8
2019-12-14 12:16:50, 1.8
2019-12-14 12:17:50, 1.9

These data are from the “data viewer” - “CSV output”.

As you can see, there is data for every ± one minute, but sometimes it’s at :15, :50 seconds etc. So when I try to export the data to a CSV file from the FEED window, it only exports data recorded on emoncms at :00 seconds, while these at :15, :50 etc. do not export.

Would there be any solution?

So this is how the Graphs module works. It looks at the start datetime and the end datetime of the graph, divides it up by a maximum number of slots and then returns data points that are interpolated. This is why the ‘timestamp’ does not actually reflect the actual data.

Read this thread Timestamp best use. PHPFINA vs PHPTIMESERIES?

This may help as well Access data files & convert to csv

Another one From PHPFina to CSV?

If you search the forum you might find other topics that help you.

The 10 s posting limit that Brian referred to is imposed at emoncms.org in order to give all users fair access to the server. If you have your own emoncms (as distinct from emoncms.org), you can send and have it accept the data however often you like.

The more often you send the data, the more it will cost in storage space for the database.
Is it necessary that you know the time of the readings with that degree of accuracy? - from what you posted, the temperature is not changing very rapidly, around 0.1 degree in 4 minutes.

I need the export the data from emoncms, so I can use the history in an other new own temperature database.

It would be for me more then enough to export the messured temperature let’s say 12 times per 24 hours, so once every two hours. But I have no clue how to export the data in the csv export, because whathever I set the export to, it still only exports the data from emoncms at 0:00 seconds, so a lot of days I just get empty time colums.

Did you look at those links I posted?

Have you checked the ‘Show Missing Data’ on the Graph? This will show where there is no actual data stored.
image
It could be there is no data to export.

Yes I did all that. I guess I just got it all wrong with my title about a one second interval. Look, When I export today afternoon temperature datas in Graphs - CSV output, I get these datas:

2019-12-20 15:27:45, 6.7
2019-12-20 15:28:45, 6.7
2019-12-20 15:29:45, 6.8
2019-12-20 15:30:45, 6.7
2019-12-20 15:31:45, 6.7
2019-12-20 15:32:45, 6.7
2019-12-20 15:33:45, 6.7
2019-12-20 15:34:45, 6.7
2019-12-20 15:35:45, 6.7

When I export in Feeds - CSV Export, for the same time window, I get only the date and time, but no °C.

So yes, emoncms has every minute a °C saved, why else I would be able to see all these in the graph, but when I try to export the history, so I can import them into an other database, I got only some times some data, when it is at :00 seconds.

Is there really no way to get the export in Feeds get all the datas saved to emoncms?

Because it creates the data points so it can create the lines.

Did you do this?

Did you read the threads I posted?

192187.csv (1.9 MB)

Yes I checked all this. My last try, as we probably don’t understand eachother. I send the attachment of a list of temperatures measured, There are a lot of them missing, while they are at a specific second and not at the whole minute saved on emoncms. For example there is a measurment at 10:32:10, while the export file only shows 10:32:00, which is empty. In the graph though I can see the temperature at 10:32:10, but in the export not. That’s why I’m asking whether I can export all the data, and not only the ones at :00

Never mind, I just copied the values manually from the dataview csv output, so I have it complete. Was a bit of work, but I just couldn’t find out how to have it exported all. Thanks for the replies anyway and merry christmas!