Importing data to an emoncms system

Tags: #<Tag:0x00007f13ede21050>

(Dave Howorth) #1

I want to import some data to my emoncms system; specifically weather data. But I haven’t found a guide to the best way or ways to do that, so I thought I’d ask. I’m happy to write up a guide to the subject if I can get to feel that I understand it.

I have two sources of weather data. One is a neighbour who runs a weather station and publishes the data on a website. I’ve interfaced to that and scraped the website and posted data into emoncms. So far so good, although I still have unanswered questions about phpfina vs phptimeseries. My other source of data is a local Met Office weather station that publishes data through their DataPoint service. Again I’ve interfaced to that but it’s the question of how to interface it to emoncms that has led to this topic.

My neighbour’s weather station updates every five minutes. I’ve chosen to look at it every fifteen minutes and have sent the data to my emoncms system using the input API. That’s all working well and I have two log feeds set up - one using phpfina and the other using phptimeseries, which is where my existing question comes in.

The DataPoint service provides hourly data for the past twenty-four hours and it’s the question of how to import that which prompted me to start this thread. I realize it’s not vitally important; at worst I’d be losing the first day’s data, but I am curious how to import that historical data?

So should I be using phpfina or phptimeseries? And should I be using the input API or the feed API? How does a feed work without an input? How can I post historical data with the input API?

I’ll be happy to post more detail in response to any questions anybody has. At the moment my brain is exploding just thinking about it.

(Trystan Lea) #2

Hello @djh

I posted an example yesterday here which may be useful:

This uses the input/bulk api, you need to create and link your feed to the input before uploading data and upload starting with the oldest data first, as the first datapoint determines the earliest start time of the feed - you cant add data further back in time.