# Uploading data to emoncms.org

**URL:** <https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863>\
**Category:** Emoncms\
**Created:** [24 January 2019 05:20 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863 "2019-01-24T05:20:20Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![franck102](https://community.openenergymonitor.org/letter_avatar_proxy/v4/letter/f/35a633/32.png) [@franck102](https://community.openenergymonitor.org/u/franck102)\
**Post date:** [24 January 2019 05:20 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/1 "2019-01-24T05:20:20Z")

</div>

Hi all,

I have searched through the forums but I can’t find a clear answer to this question: as of today what options are available to upload historical feed data to [emoncms.org](http://emoncms.org)?  
I understand that the sync module is not enabled on [emoncms.org](http://emoncms.org) yet?  
The only related option I found was to configure emonhub to post input data to [emoncms.org](http://emoncms.org), but that requires me to manually keep two feed configurations in sync doesn’t it?

Thanks!  
Franck

---

<div class="post-metadata">

**Author:** ![Robert.Wall](https://community.openenergymonitor.org/user_avatar/community.openenergymonitor.org/robert.wall/32/38246_2.png) [@Robert.Wall](https://community.openenergymonitor.org/u/Robert.Wall)\
**Post date:** [25 January 2019 22:05 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/2 "2019-01-25T22:05:49Z")

</div>

This could be one for @TrystanLea

---

<div class="post-metadata">

**Author:** ![TrystanLea](https://community.openenergymonitor.org/user_avatar/community.openenergymonitor.org/trystanlea/32/31_2.png) [@TrystanLea](https://community.openenergymonitor.org/u/TrystanLea)\
**Post date:** [29 January 2019 12:40 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/3 "2019-01-29T12:40:19Z")

</div>

Hello @franck102

I needed to do the same this week in order to upload historic uk solar data used by the emailreports module on [emoncms.org](http://emoncms.org) and so I wrote the following uploader script, which you could repurpose for your requirements.

It uses the emoncms/bulk/input.json api. There are usage examples on the input api helper page here [Emoncms - site api](https://emoncms.org/site/api#input)

```
<?php
// ---------------------------------------------------------------------------------------------
// Emoncms data upload example
//
// 1. Create an input to receive the data upload
// https://emoncms.org/input/post?node=0&csv=100
// 
// 2. Setup a feed connected to the newly created input, set the interval to a suitable interval
// e.g data from sheffield solar below has an interval of 30 minutes.
// 
// 3. Run this script 
// ---------------------------------------------------------------------------------------------
$host = "https://emoncms.org";
$apikey = "WRITE APIKEY";
$nodeid = 0;

// Break upload into blocks of around 500-1000 datapoints
$start = time()-24*3600*365;
$time_block = 24*3600*14; // 672 datapoints at 30min interval
$end = $start + $time_block;

while (true) 
{
    $start = floor($start/3600)*3600;
    $end = floor($end/3600)*3600;
    
    // 1. Fetch data from source
    $startstr = date("Y-m-d",$start)."T".date("H:i:s",$start);
    $endstr = date("Y-m-d",$end)."T".date("H:i:s",$end);
    
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,"https://api0.solar.sheffield.ac.uk/pvlive/v1?start=$startstr&end=$endstr");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $output = curl_exec ($ch);
    curl_close ($ch);
    
    print "start:".$startstr." end:".$endstr." strlen:".strlen($output)."\n";

    $data = array();
    $pv = json_decode($output);
    foreach ($pv->data as $row) {
      $time = strtotime($row[1]);
      $value = $row[2]*1;
      if ($value==null || $value==NAN) $value = 0;
      // 2. build bulk data format [time,nodeid,value,...]
      $data[] = array($time,$nodeid,$value);
    }
    
    // 3. send data to emoncms
    print "Result: ".http_request("POST","$host/input/bulk.json",array("time"=>0,"data"=>json_encode($data),"apikey"=>$apikey))."\n";
    
    // move forward to next block
    $start += $time_block;
    $end += $time_block;
    
    // please take your time uploading data
    // there is a 1s request limiter on emoncms.org
    sleep(1.0);
    
    // exit the script
    if ($end>time()) {
        print "complete\n";
        die;
    }
}

function http_request($method,$url,$data) {

    $options = array();
    $urlencoded = http_build_query($data);
    
    if ($method=="GET") { 
        $url = "$url?$urlencoded";
    } else if ($method=="POST") {
        $options[CURLOPT_POST] = 1;
        $options[CURLOPT_POSTFIELDS] = $data;
    }
    
    $options[CURLOPT_URL] = $url;
    $options[CURLOPT_RETURNTRANSFER] = 1;
    $options[CURLOPT_CONNECTTIMEOUT] = 2;
    $options[CURLOPT_TIMEOUT] = 5;

    $curl = curl_init();
    curl_setopt_array($curl,$options);
    $resp = curl_exec($curl);
    curl_close($curl);
    return $resp;
}
```

---

<div class="post-metadata">

**Author:** ![franck102](https://community.openenergymonitor.org/letter_avatar_proxy/v4/letter/f/35a633/32.png) [@franck102](https://community.openenergymonitor.org/u/franck102)\
**Post date:** [29 January 2019 18:58 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/4 "2019-01-29T18:58:01Z")

</div>

Thanks Trystan, I like the idea of using REST te obtain the source data.

For me though the approach would involve feeding feed data into an input… so a completely different set of feeds on [emoncms.org](http://emoncms.org) (also much simpler I guess, so this may ok).

Do you have any ETA for having sync enabled for feeds on [emoncms.org](http://emoncms.org)?

Thanks!  
Franck

---

<div class="post-metadata">

**Author:** ![TrystanLea](https://community.openenergymonitor.org/user_avatar/community.openenergymonitor.org/trystanlea/32/31_2.png) [@TrystanLea](https://community.openenergymonitor.org/u/TrystanLea)\
**Post date:** [30 January 2019 22:32 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/5 "2019-01-30T22:32:27Z")

</div>

No ETA yet on enabling sync upload on [emoncms.org](http://emoncms.org), its a little more complicate because of the multi-server environment and all the additional testing and load management required. There are a number of other steps to do first, but it is my intention to enable it.

---

<div class="post-metadata">

**Author:** ![franck102](https://community.openenergymonitor.org/letter_avatar_proxy/v4/letter/f/35a633/32.png) [@franck102](https://community.openenergymonitor.org/u/franck102)\
**Post date:** [1 February 2019 16:24 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/6 "2019-02-01T16:24:28Z")

</div>

Thanks Trystan, keep us posted 🙂

Franck

---

<div class="post-metadata">

**Author:** ![drc](https://community.openenergymonitor.org/user_avatar/community.openenergymonitor.org/drc/32/14341_2.png) [@drc](https://community.openenergymonitor.org/u/drc)\
**Post date:** [21 October 2021 08:13 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/7 "2021-10-21T08:13:56Z")

</div>

Hi Trystan and Team, Is it still the case that we cannot use the web gui to sync inputs, feeds and dashboards to [emoncms.org](http://emoncms.org)? I spent some time trying to figure out why it wasnt working but if it’s not enable yet then I will leave it for now because i’d prefer not to delve into php and risk my install for something that I can view over VPN anyway!

---

<div class="post-metadata">

**Author:** ![TrystanLea](https://community.openenergymonitor.org/user_avatar/community.openenergymonitor.org/trystanlea/32/31_2.png) [@TrystanLea](https://community.openenergymonitor.org/u/TrystanLea)\
**Post date:** [25 October 2021 10:12 UTC](https://community.openenergymonitor.org/t/uploading-data-to-emoncms-org/9863/8 "2021-10-25T10:12:04Z")

</div>

Hello @drc yes still not available, I’m working on some fairly substantial changes to the emoncms back-end and will as part of this be looking at enabling this feature on [emoncms.org](http://emoncms.org). Probably at least a couple of months away.