I’m integrating some local weather data into my emoncms system by uploading it using the input API. The website I’m scraping it from updates every five minutes on the hour; so I capture every fifteen minutes at one minute past the hour. I’ve created a phpfina log feed called weather:temperature and a phptimeseries log feed called just temperature. The node is called weather. I’ve noticed some odd effects when I graph the data:
There’s a bunch of nulls because I was still updating my screenscraper code and also because the website doesn’t update overnight - I suspect the weather station PC get turned off.
But what I don’t understand is why the data shown in the CSV list doesn’t match that shown in the graph? The graph shows both series one on top of the other, and the data in the feed files confirms that. But the CSV shows some large negative values for the phpfina series at a time when there ar nulls in the data. The phpfina data is:
2019-01-19 18:15:00 0.200000
2019-01-19 18:30:00 NaN NaN
2019-01-19 18:45:00 NaN NaN
2019-01-19 19:00:00 NaN NaN
2019-01-19 19:15:00 NaN NaN
2019-01-19 19:30:00 NaN NaN
2019-01-19 19:45:00 NaN NaN
2019-01-19 20:00:00 NaN NaN
2019-01-19 20:15:00 NaN NaN
2019-01-19 20:30:00 NaN NaN
2019-01-19 20:45:00 NaN NaN
2019-01-19 21:00:00 NaN NaN
2019-01-19 21:15:00 NaN NaN
2019-01-19 21:30:00 NaN NaN
2019-01-19 21:45:00 NaN NaN
2019-01-19 22:00:00 -1.900000
2019-01-19 22:15:00 -1.700000
The phpfina series just disappears from the CSV after 2019-01-19 19:46:00. In reality it is null but resumes at 22:00 as seen above.
Another mystery is why the phptimeseries disappears entirely from the graph when I check the ‘missing data’ box.
PS sorry about the blue bar through the middle of the picture. An unexpected side-effect of FF screenshot and the site’s CSS.