Import tarifs from Epex/Spot

Applications
1

As I’ve switched to a dynamic tarif last week, I would like to get these into the emoncms database on a daily base (1-day-ahead), to be able to calculate my real cost. Is there any widget (that I can’t find) which can scrape the data of this website:

https://www.epexspot.com/en/market-data?market_area=NL&trading_date=2022-10-04&delivery_date=2022-10-05&underlying_year=&modality=Auction&sub_modality=DayAhead&technology=&product=60&data_mode=graph&period=&production_period=

Vincent

1 Like
2

Welcome,

I’m not sure what you mean by widget. I think you’d need to write or use something (python script using selenium, node-red scrape-it node, whatever) to extract prices from the html and feed it to the emoncms Input API.

That page has a script element with attribute data-drupal-selector=“drupal-settings-json” that contains a json block containing a “charts” dictionary which contains a “price” dictionary, which contains a “data” array of dictionaries with x values that seem to be (seconds since the epoch) * 10000 and the y values are the prices. I may have expressed that badly, but hopefully it points you in a good direction.

There’s an SFTP server of CSV files which might be easier. It’s described in https://www.epexspot.com/sites/default/files/download_center_files/SFTP_specifications_2020-07.pdf — has your electricity company given you a login? Shouldn’t your electricity company be telling you the prices, rather than you having to get it directly from the exchange?

And finally, I notice that the epexspot website says that its data is not for commercial use, so you probably risk getting blocked if they notice what you’re doing, because you are using it to affect your commercial relationship with your electricity company.

3

My energy company also provides the tarifs 1-day ahead on their site in a graphical manner. I thought to get to the source (which is Epex/Spot) to keep short lines. You’re right in your concern on the commercial relationship, that’s why I thought to read-out the webpage as a regular “visitor”.

The section where you start with the drupal-selector (as not a software-engineer) is complete abacadabra to me. I understand the database pointing but have no idea how to get it out of there.

Actually I was hoping someone had encountered this problem before and wrote a python-script on this already. Are you anyhow familiar with buying energy on the 1-day ahead market?