Emoncms.org feeds not updating for some users today 30th September 3pm onwards

That’s strange, did the emonhub log errors change by any chance after changing the url?

Trystan - I didn’t look - and I’ve already swapped over the emonSD to a new image now. Having issues with my EmonTXs with the new image - I’ve posted a question on this.

1 Like

Thanks @DelBert would you be able to share any info on the process that you took to do that. I assume using: GitHub - acmesh-official/acme.sh: A pure Unix shell script implementing ACME client protocol ?

Thanks for the prompt email Trystan. I’m in the strange position of having two systems running 2000km apart, both writing to emoncms.org. Unfortunately the remote one has been running low-write 9.8.0 2017.02.01 for over four years now with zero problems, but is now failing. It appears to have stopped about 10:00 UTC yesterday 30 Nov. A WeeWx weather station on the same site using the Emoncms add-on is working fine. With no physical access I may have to disable https as a workaround until I next get there!

Sorry forgot to add some lines from log file:

2021-10-01 13:30:32,257 WARNING emoncmsorg emoncmsorg couldn’t send to server, URLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
2021-10-01 13:31:02,210 WARNING emoncmsorg emoncmsorg couldn’t send to server, URLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

We’re using Windows based servers and the Win-Acme client, it was just a case of running that with a different server URL. I’m guessing that won’t help you much as it looks like your site is Ubuntu/Apache based.

There is a list of Acme clients compatible with ZeroSSL here: ACME Automation - ZeroSSL. acme.sh is one of those clients and it looks like they’ve been defaulting to ZeroSSL’s server on versions released since August 2021. For other Acme clients, once you know the setting, just point it at the server: https://acme.zerossl.com/v2/DV90

Thanks @derekb

I feel your pain with sorting these issues out at such a distance!

Thanks @DelBert just tried that one and it’s just timing out unfortunately, they must be busy with everyone trying to switch certs!! Will try again soon, testing on another server first to be sure I have the process right!

Thanks @bwduncan, this worked for me.

I’m not sure which emon image I originally installed but it’s running Debian 8 (Jessie) with openssl 1.0.1t

1 Like

I’d be surprised if their servers were overloaded but I guess it could have happened.

It might be worth forcing a re-issue of the Let’s Encrypt certificate with whatever tool you currently use to maintain the certificate. Since the root CA certificate expired, any new certs published aren’t including this path, which should mean the client will find the valid path instead, similar to @bwduncan solution, but from the server side instead.

You can see the current ‘dead’ path by expanding the ‘Certification Paths’ section here: SSL Server Test: emoncms.org (Powered by Qualys SSL Labs)

Yes, it was an issue on my side.

I’ve now got the zerossl cert issued and installed on emoncms.org!! and it looks like there’s a lot of emonhub buffered data coming in as the emoncms input processing queues have jumped up.
Looks like there will be a bit of a delay on feed updates in next few hours until this clears.

Thanks @DelBert for your suggestions on this, I will add your emoncms.org account for free billing.

Emoncms.org queues have all cleared now, but there is still a lot of historic data being uploaded from the connected emonpi/emonbase units, so if you see yours as still a number of hours behind it may still be working through the upload process.

I had one of the effected systems. It had been running for almost 5 years, so I’ve set it up with a brand new SD card and the latest disk image. Very impressed with how easy it was to start from scratch and import all the new data from the old card.

2 Likes

Happy to hear that @robfox77 ! Glad you have the latest software now which will be more secure going forward too!

For anyone else originally affected by this issue, while we have fixed the immediate data upload issue with the change of SSL certificate, We would still very much recommend upgrading to the latest emonSD image as it brings wider security benefits in terms of an up to date operating system and emoncms installation.

We may also need to change back to the original certificate provider in a couple of months time and so this issue could resurface again at a later date. If you need any assistance with the upgrade process, feel free to email us on [email protected] and either @glyn.hudson , @Gwil or I can try our best to help.

You’re quite welcome, I’m glad to have been able to give something back to such a useful system. Feel free to contact me if you need any further assistance with certificates etc. Thanks for the very kind gesture.

@TrystanLea my emoncms has started working again without intervention. I’d been waiting to order a new SD-card. Do I still need to action anything or is the fix temporary? Thanks Trevor

That’s good to hear, could you take a look in /boot there should be a file starting with emonSD which will tell you which version your running?

Hi Trevor,

The fix is permanent. Well, it will last until the other root expires in 2030…

It wouldn’t do any harm to install the latest release, though.

@bwduncan Thanks very much for the confirmation.