Dumb question - how to input data over HTTP?

For years I’ve been sending data to my emoncms system from a TP-Link HS110 power switch that I control and monitor using a perl program that calls a python program to communicate with the HS110 (the HS110 has some old firmware that permits this). The data has been submitted to emoncms using HTTP like the following:

http://emonpi/emoncms/input/post?node=MVHR_Heater&fulljson={“current”:0.016113,“power”:0,“voltage”:238.708976,“total”:503.334}&apikey=d702…

We went away for a few days over Xmas and when I got back various things were broken :frowning: I’ve fixed most but this data input is still not working and I don’t understand why. I looked at the docs online and see they say to use a URL that doesn’t include the ‘emoncms/’ part. So I tried deleting that and now I see data being posted to the right device with the right input names, but not to the right inputs (see the screenshot):

So I’m confused! I haven’t updated my system recently so I don’t understand why it ‘works’ without emoncms/ but has been working up until now with the emoncms/ ? And how do I most easily get the system back to a state where the data is being sent to the correct inputs so my feeds get updated properly?

@djh - have you updated in the last day or so? Did you do that in an attempt to get it working?

@TrystanLea - could the fix for duplicate inputs have caused this?

@djh - I note you use the http://emonpi/ URI rather than the IP address. My suspicion is that something happened to your Hub that caused the mDNS name to not be found.

What other things were ‘broken’? Could it be an issue with your Router/Hub etc?

[edit] - should be mDNS not DDNS

No I haven’t updated.

I don’t remember exactly why I use emonpi - the example in the docs now uses emonpi.local - but it’s worked like that for years and it still works now sometimes, but with the modified path. The script doing the posting is running on the same pi, so I could have used localhost instead I suppose. What do you mean by my Hub ?

Other completely irrelevant things. And no, there’s no hint of any evidence of a problem with my router.

Yes your router.

emonpi is the mDNS (I mistakenly said DDNS) name allocated on your LAN. This can get lost sometimes.

However, if it is on the same machine it should work.

I think it is just one of those things.

Hello @djh @borpin

The fix I implemented was only applied on Emoncms.org as there’s no easy solution for backwards compatibility without complex migration steps. Post discusses this here: emonHP duplicate inputs from Home Assistant - #6 by TrystanLea

What I would suggest @djh is to change your script to post the data using the fulljson option, E.g:

https://emoncms.org/input/post?node=emontx&fulljson={“power1”:100,“power2”:200,“power3”:300}

Ah you already are using fulljson, hmm, perhaps updating emoncms is worth a try and deleting the duplicate inputs.

But that was caused by the integration using JSON instead of FullJSON. Using the latter fixed it for the HA component. I was meaning the MQTT input duplicates issue.

maybe Im misunderstanding? the question was about the HTTP api? I then saw that @djh is already using FullJSON, MQTT should not come into it.

Another possibility is that there are white spaces being added by the perl script into the full json e.g:

{“current ”:0.016113,“power”:0,“ voltage”:238.708976,“total ”:503.334}

emoncms is not then trimming the result, any inconsistency in white spaces will then generate duplicate inputs..

another possibility is that the redis cache of the input table has lost sync. Clearing the redis cache from the Admin info page might sort it that

1 Like

I think you’ve understood, Trystan.

Perl doesn’t arbitrarily insert spaces and I didn’t edit the source except to delete the ‘emoncms/’ from the URL I cut and pasted here.

The situation has changed now though, so I can’t investigate any further at the moment. I can’t ssh into the pi, so I suspect something corrupted/died on the SD card. :cry:

It’s annoying that pi’s just sort of die without giving any kind of warning.

Yup. I don’t use them for anything that is disk write intensive. I’ve found old Laptops with PVE installed and running emoncms in a container or VM is a good and cheap solution.

Sorry, but what’s PVE? How do you receive the radio signals from emontx-en et al if you use a laptop?

That is annoying!

Our general impression is that Pi SD card corruption issues have become very rare. I cant remember the last time I changed an SD card now. Switching to industrial SD cards around 2021 certainly seemed to help emonSD Industrial - Pre-loaded SD card - Shop | OpenEnergyMonitor .

1 Like

Google is your friend - Proxmox VE

Yes true. I just have an emonbase with just emonhub installed that takes the RF and sends it on or I use a Wi-Fi setup.

Yes, it is a while since the last one died for me.

Well it’s been a while for me too, but the lack of notice is what’s annoying, not the actual failure.

Some microSD cards support S.M.A.R.T.

If you have one that does, the smartmontools package might be worth looking into.

More info:

1 Like

So I’ve installed emoncms on another machine, and I’m hoping to connect it to my existing pi running emonhub, so it gets data. And I’m also hoping to copy my existing feed definitions and data and graphs from the pi or from a backup I regularly make into the new copy of emoncms.

Are these actions documented anywhere or at least the basic steps described, or will I need to dig through lots of code and docs to figure something out?

Ah, that’s good news. Last time I looked it was a desert. Do you know if there’s a list of cards or even manufacturers that provide that support?

Searching Google yields:

https://www.amazon.co.uk/s?k=TeamGroup+EXPERT+S.M.A.R.T.+Micro+SDXC&crid=C3UNO68Z6NAL&sprefix=teamgroup+expert+s.m.a.r.t.+micro+sdxc%2Caps%2C189&ref=nb_sb_noss

35 quid for a 256 GB card.

Update & Upgrade — OpenEnergyMonitor 0.0.1 documentation

However, depending on the size of your database, it can take quite a while to take the backups and then restore which leaves you with a data gap.

I’ve tried to syncing using rsync between the two machines but not had much success.

You might find it easiest to pull the SDCard and copy the files directly (for the Feed data).

You can tell emonhub to send the received data to as many destinations as you like.

Just add in a new/second HTTP or MQTT definition within emonhub and point it to the new machine. How have you installed it on the second machine?