HTTP post to emonCMS - gets 404 randomly? Possible data rate issue?

Emoncms
1

I’ve started using a new energy monitor, which is my own code running on an ESP32. It sends data every 10 seconds to my local emoncms instance running on an RPi 4. It ran successfully for a couple of days, and then decided to stop working, with an HTTP error 404 - page not found! Restarting the ESP32 doesn’t always fix it immediately, but the second restart usually does.

Obviously there’s a possibility that something is going wrong on the ESP32, but it has occasionally sprung back to life for a bit on its own. Given that I’m also sending MQTT messages from my HP monitoring setup, which is maybe 20 data points every 10 seconds, I was wondering if it was worth switching to the queue-based input controller? Am I likely to be exceeding a reasonable input rate?

2

It would probably help if you look at the logs on both the sending and receiving machines, and post them here if you can’t figure it out. The sending log will depend on what you built-in or can add. The receiving log will probably be the apache access log and/or the error log.

I had a problem where producing the logging information in my sending machine was itself causing the service the stop working sometimes. Turning off the debugging actually solved my problem. :frowning: :slight_smile:

You’re not likely to be hitting any input rate limits on the basis of what you posted.

3

Thanks. I should have mentioned that I checked the logs continuously while developing the monitor, and there are no signs of anything relevant in the logs, either the base Apache ones or the emoncms specific ones. 404 wouldn’t normally appear in the Apache error log anyway, being a content “error” rather than a server error, and there’s nothing in the access log.

The ESP32 logging appears to show a completely consistently correct URL, but EmonCMS regards it differently:
ESP log entry for 404
[HTTP] URL: http://emonpi.local/input/post?node=ASHP&fulljson={%22voltage%22:251.000,%22current_b%22:4.054,%22power_b%22:941.700,%22energy_forward_b%22:3477.584,%22frequency%22:50.000,%22power_factor_b%22:0.930}&apikey=0123456789ABCDEF0123456789ABCDEF
[HTTP] GET return code 404
ESP GET return data
[HTTP] Result:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /input/post was not found on this server.</p>
<hr>
<address>Apache/2.4.10 (Raspbian) Server at emonpi.local Port 80</address>
</body></html>

ESP OK log
[HTTP] URL: http://emonpi.local/input/post?node=ASHP&fulljson={%22voltage%22:251.100,%22current_b%22:4.051,%22power_b%22:941.600,%22energy_forward_b%22:3477.587,%22frequency%22:50.000,%22power_factor_b%22:0.930}&apikey=0123456789ABCDEF0123456789ABCDEF
ESP GET returned data
[HTTP] Result: {“success”: true}

Looking at the emoncms index.php, there is a section which returns 406 Not Acceptable, which I can get to return accessing “http://emonpi.local/xyz.notthere” using a browser. That returns an entire EmonCMS page, complete with menu, and is logged in /var/log/emoncms/emoncms.log which is not the case for the HTML returned in my log entry above. The 404 returns also don’t show in the apache access log, which is weird. The returned HTML indicates that it’s coming from the Apache server, rather than EmonCMS, and UTL “/input/post” is not a valid address, which is clearly not correct. So somehow it’s getting lost between Apache and EmonCMS, which is bonkers. All I can think is that there is some kind of data loss or modification I can’t see in the log entry or maybe due to wifi (but it’s TCP!!) which doesn’t show but breaks Apache’s routing.

I have also turned on access logging for the emoncms virtual host, and while successful HTTP GETs show up, the 404 efforts do not appear there. This supports the possibility that the HTTP request isn’t getting as far as EmonCms, but is processed and rejected earlier by Apache, albeit silently

Ideas welcome!

5

You should see a 404 in the access log. Maybe you need to turn up the logging level? I think you can also put LogLevel core:info in the error log configuration to have it logged there too. See

6

Yeah I was going to try that. I also realised I wasn’t completely URL escaping the query string, so i did that, worked fine for 3 minutes!

[HTTP] URL: http://emonpi.local/input/post?node=ASHP&fulljson=%7b%22voltage%22%3a252%2e400%2c%22current%5fb%22%3a4%2e071%2c%22power%5fb%22%3a961%2e400%2c%22energy%5fforward%5fb%22%3a3500%2e328%2c%22frequency%22%3a49%2e900%2c%22power%5ffactor%5fb%22%3a0%2e940%7d&apikey=0123456789ABCDEF0123456789ABCDEF
[HTTP] Result: {“success”: true}

[HTTP] URL: http://emonpi.local/input/post?node=ASHP&fulljson=%7b%22voltage%22%3a252%2e500%2c%22current%5fb%22%3a4%2e068%2c%22power%5fb%22%3a960%2e800%2c%22energy%5fforward%5fb%22%3a3500%2e331%2c%22frequency%22%3a49%2e900%2c%22power%5ffactor%5fb%22%3a0%2e940%7d&apikey=0123456789ABCDEF0123456789ABCDEF
[HTTP] Result:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /input/post was not found on this server.</p>
<hr>
<address>Apache/2.4.10 (Raspbian) Server at emonpi.local Port 80</address>
</body></html>

[HTTP] GET return code 404

Darn it. Thought I had it there!

I have also tried using Python, both on my laptop and on an ESP32… they seem to work ok, certainly for longer than this C++ version does, whcih is what made me think of trying the full escaping, which is automatically applied in CPython and simple in micropython. I couldn’t get micropython to do the MODBUS thing properly, or I’d be using that anyway, it’s much less complicated!

7

You’ve shown a lot of logs from your ESP code, but nothing from Apache? They have the advantage of being fairly standard and familiar.

8

That’s because there’s nothing from Apache…

9

Have you adjusted the logging level etc?

10

I bumped up the log level on the EmonCMS specific logs. There was no mention of the 404 in either of those.

In order to get a new angle on this, I wrote a Python script to do the same thing, which is now happily running on the ESP32, and hasn’t failed yet, as far as I can see. If that keeps going fine then there’s something going on in the C++ somewhere which Apache doesn’t like.

I can’t be bothered to spend any more time on it as I have something running which is easier to maintain! I only used C++ because I couldn’t get the ModBus to work with Python originally. Yeah, I could spend hours taking one for posterity, but nah…

Thanks for your input, most helpful.

11

Glad you found a solution. :grinning_face: My problem where debugging caused a problem was with a Python script :cowboy_hat_face: so I hope you have better luck!

It’s worth taking the time when you have a spare moment to understand how to change Apache’s logging, since it can be very useful.

13

I think I’ve found the problem, which no amount of Apache logging on the emonpi server would have found… the PlatfomIO ESP32 library WiFiClient class doesn’t do mDNS properly, so the code would randomly connect to another non-EmonCMS server, and obviously get 404! I realised this when I tried to use MQTT instead, and couldn’t get that to work. Sigh.

1 Like
14

Platfromio moved files about on my machine when I tried it - malware at best. This is another reason for me not to use it. :smiley:

15

Well at leat Apache logging would have shown that the 404 wasn’t coming from that server, and so told you that you needed to look elsewhere. But well done on tracking down an obscure problem. :grin:

1 Like