Readings gone haywire

Orionn · 2020-05-04T11:00:39.197Z

I’ve been using my EmonPi for about 2.5 years with success. However, from time to time (every few weeks or so) it would become unreachable. I was writing a cron script to test the network, and if it died either restart the interface/networking/emonpi (I was still testing) when I realised my powerline adapter was dropping packets. So I physically removed the powerline adapter and started using the wifi. When I did, my power readings started going haywire.

I’m assuming I’ve corrupted data somewhere, but I can’t figure out where. I thought maybe one of my CT readings are coming in inverted, so I tried changing it in emonhub.conf but that wasn’t right either (I’ve changed that back).

Here are what my consumption looks like on a normal day (before the problem), what it looks like today, and the solar reading from my inverter for the same period. Notice the solar in the haywire result is a completely different shape at the end of the day.

What a normal day looks like:

normal1374×607 46.8 KB

What today looked like:

Haywire11421×794 73.1 KB

The graph from my inverter from today. Notice that the shape of the graph is different at the end of the day.

Solar697×386 15.8 KB

I tried loading a backup from a few days before the failure and it was still reading wrong.

Do I have a hardware fault? Where should I be looking?

My log file suggests some files are missing, but all my services are running.
emoncms.log.txt (7.8 KB)

borpin · 2020-05-04T11:06:47.971Z

There is a strong possibility the logs are filling up or the SD card is beginning to go.

The best course of action is to get a new SD card, install a fresh version of the emonSD image, and import your data via USB.

Cheers

Orionn · 2020-05-04T11:32:37.139Z

Thanks for the quick reply. The SD card isn’t full, it still has ~30% free space - do these cards typically only last 2.5 years?

My emonpi is bolted to an asbestos sheet in my meter box, so I’ve been avoiding opening it up. Also, it seems there is no info anywhere on how to physically swap the SD card. I assume it’s fairly easy to pop open the emonpi though, I’ll give it a go soon.

Although I’m still not convinced that’s going to fix it - I feel like a calculation has inverted somewhere or something.

borpin · 2020-05-04T11:36:27.991Z

Orionn:

The SD card isn’t full

The log file is probably on a specific tmpfs partition that is limited in size. This has been a common problem.

df -h will show how full the /var/log partition is.

Orionn:

Also, it seems there is no info anywhere on how to physically swap the SD card.

Open up the end away from the USB ports.

Orionn · 2020-05-04T11:52:58.051Z

pi@emonpi(ro):~$ df -h
Filesystem             Size  Used Avail Use% Mounted on

/dev/root              3.4G  2.2G  1.1G  69% /
devtmpfs               481M     0  481M   0% /dev
tmpfs                  486M     0  486M   0% /dev/shm
tmpfs                  486M  6.6M  479M   2% /run
tmpfs                  5.0M  4.0K  5.0M   1% /run/lock
tmpfs                  486M     0  486M   0% /sys/fs/cgroup
tmpfs                   40M  6.6M   34M  17% /var/lib/openhab
tmpfs                  1.0M  4.0K 1020K   1% /var/lib/dhcpcd5
tmpfs                  1.0M  4.0K 1020K   1% /var/lib/dhcp
tmpfs                   50M  1.3M   49M   3% /var/log
tmpfs                  1.0M     0  1.0M   0% /var/tmp
tmpfs                   30M   36K   30M   1% /tmp
/dev/mmcblk0p1          60M   22M   39M  37% /boot
/dev/mmcblk0p3         3.7G  1.1G  2.4G  31% /home/pi/data

borpin · 2020-05-04T13:23:50.171Z

Orionn:

tmpfs 50M 1.3M 49M 3% /var/log

Trick is to keep an eye on this and see if it increases - it naturally gets reset on reboot.

Orionn · 2020-05-05T12:33:28.066Z

I’ve loaded the microSD but haven’t had a chance to swap it over yet.

I thought I’d figured it out. I’m using the 9VAC transformer (from the OEM shop) through a cheap international adapter (I’m in Australia). When you tilt the transformer the connection was broken and I was reading 0VAC at the DC plug. I moved things around and made the connection more secure and have confirmed the plug has ~11V, but I’m still getting an unwavereing 230V on vrms emonpi input. Do you still think it’s a faulty microSD, or has something else burnt out or something?
Here’s my VRMS graph for the last month:

vrms1540×773 94.1 KB

borpin · 2020-05-05T12:54:28.223Z

@TrystanLea @Robert.Wall - any ideas?

Robert.Wall · 2020-05-05T13:02:39.988Z

There wasn’t an a.c. supply when the “emon” front end powered up, so it’s assuming a nominal 230 V, and showing you apparent power rather than real power.

Power down the emonPi, and power up making certain that you have a good 11 V a.c. while you do so, and it should come back, with “AC Wave Detected” on the display at start-up. Restarting the Pi is not enough.

It will not go back to assuming 230 V while the “emon” part continues to run, so after starting, no a.c. voltage = power readings of zero.

Having written that - I think I spotted a bug. I powered up my emonPi to check, and it updated itself - and although the display reported “AC Wave Detected”, it appeared that it was using a fixed 230 V that didn’t go to zero when I unplugged it. Powering up again, this time without it updating, it behaved as expected, i.e. the voltage reports a very small value when a.c. is absent.

Orionn · 2020-05-05T23:40:54.545Z

Fixed! A hard reboot (power cycle) fixed it. It was the AC (well, the international plug adapter) not working.

I’m surprised it yielded such crazy results. Here’s another screenshot of overnight/this morning, then I fixed it about 30 minutes ago.

fixed888×404 17.3 KB

Notice that the power usage jumps inexplicably about 6:40am. I can confirm there is no actual power draw like that, I think it’s the solar panels kicking in but getting multiplied massively. I’ve checked to see if I have a threshold set (if <x, assume 0) which might explain that, but I don’t.
Then when I fixed it around 9:15am, you can see how the usage has dropped dramatically. I would imagine a difference of apparent vs real power wouldn’t be so marked. We’re just a regular house, and haven’t been using our AC, so we’ve only been using fridge, lights, PC/TV, etc.
In the last month the voltage has been somewhere between 230V and 247V, and currently sitting around 240V, so 230V would be a close enough guess.

Thanks to Robert and Brian. If you think it’s a bug that needs to be fixed, I’m happy to help if I can.

Robert.Wall · 2020-05-05T23:54:00.462Z

If it was a hardware problem - maybe there was some arcing going on inside the adapter - then there’s very little anyone can do about it.

As I explained above, it only makes the decision about whether to use the assumed voltage or the measured voltage at power-up, so after it had decided, it stayed with it. If, in the meantime, it saw spikes etc generated from the energy stored in the transformer core as the primary arced, it’s anyone’s guess what would have been recorded as the voltage - hence the power.

You screenshot means nothing to me as there is no scale and I don’t know what the two colours represent.

The obvious long-term solution is to replace that adapter, or adjust the contact springs to give a reliable connection. If you are prepared to recalibrate the front end, you could get this:
https://www.jaycar.com.au/9v-ac-1-amp-plugpack/p/MP3027
Voltage calibration coefficient - 300.0 (No-load = 10.4 V)

Orionn · 2020-05-06T00:24:58.046Z

Oh, sorry, blue is usage and yellow is solar generation. But it doesn’t really matter.

It would make sense if there was some sort of arcing resulting in an initial spike in voltage, meaning that it used a spurious voltage (e.g. 400V or something), but the vrms measurement graph was showing a flatline at 230V.

Anyway, I’m happy to write it off as something weird that happened to me. If it doesn’t happen to others, no point wasting our time investigating a fixed problem.

I tried flaring out the pins on the adaptor, but it didn’t make much difference. Instead of having the AC adaptor in the wall socket, I’ve used a powerboard so it’s sitting flat and gravity is holding it in the socket. I’ve also ordered a couple of spare adaptors just in case.

Thanks again for your help, I’m very happy it’s all working again.