Homebrew IoTaWatt

Believe it or not I have a plan for morphing the server functionality into classes which can be instantiated to create and remove servers. The idea is to create named servers for specified resources, running at specified intervals. Allow multiple instances of say, Emoncms posting - one going to a Pi, one to Emoncms.org every ten seconds - then maybe an instance of pvoutput posting just daily - you get the idea.

That’s one reason I don’t really want to build up the server support system just yet. Conversion can be harder than initial implementation. How that looks in the user interface is a big consideration.

That’s why I say that I’m happy you have what you need for now, and I’ll work with you, on your branch, for now until I have the time to build a more general framework for multiple server processes.

Hello, first thank you @overeasy for this project, this is exactly what I was looking for ! I started to build a similar project (but did not had time and knowledge to continue). Anyway I have parts lying around that I would like to use to try Iotawatt. So here are some questions :

  • Can I try with only one MCP3208 and then 6 CT inputs?
  • Can I replace the 2.5V voltage reference with a simple voltage divider?
  • Is the LM358 needed, how can I ommit it?
  • By reading this thread it seems that the RTC is not needed if the ESP is connected to internet, is it right?

Sorry for all theses questions, I would like to try it before ordering more parts. Thanks a lot!

  • Yes
  • No
  • No, seven voltage dividers
  • Complicated question. Depends how much functionality you want.

Hi Bob,

A very small thing that tripped me up just a moment ago, in the schematic ADCCS0 and ADCCS1 appear to be reversed compared to everything else.

If this is incorrect would you mind changing this in the schematic so it doesn’t trip anyone else up? I am not using Eagle and so cant make this change to propose as a git push.

The code has:
#define pin_CS_ADC0 0
#define pin_CS_ADC1 2

And the schematic currently has:
ADCCS1, connected to pin 27 of NodeMCU = GPIO0 serving the input CT channels 1-7 and VT channel
ADCCS0, connected to pin 26 of NodeMCU = GPIO2 serving the input CT channels 8-14 and vref channel

I assumed ADCCS0 was connected to ADC0 serving channels 1 - 7, but it wasnt the case.

Also in case you are interested, I have created branch enabling me to debug the input VT and CT waveforms from the IoTaWatt web interface. I used it to plot the waveforms using the same plotting library used for the history via the web interface. Please let me know if you would be interested in adding this. If not I wont bother cleaning it up any more, if so I will test it a bit more once I have my new hardware and create a pull request:

I used that page to debug a problem in input voltage reference. It looks like my VT is clipped before reaching IoTaWatt (still looking into this, but appeared on oscilliscope as well so maybe I have a bad plugpack, was just one I found in the garage so I have ordered another one from ebay)

I’ll take a look at that and see if I can resolve it w/o messing up the manufacturing process.

EDIT:

I’ll take a look at that as well. I do have a way to print the datapoints to the Serial, and in the past I’ve cut and pasted that to a spreadsheet to plot. Not a bad idea to zing it out in a json data packet, but I’m not sure I’d use the same format as the datalog API. Let me see if there’s something that works with my long term development goals. Thanks.

Currently i have 3 homemade IotaWatt’s running.
2 are connected to a usb hub (the amazone one Overeasy recommended) and one is powered directly from a 1.5A samsung phone charger.
One of them is running a slightly changed firmware (i added a serial output via the usb cable to push the values to my Emonpi).
About 1m from the iota’s i installed a dlink wifi router (to get perfect reception via wifi).
However it seems that they stop to respond to wifi sometimes. There is no real fixed time between the dropouts (sometimes a week sometimes a day…). However it seems that the iota is still working because the serial output still works perfectly.
The logs are not really showing the error on the one with the serial output. (does not post via wifi). One of the other 2 shows errors like this:

12/9/17 14:12:00 Resending EmonCMS data.
12/9/17 14:12:06 EmonService: GET failed. HTTP code: connection refused
12/9/17 14:13:06 Resending EmonCMS data.
12/9/17 14:13:12 EmonService: GET failed. HTTP code: connection refused

The 3the one shows no errors. (this is the one running the latest version on the git)

On 2 occasions after cycling the power (unplug usb) the iota showed the webpage to connect to a wifi network. (but it was already connecting since i was getting to it by it’s ip (and not as an access point)

Has anyone experience the same problem?

UPDATE: while browsing through the latest code i noticed that the esp is coded to restart after the 10th failed retry to upload the data…(i was looking to implement this myself…but to late it seems :wink: ) The 3th iota has this code on it (but i restarted it manually before it tried 10 times), so i’ll look what it does (and i’ll flash the others to this version if it solves the problem) :wink:

Hi Jan,

Stab from the past. I thought you were all set. Interest is picking up on three phase. Since you have been at it longer than anyone, maybe you could share some insights to other three phase users. For those who haven’t followed this thread, Jan built homebrew units with three native voltage inputs specifically for three phase. This was before I had developed the current approach of going straight into input channels with a simple adapter. Both seem to work equally well.

Communication issues seem to go with the territory with the ESP8266. I like to use the expression: “Things happen, it’s what you do about it”. So I’ll try to convey a little IoTaWatt philosophy to put this all in perspective.

The ESP8266 runs the WiFi stack in a separate layer of code that is pretty much independent of everything else. Whether the ESP is connected to WiFi or not seems to be more of an opinion than fact. The application can, and I do, register a callback to be notified of a change in status, yet I rarely get the call, and it always seems to be late. From the moment power is applied, the low level WiFi code starts working to establish and maintain the connection, even before the application is formally running. It does a pretty good job, seems to recover well, and is tenacious about retrying.

I could ask a dozen question to narrow this down, but the ultimate question is whether the data eventually gets uploaded completely. The latest release (02_02_27) is pretty robust. When starting the Emonservice, it reads the status of your Emoncms node from the server and picks up at the last known upload time. There should be no holes.

If the IoTaWatt is truly disconnected, i.e. the WiFistack has issued the disconnect callback, your LED will change from a dull green glow to a dull red glow. When the connection is restored, it will revert to green again. Throughout this, the integrity of the data upload will be preserved, and sampling to the local datalog will continue.

When data upload fails, there is a 10 retry sequence that takes up to an hour. During this sequence, the timeout is increased and the delay between retries increases progressively. Upon failure of the tenth retry, the IoTaWatt will restart.

I’ve got an IoTaWatt in an unheated outbuilding on the outer fringes of my WiFi coverage. It monitors my meter and it’s a trainwreck when it comes to communications. Doesn’t work for days in bad weather, rarely stays current for more than a few minutes, and connects and disconnects constantly. My feeling is that if this thing maintains data integrity to Emoncms, anything can. It pretty much does.

The key here is what the LED is doing. If it’s Red-Green-Green, then the firmware is running WiFiManager to connect. Regardless of whether you are accessing IoTaWatt as an AP or STA, you are using an IP to address it. The difference is that in AP mode, you are talking directly to the IoTaWatt (on IP 192.168.4.1) as opposed to an IP assigned by your WiFi router via DHCP. Sometimes there’s a latency after reboot where the IoTaWatt firmware has invoked WiFiManager to establish a connection, and the underlying WiFi stack has succeeded in reconnecting to the last known network. That does resolve after a few minutes. Once again, all of that is at lower levels, so you just have to go with the flow.

You’ve probably run into this already, but I have to caution you about the latest Git version and the appropriate core. Since02_02_24, I’ve been using the “staged” arduino/esp8266 core. I switched to this to get the Krack vulnerability fix, but this version of the core has a lot of other changes, some of which are not compatible with the older core. I had to change a few things in the process such that the latest version in the Git probably will have problems running on the old core. The platformio build using the staged-core is what’s reflected in the official releases. Anything else could have problems.

I’ve unilaterally decided that the plural of IoTaWatt is IoTaWatt, as in “I have three homemade IoTaWatt running.”.

Thanks for the fast reply :wink: as edited in the original message i only noticed the restart function just a moment ago (when looking to implement it myself :wink: )

As far as the 3 phase is going: all 3 IotaWatt (no 's this time :slight_smile: ) share the same type of pcb board, so they all have 3VT connected through a dedicated CT connection (basically a copy of the primary).
I did some measurements and it behaves the same as one would connect it through your simpler connection using the VT connections.

If i would have to do it again, i would prefer your method since it’s more flexable. (you can just choose what to connect)

The measurements seem to be quite accurate and just plug and play. One thing i experienced in the beginning was that if there is a big difference between the number of ct’s on a channel (ex. 1 on VT1 and 11 on VT0) it could experience some troubles (as described further up this thread). These problems seem to have been solved with your updates and a more stable usb power supply.

If there are any specific questions, just let me know, always happy to help. (btw: i am using a 3phase + neuter 400V system - so 400V between phase and N and 230V between the phases)

I’m working to use asyncTCP for client communications, which opens up a whole universe of options for recovery because the activity is non-blocking, and so doesn’t compromise the primary mission of data collection.

I’ve got a PCB adapter that is out for prototype. It will do the interface for 5.5mm power connector from the VT to 3.5mm jack for the IoTaWatt input, and provides the correct resistance spread over four resistors to dissipate the 350mw of power. It has two channels, so can be used to add the two additional phases for one IoTaWatt, or will act as a splitter to connect one VT to two IoTaWatt via the input channels. I hope to have them generally available with the next manufactured batch of IoTaWatt in January.

I know, been quite busy at work the last few weeks…so no time to play around with the IotaWatt :wink:

Thats what i thought… a few esp i coded myself are sometimes showing the same behavior… but the “esp easy mega” soft (www.letscontrolit.com - git link ) seems to be stable (i have 2 sensors laying around, and they seem to keep their connection)

Nice! i guess thats an addition quite some people will like to use!

I’ll try it next time (most of the times i’m quite fast to check if its working :wink: )

I think that was the reason i couldn’t compile the 24 version (it gave some cryptic warnings…). I manually updated my arduino ide and now it seems to work.

Hi Jan,

I would recommend NOT putting your ESP8266 so close to your AP. Just as it is bad to have too little signal, having TOO MUCH signal is also not good and will cause issues.

Having too much signal is similar to sitting too close to someone yelling, no one can hear your reply.

I don’t know if you can view the signal level of your ESP8266, but in WiFi -55 dBm is the magic number for ‘perfect’ signal (with dBm, closer to zero is better signal, so -90 dBm is not as good as -60 dBm).

It is unlikely your only issue you are experiencing, but could be a contributing factor - try moving the AP further away and see if there is an improvement.

Regards,

Dan

had a voltage dropout an hour ago. Nothing fancy, just a few dips it seems (my dekstop computer didn’t go out), but the alarm clocks dit go out. The iotawatt (all 3) didn’t go out either…
However the VT did go out or had a dip or so. The result was that all measurements dropped to 0 and remained there until a (soft) restart (with the /command?restart=1)
I know i had this issue some time ago, but it seemed to be solved but it seems that it’s popping up again.
Any idea’s of thing i can test?

@DJansen: i’ll give it a try to move the dlink a bit further.

Just a few:

What version of firmware and has it been modified?

Is there only one VT on each IoTaWatt?

What do the message logs indicate for this period?

What does the data log show for this event?

versions: nr1:02.02.20 / nr 2 : 02.02.19 / nr 3: 02.02.26
It has been modified: update disabled (commented out the start of the service / added a serial output (just quick and dirty piggybacked the datalog service to print some values to serial)

All 3 have 3 VT and 12CT connected (i know, i have a lot of circuits… :wink: )
All the message logs show nothing about this event…the restart is logged. Before that nr 1 shows nothing (no wifi upload on this one), nr 2 and 3 show some get fails (emoncms) but that was not really a problem as it was 20 to 30min before the power outage.
The data logs: the graps show that everything just dropped to 0…
All the voltage went down and about 10 seconds later the ct’s dropped.
Nothing recoverd :open_mouth:

Not really interested in 1 & 2 as they are quite old versions, but I’d like to see that graph from 3, at 5sec resolution. I can’t see how the voltage can go to zero while the CTs are still recording power for 10 seconds.
Edit: Also, what happens to #3 when you unplug the VT for a few seconds?

I exported the graph from the 3th one to csv: graph.csv (16.7 KB)

Did some tests with unplugging the VT, but somehow i could not replicate the complete thing:
I monitored the 3th iota (latest version) and tried

  • unplugging 1 VT (tried all 3): recovered nice.
  • unplugging 2 VT (tried some combos): recovered nice
  • unplugged 3 VT: recovered nice

Then i checked the status for the other 2: the 1st (version 02.02.20) was running ok (so must have recovered since they all share the VT), the second one (02.02.19) was not recovered.

Not sure what happened, but i’ll monitor it some more if it should happen again. (maybe it was because the power went out for maybe 3 times for a split second.)
I’ll update the older ones to the latest version next weekend, maybe that will solve some stuff?

Looks like the CTs went to zero at the same time as the VTs. That’s reassuring. That the log entries continue every 5 seconds indicates the basic scheduler was running, and there isn’t any mechanism to stop it from sampling voltage or current channels. So there’s got to be more to this story than anyone is noticing.

Must be, strange that i couldn’t reproduce…
They also kept logging to the emoncms, if it happens again, or i can reproduce, i’ll let you know.
I made a script on the raspberry pi checking every 5 min if the volts are down to warn me :wink:

I have 2 more questions:

  • does the tolerance of the resistors make a lot of difference? As far as i know i used the standard 5% tolerance through-hole. (i had them lying around). (i’m using SCT-013-030). For the voltage devider (the VT) and the led I think not, for the resitors around the LM358 and the shunt i’m not sure…

  • sometimes the emoncms upload fails (even on the 02.02.26 version). rebooting (software command) or restarting (cut power) does not always solve it. However, setting the server type to none, and then to emoncms again always works. Any idea’s why this would be so? (the local emoncms is a emonpi)

@overeasy

I have a request and a minor bug report. Is this still the best place for these questions or would you prefer these on github?

  1. Request: Would you be willing to upload the PHP code you use for the updater (iotaupdt.php) to github?

I was unable to locate it in the github source tree and want to run my own local updater (with own priv/pub key) for my hombrew IoTaWatt which would save me re-inventing the wheel and make pushing updates to my running device much easier than flashing via USB while it is in my meter box.

  1. Bug Report : (RTC startup)

I had a bug where IoTaWatt was always failing the call to rtc.initialize() in Setup.cpp and also taking a very long time to obtain the current time via NTP.

The first problem is that my RTC was in a disabled state (Control_3 was set to 0xE0). IoTaWatt would never reset the RTC into an enabled state (Typically I assume you want to use 0 for the top 3 bits of the Control_3 register of RTC to enable both “Standard Battery Switchover” and “Low Battery Detection”).

I see in timeServices.cpp the syncRtc state has a comment indicating that rtc.adjust() sets the to enable both “Standard Battery Switchover” and “Low Battery Detection”. However, rtc.adjust() was never called.

You can reproduce this problem by changing the code to set Control_3 register to 0xE0 in setup. After doing this once, the state was retained by my RTC and the problem showed up again every time I rebooted I needed a successful NTP query before anything would start working.

I dont know how my RTC got into this disabled state in the first place (I think it just started off that way from the begining). I think IoTaWatt firmware should probably be made to be resiliant against this problem.

My solution was to update the conditional for the rtc.adjust() call from:

if(timeDiff < -1 || timeDiff > 1)

to:

// Read Control_3
Wire.beginTransmission(PCF8523_ADDRESS);
Wire.write((byte)PCF8523_CONTROL_3);
Wire.endTransmission();
Wire.requestFrom(PCF8523_ADDRESS, 1);
uint8_t Control_3 = Wire.read();

if(timeDiff < -1 || timeDiff > 1 || ((Control_3 & 0xE0) != 0))

This forces the adjust call to run, which sets the RTC into the desired state (Standard Switchover and Low Detection), but only after getting a correct time from NTP.

The other problem I was having was with older firmware, where an NTP query would always timeout. I have a large ping. The increase in the timeout helps somewhat, but a better solution is using the NTP pool servers instead of the main reference.

I.e. I changed the NTP host from “time.nist.gov”, to using: “pool.ntp.org”. Changing this greatly reduced the time required to synchronize with the NTP servers for me.

Thanks,
Brendon.

1 Like