Homebrew IoTaWatt

overeasy · 2017-11-24T14:23:54.874Z

bcosta:

A very small thing that tripped me up just a moment ago, in the schematic ADCCS0 and ADCCS1 appear to be reversed compared to everything else.

I’ll take a look at that and see if I can resolve it w/o messing up the manufacturing process.

EDIT:

bcosta:

Also in case you are interested, I have created branch enabling me to debug the input VT and CT waveforms from the IoTaWatt web interface. I used it to plot the waveforms using the same plotting library used for the history via the web interface.

I’ll take a look at that as well. I do have a way to print the datapoints to the Serial, and in the past I’ve cut and pasted that to a spreadsheet to plot. Not a bad idea to zing it out in a json data packet, but I’m not sure I’d use the same format as the datalog API. Let me see if there’s something that works with my long term development goals. Thanks.

promy · 2017-12-09T13:26:57.607Z

Currently i have 3 homemade IotaWatt’s running.
2 are connected to a usb hub (the amazone one Overeasy recommended) and one is powered directly from a 1.5A samsung phone charger.
One of them is running a slightly changed firmware (i added a serial output via the usb cable to push the values to my Emonpi).
About 1m from the iota’s i installed a dlink wifi router (to get perfect reception via wifi).
However it seems that they stop to respond to wifi sometimes. There is no real fixed time between the dropouts (sometimes a week sometimes a day…). However it seems that the iota is still working because the serial output still works perfectly.
The logs are not really showing the error on the one with the serial output. (does not post via wifi). One of the other 2 shows errors like this:

12/9/17 14:12:00 Resending EmonCMS data.
12/9/17 14:12:06 EmonService: GET failed. HTTP code: connection refused
12/9/17 14:13:06 Resending EmonCMS data.
12/9/17 14:13:12 EmonService: GET failed. HTTP code: connection refused

The 3the one shows no errors. (this is the one running the latest version on the git)

On 2 occasions after cycling the power (unplug usb) the iota showed the webpage to connect to a wifi network. (but it was already connecting since i was getting to it by it’s ip (and not as an access point)

Has anyone experience the same problem?

UPDATE: while browsing through the latest code i noticed that the esp is coded to restart after the 10th failed retry to upload the data…(i was looking to implement this myself…but to late it seems ) The 3th iota has this code on it (but i restarted it manually before it tried 10 times), so i’ll look what it does (and i’ll flash the others to this version if it solves the problem)

overeasy · 2017-12-09T15:20:24.126Z

Hi Jan,

Stab from the past. I thought you were all set. Interest is picking up on three phase. Since you have been at it longer than anyone, maybe you could share some insights to other three phase users. For those who haven’t followed this thread, Jan built homebrew units with three native voltage inputs specifically for three phase. This was before I had developed the current approach of going straight into input channels with a simple adapter. Both seem to work equally well.

promy:

Has anyone experience the same problem?

Communication issues seem to go with the territory with the ESP8266. I like to use the expression: “Things happen, it’s what you do about it”. So I’ll try to convey a little IoTaWatt philosophy to put this all in perspective.

The ESP8266 runs the WiFi stack in a separate layer of code that is pretty much independent of everything else. Whether the ESP is connected to WiFi or not seems to be more of an opinion than fact. The application can, and I do, register a callback to be notified of a change in status, yet I rarely get the call, and it always seems to be late. From the moment power is applied, the low level WiFi code starts working to establish and maintain the connection, even before the application is formally running. It does a pretty good job, seems to recover well, and is tenacious about retrying.

I could ask a dozen question to narrow this down, but the ultimate question is whether the data eventually gets uploaded completely. The latest release (02_02_27) is pretty robust. When starting the Emonservice, it reads the status of your Emoncms node from the server and picks up at the last known upload time. There should be no holes.

If the IoTaWatt is truly disconnected, i.e. the WiFistack has issued the disconnect callback, your LED will change from a dull green glow to a dull red glow. When the connection is restored, it will revert to green again. Throughout this, the integrity of the data upload will be preserved, and sampling to the local datalog will continue.

When data upload fails, there is a 10 retry sequence that takes up to an hour. During this sequence, the timeout is increased and the delay between retries increases progressively. Upon failure of the tenth retry, the IoTaWatt will restart.

I’ve got an IoTaWatt in an unheated outbuilding on the outer fringes of my WiFi coverage. It monitors my meter and it’s a trainwreck when it comes to communications. Doesn’t work for days in bad weather, rarely stays current for more than a few minutes, and connects and disconnects constantly. My feeling is that if this thing maintains data integrity to Emoncms, anything can. It pretty much does.

promy:

On 2 occasions after cycling the power (unplug usb) the iota showed the webpage to connect to a wifi network. (but it was already connecting since i was getting to it by it’s ip (and not as an access point)

The key here is what the LED is doing. If it’s Red-Green-Green, then the firmware is running WiFiManager to connect. Regardless of whether you are accessing IoTaWatt as an AP or STA, you are using an IP to address it. The difference is that in AP mode, you are talking directly to the IoTaWatt (on IP 192.168.4.1) as opposed to an IP assigned by your WiFi router via DHCP. Sometimes there’s a latency after reboot where the IoTaWatt firmware has invoked WiFiManager to establish a connection, and the underlying WiFi stack has succeeded in reconnecting to the last known network. That does resolve after a few minutes. Once again, all of that is at lower levels, so you just have to go with the flow.

promy:

The 3the one shows no errors. (this is the one running the latest version on the git)

You’ve probably run into this already, but I have to caution you about the latest Git version and the appropriate core. Since02_02_24, I’ve been using the “staged” arduino/esp8266 core. I switched to this to get the Krack vulnerability fix, but this version of the core has a lot of other changes, some of which are not compatible with the older core. I had to change a few things in the process such that the latest version in the Git probably will have problems running on the old core. The platformio build using the staged-core is what’s reflected in the official releases. Anything else could have problems.

promy:

Currently i have 3 homemade IotaWatt’s running.

I’ve unilaterally decided that the plural of IoTaWatt is IoTaWatt, as in “I have three homemade IoTaWatt running.”.

promy · 2017-12-09T15:33:53.868Z

Thanks for the fast reply as edited in the original message i only noticed the restart function just a moment ago (when looking to implement it myself )

As far as the 3 phase is going: all 3 IotaWatt (no 's this time ) share the same type of pcb board, so they all have 3VT connected through a dedicated CT connection (basically a copy of the primary).
I did some measurements and it behaves the same as one would connect it through your simpler connection using the VT connections.

If i would have to do it again, i would prefer your method since it’s more flexable. (you can just choose what to connect)

The measurements seem to be quite accurate and just plug and play. One thing i experienced in the beginning was that if there is a big difference between the number of ct’s on a channel (ex. 1 on VT1 and 11 on VT0) it could experience some troubles (as described further up this thread). These problems seem to have been solved with your updates and a more stable usb power supply.

If there are any specific questions, just let me know, always happy to help. (btw: i am using a 3phase + neuter 400V system - so 400V between phase and N and 230V between the phases)

overeasy · 2017-12-09T15:53:24.521Z

promy:

Thanks for the fast reply as edited in the original message i only noticed the restart function just a moment ago (when looking to implement it myself )

I’m working to use asyncTCP for client communications, which opens up a whole universe of options for recovery because the activity is non-blocking, and so doesn’t compromise the primary mission of data collection.

promy:

If i would have to do it again, i would prefer your method since it’s more flexable. (you can just choose what to connect)

I’ve got a PCB adapter that is out for prototype. It will do the interface for 5.5mm power connector from the VT to 3.5mm jack for the IoTaWatt input, and provides the correct resistance spread over four resistors to dissipate the 350mw of power. It has two channels, so can be used to add the two additional phases for one IoTaWatt, or will act as a splitter to connect one VT to two IoTaWatt via the input channels. I hope to have them generally available with the next manufactured batch of IoTaWatt in January.

promy · 2017-12-09T16:24:39.540Z

overeasy:

Stab from the past. I thought you were all set

I know, been quite busy at work the last few weeks…so no time to play around with the IotaWatt

overeasy:

Communication issues seem to go with the territory with the ESP8266. I like to use the expression: “Things happen, it’s what you do about it”. So I’ll try to convey a little IoTaWatt philosophy to put this all in perspective.

Thats what i thought… a few esp i coded myself are sometimes showing the same behavior… but the “esp easy mega” soft (www.letscontrolit.com - git link ) seems to be stable (i have 2 sensors laying around, and they seem to keep their connection)

overeasy:

I’ve got a PCB adapter that is out for prototype. It will do the interface for 5.5mm power connector from the VT to 3.5mm jack for the IoTaWatt input, and provides the correct resistance spread over four resistors to dissipate the 350mw of power.

Nice! i guess thats an addition quite some people will like to use!

overeasy:

The key here is what the LED is doing. If it’s Red-Green-Green,…That does resolve after a few minutes. Once again, all of that is at lower levels, so you just have to go with the flow.

I’ll try it next time (most of the times i’m quite fast to check if its working )

overeasy:

You’ve probably run into this already, but I have to caution you about the latest Git version and the appropriate core.

I think that was the reason i couldn’t compile the 24 version (it gave some cryptic warnings…). I manually updated my arduino ide and now it seems to work.

DJansen · 2017-12-11T11:20:47.862Z

promy:

About 1m from the iota’s i installed a dlink wifi router (to get perfect reception via wifi).

Hi Jan,

I would recommend NOT putting your ESP8266 so close to your AP. Just as it is bad to have too little signal, having TOO MUCH signal is also not good and will cause issues.

Having too much signal is similar to sitting too close to someone yelling, no one can hear your reply.

I don’t know if you can view the signal level of your ESP8266, but in WiFi -55 dBm is the magic number for ‘perfect’ signal (with dBm, closer to zero is better signal, so -90 dBm is not as good as -60 dBm).

It is unlikely your only issue you are experiencing, but could be a contributing factor - try moving the AP further away and see if there is an improvement.

Regards,

Dan

promy · 2017-12-11T17:19:09.933Z

had a voltage dropout an hour ago. Nothing fancy, just a few dips it seems (my dekstop computer didn’t go out), but the alarm clocks dit go out. The iotawatt (all 3) didn’t go out either…
However the VT did go out or had a dip or so. The result was that all measurements dropped to 0 and remained there until a (soft) restart (with the /command?restart=1)
I know i had this issue some time ago, but it seemed to be solved but it seems that it’s popping up again.
Any idea’s of thing i can test?

@DJansen: i’ll give it a try to move the dlink a bit further.

overeasy · 2017-12-11T17:47:20.505Z

promy:

Any idea’s of thing i can test?

Just a few:

What version of firmware and has it been modified?

Is there only one VT on each IoTaWatt?

What do the message logs indicate for this period?

What does the data log show for this event?

promy · 2017-12-11T18:29:19.835Z

versions: nr1:02.02.20 / nr 2 : 02.02.19 / nr 3: 02.02.26
It has been modified: update disabled (commented out the start of the service / added a serial output (just quick and dirty piggybacked the datalog service to print some values to serial)

All 3 have 3 VT and 12CT connected (i know, i have a lot of circuits… )
All the message logs show nothing about this event…the restart is logged. Before that nr 1 shows nothing (no wifi upload on this one), nr 2 and 3 show some get fails (emoncms) but that was not really a problem as it was 20 to 30min before the power outage.
The data logs: the graps show that everything just dropped to 0…
All the voltage went down and about 10 seconds later the ct’s dropped.
Nothing recoverd

overeasy · 2017-12-11T19:08:28.842Z

promy:

The data logs: the graps show that everything just dropped to 0…

All the voltage went down and about 10 seconds later the ct’s dropped.

Not really interested in 1 & 2 as they are quite old versions, but I’d like to see that graph from 3, at 5sec resolution. I can’t see how the voltage can go to zero while the CTs are still recording power for 10 seconds.
Edit: Also, what happens to #3 when you unplug the VT for a few seconds?

promy · 2017-12-11T19:59:48.085Z

I exported the graph from the 3th one to csv: graph.csv (16.7 KB)

Did some tests with unplugging the VT, but somehow i could not replicate the complete thing:
I monitored the 3th iota (latest version) and tried

unplugging 1 VT (tried all 3): recovered nice.
unplugging 2 VT (tried some combos): recovered nice
unplugged 3 VT: recovered nice

Then i checked the status for the other 2: the 1st (version 02.02.20) was running ok (so must have recovered since they all share the VT), the second one (02.02.19) was not recovered.

Not sure what happened, but i’ll monitor it some more if it should happen again. (maybe it was because the power went out for maybe 3 times for a split second.)
I’ll update the older ones to the latest version next weekend, maybe that will solve some stuff?

overeasy · 2017-12-11T20:29:15.408Z

promy:

Not sure what happened

Looks like the CTs went to zero at the same time as the VTs. That’s reassuring. That the log entries continue every 5 seconds indicates the basic scheduler was running, and there isn’t any mechanism to stop it from sampling voltage or current channels. So there’s got to be more to this story than anyone is noticing.

promy · 2017-12-11T21:11:00.737Z

overeasy:

Looks like the CTs went to zero at the same time as the VTs. That’s reassuring. That the log entries continue every 5 seconds indicates the basic scheduler was running, and there isn’t any mechanism to stop it from sampling voltage or current channels. So there’s got to be more to this story than anyone is noticing.

Must be, strange that i couldn’t reproduce…
They also kept logging to the emoncms, if it happens again, or i can reproduce, i’ll let you know.
I made a script on the raspberry pi checking every 5 min if the volts are down to warn me

promy · 2018-01-30T18:36:49.330Z

I have 2 more questions:

does the tolerance of the resistors make a lot of difference? As far as i know i used the standard 5% tolerance through-hole. (i had them lying around). (i’m using SCT-013-030). For the voltage devider (the VT) and the led I think not, for the resitors around the LM358 and the shunt i’m not sure…
sometimes the emoncms upload fails (even on the 02.02.26 version). rebooting (software command) or restarting (cut power) does not always solve it. However, setting the server type to none, and then to emoncms again always works. Any idea’s why this would be so? (the local emoncms is a emonpi)

bcosta · 2018-02-07T10:37:36.571Z

@overeasy

I have a request and a minor bug report. Is this still the best place for these questions or would you prefer these on github?

Request: Would you be willing to upload the PHP code you use for the updater (iotaupdt.php) to github?

I was unable to locate it in the github source tree and want to run my own local updater (with own priv/pub key) for my hombrew IoTaWatt which would save me re-inventing the wheel and make pushing updates to my running device much easier than flashing via USB while it is in my meter box.

Bug Report : (RTC startup)

I had a bug where IoTaWatt was always failing the call to rtc.initialize() in Setup.cpp and also taking a very long time to obtain the current time via NTP.

The first problem is that my RTC was in a disabled state (Control_3 was set to 0xE0). IoTaWatt would never reset the RTC into an enabled state (Typically I assume you want to use 0 for the top 3 bits of the Control_3 register of RTC to enable both “Standard Battery Switchover” and “Low Battery Detection”).

I see in timeServices.cpp the syncRtc state has a comment indicating that rtc.adjust() sets the to enable both “Standard Battery Switchover” and “Low Battery Detection”. However, rtc.adjust() was never called.

You can reproduce this problem by changing the code to set Control_3 register to 0xE0 in setup. After doing this once, the state was retained by my RTC and the problem showed up again every time I rebooted I needed a successful NTP query before anything would start working.

I dont know how my RTC got into this disabled state in the first place (I think it just started off that way from the begining). I think IoTaWatt firmware should probably be made to be resiliant against this problem.

My solution was to update the conditional for the rtc.adjust() call from:

if(timeDiff < -1 || timeDiff > 1)

to:

// Read Control_3
Wire.beginTransmission(PCF8523_ADDRESS);
Wire.write((byte)PCF8523_CONTROL_3);
Wire.endTransmission();
Wire.requestFrom(PCF8523_ADDRESS, 1);
uint8_t Control_3 = Wire.read();

if(timeDiff < -1 || timeDiff > 1 || ((Control_3 & 0xE0) != 0))

This forces the adjust call to run, which sets the RTC into the desired state (Standard Switchover and Low Detection), but only after getting a correct time from NTP.

The other problem I was having was with older firmware, where an NTP query would always timeout. I have a large ping. The increase in the timeout helps somewhat, but a better solution is using the NTP pool servers instead of the main reference.

I.e. I changed the NTP host from “time.nist.gov”, to using: “pool.ntp.org”. Changing this greatly reduced the time required to synchronize with the NTP servers for me.

Thanks,
Brendon.

overeasy · 2018-02-07T13:43:28.804Z

bcosta:

I have a request and a minor bug report. Is this still the best place for these questions or would you prefer these on github?

Overeasy is moving IoTaWatt

I’ll be at: community.iotawatt.com going forward. It uses the same forum software (Discourse) as this community space, so should be easy transition. You will need to establish an account there to contribute. See you later.

Mario · 2018-03-17T11:07:50.866Z

I would like to build an IoTaWatt. I already have most of the components.

Anyone has an spare PCB (or two) to sell me? (Or can provide me with the files ready to order at some cheap Chinese PCB manufacturer?)

Thanks.

Mario

TrystanLea · 2018-03-17T11:34:08.000Z

Hello Mario, thanks for your interest, might be worth asking Bob Lemaire @overeasy over at https://community.iotawatt.com/

Mario · 2018-03-17T11:50:51.951Z

Thanks!

Will do