Truly grateful for your perseverance Brian and everyone else - Thanks. 
2 Likes
Sort of yes. Since you are working with relative time, it is essential that all devices involved have the correct time. If all device clocks are correct, they will as a result be synced.
No as I’ve explained, the “sentat=” directive in fact removes any mismatch. The idea behind that feature is that (for example) imagine a Pi that is unable to sync it’s fake-clock for some reason and it thinks it’s currently yesterday (1 day slow), emoncms would see that the incoming request was sent just now by a device that thinks it is yesterday and by using a ( now - sentat = adjustment ) sum, it will adjust every supplied timestamp by that “adjustment” value, in this example, by +86400 secs.
This theoretically works well, but, in reality, when working with a device that is able to maintain a good time, it’s benefit, arguably isn’t really needed and the downside can cause issues of it’s own. Imaging a Pi that has good time, sends a request to emoncms and all is tickety-boo except there is a network delay of 90s (the default timeout is 2mins I think), now emoncms will assume that the sender’s clock is out and adjust all the timestamps by 90s to bring inline with the server time, effectively fudging your once accurate data.
This is why there are several discussions on this forum about sending absolute timestamps that do not get adjusted. The only way to do that is to use absolute timestamps in each data array and then use “time=0” (the zero is most important) instead of “sentat=” whatever the time is at sending. My own copy of emonhub and IoTaWatt both use “time=0”.
No, not if using “sentat=” then the result is the same, eg if our Pi thought it was currently tomorrow, then emoncms would apply a (minus) -86400 secs so that the data aligned with the server time. However, if you used “time=0” then the data would be processed with the advanced timestamp up to 48hrs in advance, IIRC feeds can be written to up to 48hrs in advance or 5years behind (unless using feedwriter).
When I reread the thread this morning this threw me, I’m now guessing this isn’t entirely accurate? The time difference will not cause sporadic posting, it would have been a constant 120s delay +/- the 10s fresh rate, “appearing every 120s” would have seen the update interval vary by 120s, eg gradually build up to 240s and then drop to 120s in one go.
I was also thown by Bill’s post being marked as the solution, surely correcting the Pi’s time had no effect, correcting the browser time was the solution?
1 Like
Time was the essence of the fault - it turns out it wasn’t inconsistent uploads - it was a constant roughly 2min delay.
I marked Bills answer as the solution primarily as that was the trigger for digging deeper into time. 
If anyone has the same problem and reads the thread - I would like to believe it would give them some things to check, including the time on all devices - not just the pi?
1 Like
Yes I suspect you are right.
Could there be something in on the display side of Emoncms that could indicate if there is a browser/server time difference?
I guess there could be, but IMO the browser time is irreverent and shouldn’t even be used. I see no operational benefit to using the browser time over server time, the only time it would be noticeable to the user is when there’s a problem caused by a difference as we’ve seen here. I assume the browser time is used because it’s easier to code perhaps? This is after all most likely an edge case, most devices keep good time these days and there is always a potential for a couple of seconds error due to rounding and/or network connection delays and refresh rates etc etc, so would we want annoying error messages under normal running or would we set a tolerance? I think just not using browser time at all would be a better option, if we did check for a difference between browser and server time, I would prefer it was then used to automatically adjust any timestamps so the displayed data was accurate rather than bother me with a message and expect me to do the maths in my head on the fly.
Just local if I recall, I have deleted them now.
Is it possible these are created if power goes off so the pi gets a dirty shutdown? I have had a few power cuts and the pi isn’t (yet) on UPS. Now I realise the voltage sensor needs to be direct mains, but the power for the Pi should be ok UPS’d?
It is possible, however the smaller details are the telltales. You previously said that inputs and nodes were created. You can get duplicated inputs to the local emoncms via mqtt due to ungraceful shutdowns, BUT these will only be a single 2nd set of duplicated inputs not nodes (and not a “million and one”), you would see a second complete set of inputs under the same node. The new phantom nodes are a common symptom of RF noise as Robert suggested, they can appear in large numbers but these would appear in both the local and remote emoncms, not just the local one. Unless you live literally in the middle of nowhere, or in a Faraday cage there is always a chance of rogue inputs getting in. I doubt that none of your neighbours have a remote locking vehicle between them (no matter how old they are) and even if that was the case, they, and you most likely get visitors, or couriers, if the’re all in their senior years there maybe a lack of baby monitors perhaps, but garage openers, garden watering systems, weather stations etc are all possible sources and if that’s not, maybe a HAM radio in the next town or a CB’er drove a truck through your village etc etc This is by far the most common cause of duplicate or phantom inputs, users are usually very aware of when the duplicate mqtt inputs occur as it will stop data dead in it’s tracks until the duplicated inputs are manually deleted because the input processing will lie dormont on the original inputs whilst the new (duplicate) inputs with no processing get regularly updated, once deleted, the original inputs start to get updated again.
If you are prone to frequent power outages, I would seriously consider using an older emonSD image, the images prior to the Oct2018 image were all configured as RO OS’s and less vulnerable to SD card corruption due to ungraceful shutdowns.
Yes and No, It will be fine as long as the emonPi is not restarted during an outage. The emonPi only checks for a AC input at boot up. When using a UPS it will remain in “AC signal” mode when the AC signal is removed (power outage) and remain so when the AC is restored. But if the emonPi is rebooted oe the emonPi add-on board is reset, during the outage it will detect there is no AC present and go into “no AC mode” and use a constant “240vac” value in place of the AC signal, so when the AC does come back on, you will only get apparent power (not real power) until the emonPi add-on board is reset again.
There really should be a configuration switch to prevent that behaviour for UPS users. Otherwise, it means a change to the sketch in the emonPi front end.
I agree, I would much prefer a setting to auto-detection, either a dip sw or a software setting saved to the eeprom, it is after all an installation level setting, once set it rarely needs changing, after install day, auto-detection can only at best maintain the same setting or otherwise cause issues.
With the emonPi there is the ability to reset the add-on board via a software switch and the gpio, but how would the emonpi know if the AC was back up if the AC sensor is not in use? The sketch would need to test the AC signal regularly, in which case the Pi doesn’t need to get involved, the FW can sort itself out, but the regular auto-detection would no doubt impact the monitoring.
However I think this could be an easy fix on the emonPi since it is permanently connected to emonHub, the “230V/110V calibration setting” may not be the best example but it is essentially the same sort of thing.
Auto-detection is, I think, non-optimal as demonstrated by this case: If you have an UPS and auto detection, then it will continue to show a voltage - the default 240 V or whatever - even when the mains supply has failed. Power usage will however be correct (as there’s no possibility of a current being measured), but it could still provide misleading information.
By far the most appropriate is something manual, because it’s only the UPS user that might be affected, and not something that’s likely to change too often during the life of the installation.
Totally agree.
I don’t think this is necessarily true though, how many users have had issues due to not getting the order or timing right when connecting sensors and power?
When I can get round to figuring out how the emonPi can do continuous monitoring and handle everything else, that’s going down on my list.
Isn’t the Browser time needed to account for timezones?
I think my point is that it seems the browser (user device) time is used to generate the time since display and in cases where there is a disparity, these issues are then seen.
@TrystanLea @glyn.hudson I’d be interested in your thoughts - the problems seen were apparently due to the end device time not being set correctly.
No, using the browser TZ creates it’s own related but independent issues. It would be better to use the user account TZ. Unless you live in a mobile home, your house stays where it is no matter who looks at the data from where (or when).
IMO It is usually better practice to fix the root issue than to just accommodate the symptoms of the problem.
I’ve added this to the same issue previously raised about not using the browser TZ.