I am having troubles with my daily kwh totals resetting at 10am
I think this is time zone related
emoncms is running on pi desktop
the OS is set to sync with my time ntp tine server the the time zone Australia/Brisbane (UTC +10)
From the admin screen
OS Linux 4.19.0-9-amd64
Host emonpi | emonpi | (172.31.222.222)
Date 2020-06-23 17:46:27 AEST
I notice the SQL section shows
MYSQL
Version 5.5.5-10.3.22-MariaDB-0+deb10u1
Host localhost:6379 (127.0.0.1)
Date 2020-06-23 17:46:26 (UTC 10:00ââ)
My meter data us fed from estron SDM meters via MQTT
kwh readings are lifetime readings
Is this happening because the data us translated to UTC and is then reset at midnight UTC = 10am?
I assume if I set emoncms and the OS to UTC and feed the local time I will get the kwh to kwhd resetting at midnight. is this the only work around or am I missing something?
First thing to know is that all data is stored with a unix timestamp which is equal to UTC (as that is what a timestamp is). So no matter where you are in the world, right now(), is the same unix timestamp.
Have you set the timezone under your user profile?
yes it was always set to local time zone however i cant see how this would ever be related.
You may have users with many timezones but the feed would reset at the same time
Will confirm tomorrow after it resets correctly a second time but as I indicated php 7.3 has two php.ini files that need the local timezone set.
Because internally emoncms looks for that. You can edit it to suit your timezone. It isnât very obvious you can edit it, but place the mouse over it and a pen icon appears
Ok good. There have been issues before over this problem and you may have hit upon the solution that needs to be fixed as part of the install process @TrystanLea.
There is a big discussion on GitHub over timezones. Basically, the system was designed for a single timezone (UK) and user. Outside of that it has some issues.
Conceptually I favour storing the timezone of the Feed in the meta data but others have different views.
The other thing is when should the feed reset? Is it midnight local time at the location of data collection (i.e. where the sensor is), or midnight for the viewer? This relates to my concept above!
For your users, are they looking at the same feed?
Thatâs exactly my take on the problem, for what itâs worth: the timezone (and donât forget daylight saving time offsets - and these can change year on year so the history as well) where the use happens needs to be stored in the database, and that implies itâs âper feedâ.
The browser or whatever is used to view and analyse the data needs to be able to do that using any reasonable timezone - the obvious three are local to the data, local to the viewer, or UTC. A free choice (enter a number ±hh:mm) would be a nice 4th option.
What is the purpose of the reset? If itâs to synchronise with the supplierâs billing, it needs to be the same as that. (e.g. Some old UK âEconomy7â billing periods donât change with daylight saving time.) Otherwise, itâs likely to be midnight local to the usage, but for a shift worker, it could be 6 am local. [I canât find the :minefield: emoticon.]
No you just need the timezone as in Europe/London so the system can look up the offset.
Completely OT, but I am going to be interested in what happens when the EU stops using DST. What will happen to historical data? Will they introduce a new set of timezones?
So you know what has been used within any given period, usually a âdayâ. e.g. the My Electric app
and a daily graph
Not reset as in a hardware reset just when one accumulation stops and another starts.
No, itâs not OT at all - thatâs exactly why you need to store the local timezone and offset as the data is gathered. Thereâs no guarantee the look-up will work in the future. Thereâs no guarantee that the user will apply the changing time offsets anyway, but thatâs their option.
Itâs not as if itâs a massive amount of data: date & time of change and the name of the new offset and its value.
So what is the definition of a âdayâ - thatâs what I was getting at. A shift worker on permanent nights might regard his day as starting at 6 or 7 am.
Personally I think it should be sunset or sunrise.
This shifts over the year but for me is most relevant to solar production and billing
We are net metered here in Australia.
However the meter records import and export not net import.
We pay roughly 3 times more for imported power vs exported power
So the name of the game here is maximising self consumption not exports
You also pay a daily fee for connection to the grid this means I need to export just under 10kwh to offset the daily fee.
The only way to negate the night time cost of imported power is to use battery.
So for me it all comes back to sunrise/sunset
But isnât that the same as midnight? Any time between sunset and sunrise is equivalent from a solar generation point of view. Midnight is a nice fixed time and is shared with lots of other definitions of the change of day.
I have 3 Phase main power plus a single phase off peak.
Off Peak is on a separate meter, fed from L1 (before main meter) and switched on using a ripple control relay.
Generally off peak begins around 10pm and ends around 4pm the next day.
Time may shift at the will of the energy supplier but is for a minimum of 18 hours per 24
With midnight solar production is then split between 2 cycles of off peak.
Sunset/Sunrise aligns solar with off peak.
Aligning with sunset/sunrise simplifies controlling imports/exports and battery.
IE Has my exported power covered the cost of peak and off peak imports.
Does the battery need to be topped up from off peak and is so is it better to run the battery lower or pay for the off peak
I have the first one done processing data externally, working on the second one.
You seem to have a very unusual billing arrangement - Iâve never heard of anything like it before. Certainly I wouldnât say that it represented the mainstream when considering how end of day should be defined. In any case sunset can be before 4pm where I live, so would still lead to the same problem if I had the same billing as you.
In short, I think your argument is very specific to your particular situation, and is not a good argument for a general solution.
Does not seem that unusual to have off peak power
Take a look at the âTime of use - flexible CLâ app
In any case the original issue was solved by editing both ini files
Kwh/D in EmonCMS now resets at midnight as expected
I am getting Kwh/D starting at sunset externally and feeding that to Domoticz which handles load control and the Selectronic Inverter/Charger
All the data from all sources ends up in influxdb and graphing is done in Grafana
Indeed no, I have it myself. But mine and all other cases Iâve heard of are pretty low tech - mine has fixed hours and doesnât even change for daylight saving. The fixed hours vary around the country a bit to minimise the switch-on surge I suppose.
Itâs only with modern digital metering that weâre beginning to see flexible pricing, of which the best known here is Octopus Agile. Based on the wholesale half-hour interval pricing used in commercial supply.
But Iâm glad youâve got your system sorted out.
Rather off-topic, I came across this post in another forum. It raises some very interesting points I for one would never have even thought about:
Yet⊠In a Western framework, we have very limited understanding of calendar issues in a global framework. Date.ToString() may be far from sufficient. I was working with some Koreans: If you asked them about their age, they first had to convert their lunar calendar to a sun calendar, and then correct for their convention of giving your age as the year of your living: At birth, you are starting your first year, not your zeroth.
Another case: When I was working with web archiving, there were native Americans who wanted access control to their archived web pages to be restricted by their concepts of season: Some documents should be accessible during the seeding season only, others only during the harvesting season. (They also wanted some documents to be available to males only, others to females only - but that is not calendar related.)
Some Asian cultures restart their year count from some astronomical or social event, indicating the base as part of their year indication - similar to our BC/AC, but on a lot more dynamic scale.
And so on. Date.ToString() does not cut it. MS has done a lot to support various calendars, but you canât take for granted that it is supported in all applications. (In those I devlop, there is certainly not support for all sorts of Asian calendars!)
So there may be a market for the library that the TS (or some âcoworker/friendâ of his) has created - but mostly in Asian markets. Western software developers generally ignore such issues, taking for granted that the only calendar adaptations required is adapting to the time zone and 24/12 hour clocks.
Lots of Western software have no chance of succeeding in Asia, because the developers have no understanding of Asian culture and the requiremnts that the software should satisfy.