The main problem that everybody seems to be skirting around is that the tz database has NO interest in exactly where someone is located and there is no remit TO manage that. Even tzdist which was the latest attempt to plug a major distribution hole has no remit on supplying the correct data for a particular location! Both are simply managing the set of rules that anybody can use where ever they are. Importantly allowing third parts to work out a local time for some elsewhere in the world. Some other data source is really needed to serve up the CURRENT set of rules for a location, and that is probably only currently possible using geonames.org which uses the current timezone_id and it's own simplified timeZone.txt table! What is missing is any historic (or ongoing) changes to that information, such as a location that was using one tz rule set but moved to another. In addition any pre-1970 material can not be identified. tzdist has the remit and tools to fill one of the holes that tz on it's own can not and that is historic changes TO the rule sets. My interest in the tz database came from a major problem with material that had been normalised to UTC time but there was no record as to what rules had been used TO normalise the data, so one could not verify if the CURRENT tz rules were appropriate for establishing the then local times. As Paul has quite rightly pointed out, much historic data was invented, so CORRECTING the data when more accurate information is established is perfectly correct, but one needs to be able to also establish just which rules have changed and one can then adjust the material that is now suspect ( - IF APPROPRIATE ;) ) The whole point of tzdist was to sort current day problems, so if a mobile device with a stored calendar lands in a new location where there has been an 'unscheduled' change of tz rules that change CAN be propagated quickly without needing a full tz update but that will only work if someone is actually running the primary tzdist source. Moving forward, that primary tzdist source needs to maintain a full historic record prior to 1970 since it also provides the base from which rules for historic data normalization can be managed. It allows for the tz timezone_id's to be used to identify the base rule sets, but allows new rule sets to be created ... just without any rules on just how THAT is managed :( Potentially any geonameid could have it's own individual tzdist dataset and track the history of changes as a location moves from one political jurisdiction to another and show it's simple solar time as appropriate. Something that neither tzdist or tz is actually mandated to supply? Merging geonames.org with openstreetmap.org one gets graphic displays of the area that timezones CURRENTLY cover, but one can not establish if the same offsets applied back in the 1960's when DST local times are well documented. And both databases throw away the historic data to those mappings along with a whole load of other historic material. While the vast majority of users probably could not care less, the annoyance here is that the material existed so it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another. Once the mechanism is in place, then as new historic material comes to light, the historic record can be corrected and tzdist has the mechanism for that in relation to timezone rules. So as a start ... what has happened to tzdist. The full historic record of tz changes can provide a complete set of data that has been used over the last 20 years and as new changes are logged in now they can be pushed to tzdist while also maintaining tz distribution for off-line population systems. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Dec 5, 2017, at 1:31 AM, Lester Caine <lester@lsces.co.uk> wrote:
The main problem that everybody seems to be skirting around is that the tz database has NO interest in exactly where someone is located
It's obviously not the tzdb's job to determine their location, so presumably you mean "the tz database has NO interest in providing information to determine, for a given longitude and latitude, the tzdb region containing that longitude and latitude".
Even tzdist which was the latest attempt to plug a major distribution hole has no remit on supplying the correct data for a particular location!
I.e., they don't offer, for example, a service where you send it a longitude/latitude pair and it returns data for the tzdb region containing that location.
Some other data source is really needed to serve up the CURRENT set of rules for a location, and that is probably only currently possible using geonames.org which uses the current timezone_id and it's own simplified timeZone.txt table! What is missing is any historic (or ongoing) changes to that information, such as a location that was using one tz rule set but moved to another.
That would happen only if the tzdb region containing that location were to split, with the location being in the newly-created tzdb region (rather than remaining in the old region). In that case, I'm not sure why the tzdb region it was in before the split is relevant; before the split, the old region and the new region have the same rules, as they weren't yet separate regions, and after the split, the rules of the old region aren't relevant to the location.
In addition any pre-1970 material can not be identified.
That's because, as has been noted in the past, having separate tzdb regions for locations that used different rules prior to 1970 but that didn't use different rules in 1970 or afterwards isn't a high priority for the tzdb. That's a separate issue unrelated to the "mapping locations to the tzdb region in which they're located" issue.
tzdist has the remit and tools to fill one of the holes that tz on it's own can not and that is historic changes TO the rule sets.
If you're thinking of https://tools.ietf.org/html/rfc7808#section-3.10 that doesn't seem to provide anything that access to a complete archive of tzdb releases would. And that's an issue separate from the "mapping a location to a tzdb region" issue.
The whole point of tzdist was to sort current day problems, so if a mobile device with a stored calendar lands in a new location where there has been an 'unscheduled' change of tz rules that change CAN be propagated quickly without needing a full tz update but that will only work if someone is actually running the primary tzdist source. Moving forward, that primary tzdist source needs to maintain a full historic record prior to 1970
...assuming they wish to provide services for people processing pre-1970 local times. Perhaps they might not wish to, disappointing though that may be for those who want to process pre-1970 local times.
Merging geonames.org with openstreetmap.org one gets graphic displays of the area that timezones CURRENTLY cover, but one can not establish if the same offsets applied back in the 1960's when DST local times are well documented. And both databases throw away the historic data to those mappings along with a whole load of other historic material. While the vast majority of users probably could not care less, the annoyance here is that the material existed so it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another.
...and what the tzdb did in response was to add some additional lines to the entry for the *existing* entry for the Europe/Simferopol tzdb region. I don't know whether any tzdb regions changed their boundaries; if not, then, from the point of view of the tzdb, no location changed which tzdb region it was in, the *existing* rules for a tzdb region changed by adding new transitions. That change isn't a correction to historical tzdb entries, of course, it's a consequence of the annexation of Crimea by the Russian Federation, so presumably by "it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another", you mean "a mechanism that remembers that new entries were added to the Europe/Simferopol rules in 2014 can also be used to remember that Southeast Nowheresville's UTC offset in 1905 was corrected from 1:30 to 1:47". BTW, if there's any mechanism in tzdist to request particular versions of the tzdb, or to enumerate all of the versions that have ever existed, I'm not seeing it anywhere in RFC 7808, so either I missed it or it was added subsequently.
On 12/05/2017 05:52 AM, Guy Harris wrote:
On Dec 5, 2017, at 1:31 AM, Lester Caine <lester@lsces.co.uk> wrote:
The main problem that everybody seems to be skirting around is that the tz database has NO interest in exactly where someone is located It's obviously not the tzdb's job to determine their location, so presumably you mean "the tz database has NO interest in providing information to determine, for a given longitude and latitude, the tzdb region containing that longitude and latitude".
Even tzdist which was the latest attempt to plug a major distribution hole has no remit on supplying the correct data for a particular location! I.e., they don't offer, for example, a service where you send it a longitude/latitude pair and it returns data for the tzdb region containing that location.
Its not standardized yet, and I probably have the only implementation at this point, but this does exist and I have been unsuccessful in soliciting feedback: https://tools.ietf.org/html/draft-murchison-tzdist-geolocate-01 -- Kenneth Murchison Cyrus Development Team FastMail Pty Ltd
On 05/12/17 10:52, Guy Harris wrote:
On Dec 5, 2017, at 1:31 AM, Lester Caine <lester@lsces.co.uk> wrote:
The main problem that everybody seems to be skirting around is that the tz database has NO interest in exactly where someone is located
It's obviously not the tzdb's job to determine their location, so presumably you mean "the tz database has NO interest in providing information to determine, for a given longitude and latitude, the tzdb region containing that longitude and latitude".
Actually my English was probably exactly what I meant. tzdb IS purely a database of time change rules. While some locations seem to expect their own personal set of rules, the reality is that many locations simply share one set of rules.
Even tzdist which was the latest attempt to plug a major distribution hole has no remit on supplying the correct data for a particular location!
I.e., they don't offer, for example, a service where you send it a longitude/latitude pair and it returns data for the tzdb region containing that location.
The geographic lookup was certainly part of the discussion, but deemed out of scope for RFC7808. I can't remember now what happened about the geolocate proposals :( 4.1.8 seems to be the 'coverall' for bits that did not fully make the RFC ...
Some other data source is really needed to serve up the CURRENT set of rules for a location, and that is probably only currently possible using geonames.org which uses the current timezone_id and it's own simplified timeZone.txt table! What is missing is any historic (or ongoing) changes to that information, such as a location that was using one tz rule set but moved to another.
That would happen only if the tzdb region containing that location were to split, with the location being in the newly-created tzdb region (rather than remaining in the old region).
In that case, I'm not sure why the tzdb region it was in before the split is relevant; before the split, the old region and the new region have the same rules, as they weren't yet separate regions, and after the split, the rules of the old region aren't relevant to the location.
So you are not interested in maintaining accurate local time normalization for say the 2nd world war. As I said for many people it is irrelevant, which is why RFC7808 has the option ONLY to provide a short version of the database. The problem comes when someone accesses data that HAS been normalised and expects a tz look up to provide the then current local time. Event X happened an hour before event Y because the correct DST offset is missing in your version of tz data.
In addition any pre-1970 material can not be identified.
That's because, as has been noted in the past, having separate tzdb regions for locations that used different rules prior to 1970 but that didn't use different rules in 1970 or afterwards isn't a high priority for the tzdb. That's a separate issue unrelated to the "mapping locations to the tzdb region in which they're located" issue.
Pre-1970 data IS all about the location and the start 'event' is the move from solar time to some agreed standard time. I accept that this is 'unrelated' to tzdb but it IS a major gap in the standardisation of what we are using.
tzdist has the remit and tools to fill one of the holes that tz on it's own can not and that is historic changes TO the rule sets.
If you're thinking of
https://tools.ietf.org/html/rfc7808#section-3.10
that doesn't seem to provide anything that access to a complete archive of tzdb releases would.
tzdb is one possible publisher of tz data and as such follows the monolithic model. When a copy of that data is downloaded it will identify the version that has been download. Because of the 'committee' nature of defining RFC's there is as much unwritten as is written! and a user needs to know what version of data they are looking at. The 'get' clause that is actually used needs to be able to 'get' the version of tzdist data that matches the version it is to be used with. This may indeed be an alternate publisher even but there is no guarantee that the CURRENT set of rules for a timezone actually matches the set required. In the case of notification of short term changes to offset, the 'changedsince' value only ensure that a current set of rules have changed, not the rules that were in place when a calender of events was created.
And that's an issue separate from the "mapping a location to a tzdb region" issue.
Not saying otherwise ...
The whole point of tzdist was to sort current day problems, so if a mobile device with a stored calendar lands in a new location where there has been an 'unscheduled' change of tz rules that change CAN be propagated quickly without needing a full tz update but that will only work if someone is actually running the primary tzdist source. Moving forward, that primary tzdist source needs to maintain a full historic record prior to 1970
...assuming they wish to provide services for people processing pre-1970 local times. Perhaps they might not wish to, disappointing though that may be for those who want to process pre-1970 local times.
The rules for a 'Root Provider' on RFC7808 is that it has to provide a full set of tz data. It can't simply provided a truncated set of data.
Merging geonames.org with openstreetmap.org one gets graphic displays of the area that timezones CURRENTLY cover, but one can not establish if the same offsets applied back in the 1960's when DST local times are well documented. And both databases throw away the historic data to those mappings along with a whole load of other historic material. While the vast majority of users probably could not care less, the annoyance here is that the material existed so it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another.
...and what the tzdb did in response was to add some additional lines to the entry for the *existing* entry for the Europe/Simferopol tzdb region. I don't know whether any tzdb regions changed their boundaries; if not, then, from the point of view of the tzdb, no location changed which tzdb region it was in, the *existing* rules for a tzdb region changed by adding new transitions.
That change isn't a correction to historical tzdb entries, of course, it's a consequence of the annexation of Crimea by the Russian Federation, so presumably by "it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another", you mean "a mechanism that remembers that new entries were added to the Europe/Simferopol rules in 2014 can also be used to remember that Southeast Nowheresville's UTC offset in 1905 was corrected from 1:30 to 1:47".
This is about having a mechanism that can at the very least say "The normalised event time I have may now be wrong". If I've created a timetable for an international conference that has events across several timezones, if one event site moves in time one needs to know? It would be nice if the system also said "event X's local time has change to XX:XX" but any decision on does the event's UTC stay fixed or does the other sites timetable need to be adjusted to compensate IS outside the process of updating that timezone data.
BTW, if there's any mechanism in tzdist to request particular versions of the tzdb, or to enumerate all of the versions that have ever existed, I'm not seeing it anywhere in RFC 7808, so either I missed it or it was added subsequently.
The problem is when a majority of contributors are ONLY concerned with a current set of working offsets some things are very difficult to get accepted. Does ANY RFC actually nail down all of the rules? In an ideal world tzdist would only distribute tzdb data and updates, but that was not acceptable to all, so the multiple publishers arrived. They are going to define different timezone id's, hence the need when publishing an event calendar to tell your users just who's id's you are using and hence who's version of the data ... and track changes form that ... Is this why nobody has picked up the baton and started actually using tzdist. tzdb has a lot of limitations, but at least it IS a single set of data! -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
participants (3)
-
Guy Harris -
Ken Murchison -
Lester Caine