Hi all, As most of you probably know, there is a dispute about the tzdb maintainer's recent changes to merge large numbers of time-zones [1][2]. These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years. It appears that there is soon to be a new release of tzdb containing these changes. In my opinion, the correct behaviour of the tzdb maintainer would be to revert the controversial changes before doing a release. In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream). Is there support for a fork? Is anyone willing to help out? Is any other group willing to sponsor tzdb (eg. CLDR or Red Hat)? Please reply to this thread with an indication of support and/or help. thanks Stephen Colebourne [1] https://mm.icann.org/pipermail/tz/2021-June/thread.html [2] https://mm.icann.org/pipermail/tz/2021-September/030372.html
I think it's a bad idea for a great many reasons, only a few of which are the following: * Time confusion going forward, with new inconsistencies being introduced. * Implementer confusion in terms of which code base is more up-to-date; worse if the code base fragments. * Fragmentation of expertise among volunteers I fear that you drastically underestimate the effort that has been required to maintain both the code and the data. I would suggest an alternative: the policy of this group is set in an RFC, and that RFC can be updated if there is consensus in this community to do so. If the community doesn't like the policy, it can change it, and continue to gain the benefit of one another. Eliot On 20.09.21 10:06, Stephen Colebourne via tz wrote:
Hi all, As most of you probably know, there is a dispute about the tzdb maintainer's recent changes to merge large numbers of time-zones [1][2]. These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
It appears that there is soon to be a new release of tzdb containing these changes. In my opinion, the correct behaviour of the tzdb maintainer would be to revert the controversial changes before doing a release.
In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream).
Is there support for a fork? Is anyone willing to help out? Is any other group willing to sponsor tzdb (eg. CLDR or Red Hat)?
Please reply to this thread with an indication of support and/or help.
thanks Stephen Colebourne
[1] https://mm.icann.org/pipermail/tz/2021-June/thread.html [2] https://mm.icann.org/pipermail/tz/2021-September/030372.html
[Resending] I think it's a bad idea to fork for a great many reasons, only a few of which are the following: * Time confusion going forward, with new inconsistencies being introduced. * Implementer confusion in terms of which code base is more up-to-date; worse if the code base fragments. * Fragmentation of expertise among volunteers I fear that you drastically underestimate the effort that has been required to maintain both the code and the data. I would suggest an alternative: the policy of this group is set in an RFC, and that RFC can be updated if there is consensus in this community to do so. If the community doesn't like the policy, it can change it, and continue to gain the benefit of one another. Eliot On 20.09.21 10:06, Stephen Colebourne via tz wrote:
Hi all, As most of you probably know, there is a dispute about the tzdb maintainer's recent changes to merge large numbers of time-zones [1][2]. These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
It appears that there is soon to be a new release of tzdb containing these changes. In my opinion, the correct behaviour of the tzdb maintainer would be to revert the controversial changes before doing a release.
In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream).
Is there support for a fork? Is anyone willing to help out? Is any other group willing to sponsor tzdb (eg. CLDR or Red Hat)?
Please reply to this thread with an indication of support and/or help.
thanks Stephen Colebourne
[1]https://mm.icann.org/pipermail/tz/2021-June/thread.html [2]https://mm.icann.org/pipermail/tz/2021-September/030372.html
On Mon, 20 Sept 2021 at 11:48, Eliot Lear via tz <tz@iana.org> wrote:
I think it's a bad idea to fork for a great many reasons, only a few of which are the following:
Time confusion going forward, with new inconsistencies being introduced. Implementer confusion in terms of which code base is more up-to-date; worse if the code base fragments. Fragmentation of expertise among volunteers
I fear that you drastically underestimate the effort that has been required to maintain both the code and the data.
If you read carefully, my original mail proposed forking the data - it did not propose forking the code. I imagine this would be a case where the fork would follow each commit in tzdb, including the code changes, but seeking to maintain the data set as it should be.
I would suggest an alternative: the policy of this group is set in an RFC, and that RFC can be updated if there is consensus in this community to do so. If the community doesn't like the policy, it can change it, and continue to gain the benefit of one another.
Right now there is effectively a schedule gun being held to our heads - if we do not act a new release will come out with the disputed changes in it. The tzdb coordinator still has the option to revert the changes and reset the discussion. Stephen
Stephen Colebourne via tz <tz@iana.org> writes:
On Mon, 20 Sept 2021 at 11:48, Eliot Lear via tz <tz@iana.org> wrote:
I would suggest an alternative: the policy of this group is set in an RFC, and that RFC can be updated if there is consensus in this community to do so. If the community doesn't like the policy, it can change it, and continue to gain the benefit of one another.
Right now there is effectively a schedule gun being held to our heads - if we do not act a new release will come out with the disputed changes in it.
Indeed, and at that point it would pretty much be a fait accompli, because reverting the changes later would create an even huger mess. As far as the RFC (BCP 175) goes, I'll just quote it:
5. Appealing Database Decisions
The TZ Coordinator makes decisions based on expertise, as well as with guidance from the TZ mailing list. If a member of the community has a concern with an individual decision made by the TZ Coordinator with regard to the TZ database, the individual shall proceed as follows:
1. Attempt to resolve the concern directly with the TZ Coordinator.
2. If a resolution cannot be reached directly with the TZ Coordinator, express the concern to the community and attempt to achieve rough consensus regarding a resolution on the TZ mailing list. The Area Directors of the IETF Applications Area may at their discretion attempt to guide the members of the community to rough consensus.
3. As a last resort, if a resolution cannot be reached on the TZ mailing list, appeal to the IESG for a resolution. The appellant must show that the decision made by the TZ Coordinator (a) was materially in error and (b) has caused material harm. In its deliberations regarding an appeal, the IESG shall weigh all the evidence presented to it and use its best judgment in determining a resolution.
Seems to me that we have failed at step 1, and that to the extent that there is any community consensus per step 2, it is against this set of changes. So per process we ought to move on to step 3. (This should have been done months ago, likely, but everyone was hoping for a less unfriendly outcome.) The problem now is that there is no time for any such appeal. We need 2021b now, and if it comes out with these changes, the only way for people to retain stability of the data is to fork. (I suppose for a lot of us, the least painful choice is to sit on 2021a for awhile, ignoring the Apia changes.) regards, tom lane
Tom Lane via tz said:
2. If a resolution cannot be reached directly with the TZ Coordinator, express the concern to the community and attempt to achieve rough consensus regarding a resolution on the TZ mailing list. The Area Directors of the IETF Applications Area may at their discretion attempt to guide the members of the community to rough consensus.
Seems to me that we have failed at step 1, and that to the extent that there is any community consensus per step 2, it is against this set of changes.
I'm not convinced that there is even that much consensus. We're hearing from the objectors, but - as usual - not from those who approve or aren't affected. I'm currently unclear what the issues are, being mostly an observer rather than a significant user of the database. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On Mon, 20 Sep 2021, Clive D.W. Feather via tz wrote:
Tom Lane via tz said:
2. If a resolution cannot be reached directly with the TZ Coordinator, express the concern to the community and attempt to achieve rough consensus regarding a resolution on the TZ mailing list. The Area Directors of the IETF Applications Area may at their discretion attempt to guide the members of the community to rough consensus.
Seems to me that we have failed at step 1, and that to the extent that there is any community consensus per step 2, it is against this set of changes.
I'm not convinced that there is even that much consensus. We're hearing from the objectors, but - as usual - not from those who approve or aren't affected. I'm currently unclear what the issues are, being mostly an observer rather than a significant user of the database.
I think I fall into the class of users which you labelled "aren't affected" although the adverb "directly" might be missing, since to some degree the whole world is afffected! My contribution count to tzdb is - barely - larger than zero, so I'm not a total lurker! And I am a member of the NetBSD developer community, where tzdb is included in the base software distribution. (HOWEVER, I am writing this note as an individual, and not on behalf of NetBSD.) I personally don't have a horse in this race, and either revert or not-revert works for me. If I had to lean in one direction, I'd probably lean towards reverting, and thus returning to something closer to historical conventions. However, if a split/fork were to happen, I'd certainly follow the "original" fork, while perhaps browsing the "divergent" fork now and then just to monitor the situation. Any attempt I'd be inclined to further contribute would be directed at the original fork. Overall, I think a fork of tzdb would be harmful to the community at large, but I really have no other practical alternatives to suggest for resolving the conflict. So, +1 for the approve-or-not-affected category! +--------------------+--------------------------+----------------------+ | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | (Retired) | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com | | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org | | | | pgoyette99@gmail.com | +--------------------+--------------------------+----------------------+
On 20 September 2021 17:22:56 BST, "Clive D.W. Feather via tz" <tz@iana.org> wrote:
Tom Lane via tz said:
2. If a resolution cannot be reached directly with the TZ Coordinator, express the concern to the community and attempt to achieve rough consensus regarding a resolution on the TZ mailing list. The Area Directors of the IETF Applications Area may at their discretion attempt to guide the members of the community to rough consensus.
Seems to me that we have failed at step 1, and that to the extent that there is any community consensus per step 2, it is against this set of changes.
I'm not convinced that there is even that much consensus. We're hearing from the objectors, but - as usual - not from those who approve or aren't affected. I'm currently unclear what the issues are, being mostly an observer rather than a significant user of the database.
Who asked for historical information to be unavailable from the default tzdb distribution? cheers, Derick
On 9/20/21 10:18 AM, Derick Rethans via tz wrote:
Who asked for historical information to be unavailable from the default tzdb distribution?
In June, Stephen Colebourne suggested that as an acceptable solution: https://mm.icann.org/pipermail/tz/2021-June/030273.html and yesterday wrote that he'd still have more respect for that approach than for what's currently in the development database: https://mm.icann.org/pipermail/tz/2021-September/030421.html However, this would be a bigger change to tzdb than other proposed suggestions. Although it wouldn't affect all that many uses (none of the proposed changes do) I expect it would be even less popular than the other proposals.
If we're voting - I'd vote against forking. And - once again - I have to point out that it's well past time we disassociate zone data from political regions. Give each zone a unique numeric id then provide a default mapping. The tools can regenerate something that looks like the old form. On 9/20/21 12:22, Clive D.W. Feather via tz wrote:
Tom Lane via tz said:
2. If a resolution cannot be reached directly with the TZ Coordinator, express the concern to the community and attempt to achieve rough consensus regarding a resolution on the TZ mailing list. The Area Directors of the IETF Applications Area may at their discretion attempt to guide the members of the community to rough consensus.
Seems to me that we have failed at step 1, and that to the extent that there is any community consensus per step 2, it is against this set of changes. I'm not convinced that there is even that much consensus. We're hearing from the objectors, but - as usual - not from those who approve or aren't affected. I'm currently unclear what the issues are, being mostly an observer rather than a significant user of the database.
On Mon, Sep 20, 2021 at 8:52 AM Stephen Colebourne via tz <tz@iana.org> wrote:
On Mon, 20 Sept 2021 at 11:48, Eliot Lear via tz <tz@iana.org> wrote:
I think it's a bad idea to fork for a great many reasons, only a few of which are the following:
Time confusion going forward, with new inconsistencies being introduced. Implementer confusion in terms of which code base is more up-to-date; worse if the code base fragments. Fragmentation of expertise among volunteers
I fear that you drastically underestimate the effort that has been required to maintain both the code and the data.
If you read carefully, my original mail proposed forking the data - it did not propose forking the code. I imagine this would be a case where the fork would follow each commit in tzdb, including the code changes, but seeking to maintain the data set as it should be.
Speaking as a participant only: I agree with Eliot. Though I'm a relative newcomer to this particular space, I have never seen a successful fork of the nature you're describing. You make it sound simple here, but I would bank on the inevitability of some change being introduced into the original data that cannot be trivially merged into the fork. I wouldn't want to be the person responsible for sorting that out while simultaneously tracking all of the other issues the current coordinator handles on a regular basis. -MSK
I want to thank tzdb for all the work that has been done providing recent timezone data for those that need this central repository.I would like to join with others in forking the db so we can start and maintain a new tz db that also becomes a primary db for historical tz data.Archeologists do not wait until they have every detail perfect and complete about the fossil of a dinosaur before publishing the discovery of a new bone. Historians do not wait until they understand the complete civilization before publishing new evidence of a new society. Historical information can never be complete, but a standard and repository for historical data must be available for the collection of this type of data as a focal point for research in this type of data.I'm in, and can bring 45 years of professional software development experience, including 30 years of C, 25 of c++, and 10 of javascript, and 5 of nodejs.DavidDavidSent from my Galaxy -------- Original message --------From: Stephen Colebourne via tz <tz@iana.org> Date: 2021-09-20 04:07 (GMT-05:00) To: Time Zone Mailing List <tz@iana.org> Subject: [tz] Preparing to fork tzdb Hi all,As most of you probably know, there is a dispute about the tzdbmaintainer's recent changes to merge large numbers of time-zones[1][2]. These have the effect of wiping out historic time-zoneinformation on many locations where the data has been in tzdb for manyyears.It appears that there is soon to be a new release of tzdb containingthese changes. In my opinion, the correct behaviour of the tzdbmaintainer would be to revert the controversial changes before doing arelease.In the event that the tzdb maintainer does not revert, considerationmust be given to forking the project. The purpose of the fork wouldinitially be to maintain the tzdb data set as it was prior to thedispute. This would then be released in parallel to the original tzdbto ensure that downstream projects do not each do their own thing (ie.to minimize incompatibilities downstream).Is there support for a fork?Is anyone willing to help out?Is any other group willing to sponsor tzdb (eg. CLDR or Red Hat)?Please reply to this thread with an indication of support and/or help.thanksStephen Colebourne[1] https://mm.icann.org/pipermail/tz/2021-June/thread.html[2] https://mm.icann.org/pipermail/tz/2021-September/030372.html
Stephen Colebourne via tz <tz@iana.org> writes:
In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream).
It pains me to say it, but since there apparently is no willingness to revert that decison, a fork is going to be necessary. I support this, and it is likely that the Postgres project would follow the fork. regards, tom lane
On 20 September 2021 14:34:27 BST, Tom Lane via tz <tz@iana.org> wrote:
Stephen Colebourne via tz <tz@iana.org> writes:
In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream).
It pains me to say it, but since there apparently is no willingness to revert that decison, a fork is going to be necessary. I support this, and it is likely that the Postgres project would follow the fork.
So will the PHP project. cheers Derick
On 9/20/21 6:52 AM, Derick Rethans via tz wrote:
So will the PHP project.
With the patches I emailed in <https://mm.icann.org/pipermail/tz/2021-September/030456.html>, the PHP project should be able to avoid the effects of the alike-since-1970 patches without our having to fork tzdb, by using something like the attached (untested) patch to PHP's timelib. However before taking such an approach, please read the comments of the bias2021a.bp file, which suggest that you'll need to adjust that file. For more, please see my other recent email <https://mm.icann.org/pipermail/tz/2021-September/030458.html>.
Hello all, Might I point out that there is another option available — section 4 of RFC 6557 (Procedures for Maintaining the Time Zone Database) [1] provides a mechanism for replacing the TZ coordinator in the case that "The TZ Coordinator is not performing the function in accordance with community wishes." I personally am ambivalent about the changes in question, and overall have very little stake in this, but I would hate to see such drastic action as a fork taken before all other avenues have been exhausted. Best, Emily Crandall Fleischman [1] https://www.rfc-editor.org/rfc/rfc6557.html#section-4 On Mon, 20 Sept 2021 at 04:07, Stephen Colebourne via tz <tz@iana.org> wrote:
Hi all, As most of you probably know, there is a dispute about the tzdb maintainer's recent changes to merge large numbers of time-zones [1][2]. These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
It appears that there is soon to be a new release of tzdb containing these changes. In my opinion, the correct behaviour of the tzdb maintainer would be to revert the controversial changes before doing a release.
In the event that the tzdb maintainer does not revert, consideration must be given to forking the project. The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute. This would then be released in parallel to the original tzdb to ensure that downstream projects do not each do their own thing (ie. to minimize incompatibilities downstream).
Is there support for a fork? Is anyone willing to help out? Is any other group willing to sponsor tzdb (eg. CLDR or Red Hat)?
Please reply to this thread with an indication of support and/or help.
thanks Stephen Colebourne
[1] https://mm.icann.org/pipermail/tz/2021-June/thread.html [2] https://mm.icann.org/pipermail/tz/2021-September/030372.html
On Sep 20, 2021, at 11:06, Emily Crandall Fleischman via tz <tz@iana.org> wrote:
I personally am ambivalent about the changes in question, and overall have very little stake in this, but I would hate to see such drastic action as a fork taken before all other avenues have been exhausted.
++ Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | A room without books is like a body without a soul. | | | | -- Cicero | |---------------------------------------------------------------------|
On 9/20/21 1:06 AM, Stephen Colebourne via tz wrote:
The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute.
Such a fork would arbitrarily discriminate against countries like Angola and Niger, and in favor of countries like Norway and Sweden. A primary goal of the recent patches was to avoid racial or national preferences that were present in the previous setup. Arguably these preferences were not intentional, or were apparent and not real; however, that's not an argument I would want to defend. Although the problem of discrimination could also be fixed in the fork you suggest, any such fix would take considerable work. It couldn't be done merely by reverting a patch or two. All this work would be need to be done before any such fork could be used by an organization that is committed to equity, diversity and inclusion. Instead of forking, I suggest that we work together to address the technical issues raised by the change. I've been attempting to do that with the recent "Revert May patch to zone.tab" patch, which Almaz today said fixes the problems Google had with region-timezone mapping. I hope that we can address remaining technical problems with similar patches. If the "Revert May patch to zone.tab" patch is not enough, one possible path forward would be to add a build-time option along the lines that I suggested to Tom Lane in <https://mm.icann.org/pipermail/tz/2021-June/030197.html>. This would let projects like PostgreSQL generate a tzdata.zi file containing whatever backzone entries they choose, though I would urge these projects to consider equity, diversity and inclusion issues when they make their choices. I have drafted a patch along these lines and could circulate it if there's interest. The idea is to commit to an approach that accommodates users who need at least one tzdb entry per country, either via the "Revert May patch to zone.tab" or by further patches if necessary.
These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
None of the information has been "wiped out". It has merely moved from one source file to another, and builds with PACKRATDATA=backzone still install all the info in question. This disagreement is not about "wiping out" info; it's about what category the info falls into.
Paul Eggert via tz <tz@iana.org> writes:
On 9/20/21 1:06 AM, Stephen Colebourne via tz wrote:
These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
None of the information has been "wiped out". It has merely moved from one source file to another, and builds with PACKRATDATA=backzone still install all the info in question. This disagreement is not about "wiping out" info; it's about what category the info falls into.
The concern I'm having is that according to experiments I did back in June, the above is not true. As far as I can tell, adding PACKRATDATA=backzone does *not* reproduce what was formerly the default set of zones. I'm all for improving equity in tzdb's coverage, but I think it should be done by adding coverage for underserved areas, not removing data from areas that had been well-covered. And let's make no mistake: removing data from the default build is removing data, for many downstream users who won't have an opportunity to make their own decisions about what their platforms provide. regards, tom lane
On 9/20/21 10:10 AM, Tom Lane wrote:
As far as I can tell, adding PACKRATDATA=backzone does *not* reproduce what was formerly the default set of zones.
It sounds like you formerly did not use PACKRATDATA=backzone and now started using it. That should yield quite a few changes, because it'll use all the 'backzone' data instead of just the data that migrated to 'backzone' recently. This doesn't mean data were lost; on the contrary, it means you're getting more data than before (some of it low-quality and all of it out of tzdb's post-1970 scope). If it's important for PostgreSQL to get data that exactly matches 2021a (except for changes you don't object to), I suggested in <https://mm.icann.org/pipermail/tz/2021-June/030197.html> adding a build-time option to let projects like PostgreSQL choose whatever 'backzone' data they want, instead of 'backzone' being an all-or-nothing decision as it is now. This would let these projects avoid the objected-to changes, while keeping the changes for Samoa etc. that are not objected to, and while at the same time avoiding obsolete or out-of-scope 'backzone' data that they don't want. Of course I would urge these projects to consider equity, diversity and inclusion issues when choosing from 'backzone'; but the choices would be up to them. If you're interested in this approach, I think I could develop it quickly, before any new tzdb release. If not, then let's try to think of a better way to address the issue.
I'm all for improving equity in tzdb's coverage, but I think it should be done by adding coverage for underserved areas
Adding coverage could be part of the laborious process I mentioned in <https://mm.icann.org/pipermail/tz/2021-September/030413.html>. This process would helpful for tzdb, and it would be needed in the proposed fork if the fork wants to satisfy equity, diversity, and inclusion concerns. However, I doubt whether this effort could be done well anytime soon, regardless of whether tzdb is forked.
On 9/20/21 11:04 AM, Paul Eggert via tz wrote:
If it's important for PostgreSQL to get data that exactly matches 2021a (except for changes you don't object to), I suggested in <https://mm.icann.org/pipermail/tz/2021-June/030197.html> adding a build-time option to let projects like PostgreSQL choose whatever 'backzone' data they want, instead of 'backzone' being an all-or-nothing decision as it is now. This would let these projects avoid the objected-to changes
Attached is a proposed implementation of this suggestion. The idea is to make it easy for projects like PostgreSQL to act as if the recent alike-since-1970 changes had not occurred, without having to fork tzdb. To try this out in PostgreSQL, you can copy the attached file tzdata.zi to postgres/src/timezone/data/tzdata.zi. This tzdata.zi file contains the other post-2021a changes; just not the alike-since-1970 changes. To generate this tzdata.zi file automatically from the tz main branch, you can do the following: 1. Apply the attached patches to tzdb's current main branch. 2. Save the attached file bias2021a.bp. 3. Run the following shell command: make BACKPICK=bias2021a.bp PACKRATDATA=backzone tzdata.zi The file bias2021a.bp is a "backpick" file that says which Zones and Links should be picked out from the file 'backzone', overriding the other data. bias2021a.bp's comments contain more description. As I hope the comments make clear, this bias2021a.bp file is intended only as a demonstration; the PostgreSQL developers can and should tailor it to fit their needs and project goals. My idea is to apply the attached patches to the tzdb main development branch, with the PostgreSQL developer choosing whatever .bp file they like. Comments welcome.
On Mon, 20 Sept 2021 at 17:16, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 9/20/21 1:06 AM, Stephen Colebourne via tz wrote:
The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute.
Such a fork would arbitrarily discriminate against countries like Angola and Niger, and in favor of countries like Norway and Sweden.
The approach you argue for arbitrarily discriminates too, saying that Germany is more important than Norway or Sweden. I would have more respect for your position if you had also proposed wiping out all pre-1970 data from tzdb, but the current state of tzdb is not only offensive but nonsensical. I also believe that discriminating against countries like Angola and Niger is unacceptable. But the solution to that is the same as it always has been - to include full data for each ISO country, using best efforts for data that is not certain.
Instead of forking, I suggest that we work together to address the technical issues raised by the change.
The *only* good faith move you can make right now is to revert the patch. I'm quite happy to discuss practical solutions once that is done. If 2021b is released with the disputed patch then the fork will occur, and you as TZ coordinator will have directly caused the fork.
one possible path forward would be to add a build-time option along the lines that I suggested to Tom Lane in <https://mm.icann.org/pipermail/tz/2021-June/030197.html>. This would let projects like PostgreSQL generate a tzdata.zi file...
As previously discussed, it is not acceptable to expect all downstream projects to change their build systems. Especially with zero notice. Stephen
On 9/20/21 11:03 AM, Stephen Colebourne via tz wrote:
I also believe that discriminating against countries like Angola and Niger is unacceptable. But the solution to that is the same as it always has been - to include full data for each ISO country, using best efforts for data that is not certain.
I addressed this issue toward the end of my recent email to Tom Lane <https://mm.icann.org/pipermail/tz/2021-September/030422.html>, which I sent about the same time you sent your email.
The *only* good faith move you can make right now is to revert the patch.
We disagree on this point. This morning I made a different good-faith move, by installing the "Revert May patch to zone.tab" patch into the development database. I suggested another potential good-faith move in that recent email to Tom Lane. I continue to think that it would be better to work together to find a common solution, than to insist on one unalterable position.
The approach you argue for arbitrarily discriminates too, saying that Germany is more important than Norway or Sweden.
The guideline is not arbitrary; it chooses the most-populous city, in this case Berlin. Although every guideline has a bias of some sort, it would be a quite a stretch to say that this particular guideline has racial, ethnic or national bias, which is the concern here.
On 20 September 2021 19:27:27 BST, Paul Eggert via tz <tz@iana.org> wrote:
On 9/20/21 11:03 AM, Stephen Colebourne via tz wrote:
I also believe that discriminating against countries like Angola and Niger is unacceptable. But the solution to that is the same as it always has been - to include full data for each ISO country, using best efforts for data that is not certain.
I addressed this issue toward the end of my recent email to Tom Lane <https://mm.icann.org/pipermail/tz/2021-September/030422.html>, which I sent about the same time you sent your email.
The *only* good faith move you can make right now is to revert the patch.
We disagree on this point. This morning I made a different good-faith move, by installing the "Revert May patch to zone.tab" patch into the development database. I suggested another potential good-faith move in that recent email to Tom Lane.
I continue to think that it would be better to work together to find a common solution, than to insist on one unalterable position.
I'm with Stephen here. I don't think that unless the offending patch is reversed there can be an agreeable way forward. We can then take it from there.
The approach you argue for arbitrarily discriminates too, saying that Germany is more important than Norway or Sweden.
The guideline is not arbitrary; it chooses the most-populous city, in this case Berlin. Although every guideline has a bias of some sort, it would be a quite a stretch to say that this particular guideline has racial, ethnic or national bias, which is the concern here.
You might not see that, but developers and users in Norway or Sweden definitely will see this as being a snipe. Stephen tried many months ago to come up with a set of statements and suggestions to come up with a reasonable policy on what to do, and although there were some issues with that, there was at least an attempt to codify a way forwards. These suggestions have been met by silence from the coordinator. I've also not seen any response to the formal complains that were sent to IANA. I don't like the idea of a fork, mostly because it serves nobody to have two competing versions of tzdb, but right now I can't see another way out... cheers Derick
Quoting Derick Rethans via tz on Monday September 20, 2021:
I've also not seen any response to the formal complains that were sent to IANA.
To clarify on this, I am not aware of any pending complaints that have been sent to IANA. We host this community project in accordance with RFC 6557, which means that the TZ coordinators are appointed by the IESG, and also the appeals process is operated by them <https://www.ietf.org/about/groups/iesg/>. I understand there is an appeal pending that is being considered by the IESG. <https://mm.icann.org/pipermail/tz/2021-June/030308.html> Importantly, this is a community project and the intention is for consensus to be built in this very forum for changes to be made, and IANA does not play a role in editorial decisions concerning timezones.
I don't like the idea of a fork, mostly because it serves nobody to have two competing versions of tzdb, but right now I can't see another way out...
I agree that forking this work leaves everyone the poorer, I hope a resolution can be found that is satisfactory to everyone. kim
On 20.09.21 20:53, Derick Rethans via tz wrote:
You might not see that, but developers and users in Norway or Sweden definitely will see this as being a snipe.
You have but a compiler flag to add, but instead you want to fork? Ok. That'll be fun when I get one time from the OS and one time from PHP when the update policies diverge – and you can bet they will.
To me this looks like a fork whether another repository is created or not. With multiple down stream database parsers (as is the case for C++ vendors, and other languages as well), each vendor is going to make a choice as to include backzone or not. Today the choice is obvious: don’t include, because: * there’s not much in it. * the comments in the file discourage its use. * It has different parsing rules than africa, asia, europe, etc. By putting more data into backzone, this greatly increases the odds that some vendors are going to adopt backzone, and other’s won’t. That’s a fork. ************** I would instead like to see a move in the opposite direction: Consolidation to a single accepted version of the data, even if flawed, that slowly gets better as time goes on. Howard On Sep 21, 2021, at 1:03 AM, Eliot Lear via tz <tz@iana.org> wrote:
You have but a compiler flag to add, but instead you want to fork?
Ok. That'll be fun when I get one time from the OS and one time from PHP when the update policies diverge – and you can bet they will.
Howard Hinnant via tz <tz@iana.org> writes:
By putting more data into backzone, this greatly increases the odds that some vendors are going to adopt backzone, and other’s won’t. That’s a fork. **************
Well, no. The objections to a fork in this thread are based on (very reasonable) concerns about duplicated maintenance effort, lack of one source of truth, and so on. This would not be that. However, you have a very good point that if any significant number of vendors start including backzone to restore some approximation of the way things stood before, then there is going to be the same mess from end users' standpoint as a true fork would produce. There will be more than one "canonical" version of tzdb, and that's going to be barely distinguishable from a true fork in terms of confusion on the ground.
I would instead like to see a move in the opposite direction: Consolidation to a single accepted version of the data, even if flawed, that slowly gets better as time goes on.
+1 regards, tom lane
On 9/21/21 7:08 AM, Tom Lane via tz wrote:
if any significant number of vendors start including backzone to restore some approximation of the way things stood before, then there is going to be the same mess from end users' standpoint as a true fork would produce.
True, and a good reason to not make use of the draft patches that I emailed earlier today in <https://mm.icann.org/pipermail/tz/2021-September/030456.html>. There's another good reason to not use that approach, noted in the bias2021a.bp file in that email: # This file should be not be used in production by organizations # committed to equity, diversity, and inclusion because it restores # the previous tzdb setup, which arguably exhibited racial or national # preferences. Both of these reasons argue against using the draft patches, and argue with equal force against using the proposed fork. We'd be better off avoiding either approach. That being said, if the only alternatives are the proposed fork or the draft patches, the patches should be better as they should make it easier to keep the two approaches coherent. Perhaps the fork could even be implemented by a different repository which has a one-line Makefile change (I'm just thinking out loud here).
Paul Eggert <eggert@cs.ucla.edu> writes:
On 9/21/21 7:08 AM, Tom Lane via tz wrote:
if any significant number of vendors start including backzone to restore some approximation of the way things stood before, then there is going to be the same mess from end users' standpoint as a true fork would produce.
True, and a good reason to not make use of the draft patches that I emailed earlier today in <https://mm.icann.org/pipermail/tz/2021-September/030456.html>.
Yeah, I was about to reply to that along similar lines: I think we should put some value on most people using the same version of tzdb, so adding options to pick-and-choose subsets of the data is arguably counterproductive.
There's another good reason to not use that approach, noted in the bias2021a.bp file in that email:
# This file should be not be used in production by organizations # committed to equity, diversity, and inclusion because it restores # the previous tzdb setup, which arguably exhibited racial or national # preferences.
TBH, I find that argument to have just about zero merit. It's not like the previous state of affairs was deliberately non-diverse, nor do I believe that the May patches magically removed all potential complaints of that sort. I think the way to proceed here is what has been suggested by several people including me: revert to the pre-May set of zone data, and make a commitment that we'll accept properly-researched patches to add per-country zones over time. (This need not be a commitment that the TZ Coordinator would do such research.) I think the reason that e.g. Norway has its own zone is that somebody did the research to add it once upon a time. Why should we devalue that effort now? regards, tom lane
On 9/21/21 11:40 AM, Tom Lane wrote:
It's not like the previous state of affairs was deliberately non-diverse
It'd be hard for me to defend that assertion, unfortunately. And I am in a position where I could well have to defend it, as I am UCLA teaching professor who has institutional obligations in the areas of equity, diversity and inclusion. I think other organizations who have similar obligations should also be aware of these issues.
I think the reason that e.g. Norway has its own zone is that somebody did the research to add it once upon a time.
Yes. That "somebody" was me.
Why should we devalue that effort now?
It's not being devalued. I certainly don't feel any devaluement (is that a word? :-) The data are just being put in their proper place.
On 9/20/21 11:53 AM, Derick Rethans wrote:
Stephen tried many months ago to come up with a set of statements and suggestions to come up with a reasonable policy on what to do, and although there were some issues with that, there was at least an attempt to codify a way forwards.
Although there were definitely issues, we could reopen that discussion. Eliot Lear also suggested something along those lines. As I recall quite a few words were exchanged but the details are fuzzy in my mind. To help narrow things down, it'd be helpful to propose a specific patch to theory.html. (theory.html is the file that contains the guidelines in question.) You'd surely need such a patch anyway if you fork. And now's a good time to write the patch, if only to justify why a fork is necessary. However, I hope the exercise of writing the patch can help us find a way to avoid the fork.
On Mon, 20 Sept 2021 at 19:27, Paul Eggert via tz <tz@iana.org> wrote:
On 9/20/21 11:03 AM, Stephen Colebourne via tz wrote:
I also believe that discriminating against countries like Angola and Niger is unacceptable. But the solution to that is the same as it always has been - to include full data for each ISO country, using best efforts for data that is not certain.
I addressed this issue toward the end of my recent email to Tom Lane <https://mm.icann.org/pipermail/tz/2021-September/030422.html>, which I sent about the same time you sent your email.
The argument you make is that fixing tzdb to have better historic data in an equitable way is a big task, which we agree on. But where we disagree is the need to nuke what we have now as a first step. The right approach would be to start from the 2021a data and actively add in the missing pieces to reach the equitable goal. As I've said before, there are only two equitable solutions for the default tzdb files (ie minus backzone). Either they contain full history for every ISO country (even if that history is inaccurate), or they contain no pre-1970 data at all. It is completely inequitable to allow only some locations to have pre-1970 data (and to the average user, the choice *looks* very arbitrary even if it isn't).
The *only* good faith move you can make right now is to revert the patch.
We disagree on this point. This morning I made a different good-faith move, by installing the "Revert May patch to zone.tab" patch into the development database. I suggested another potential good-faith move in that recent email to Tom Lane.
Expecting downstream projects to change their build systems with no notice to solve an artificially created crisis isn't good project management. It isn;t appropriate behaviour for the role of TZ coordinator.
I continue to think that it would be better to work together to find a common solution, than to insist on one unalterable position.
I have absolutely no desire to support a fork. I'm being forced into it by your unwillingness to listen to this group's feedback and perform the reversion. All these additional partial reversions have actually had the effect of making the issue harder to resolve, not easier. I have also laid out a possible future position for tzdb [1]. But to move forward, the data set needs to be reset back to 2021a. The question you as TZ coordinator should consider is which route has the best outcome? - revert now, then discuss and agree a long-term solution that solves the equitability issue - or, do not revert, and trigger a painful fork, with different downstream projects choosing one or the other I have always wanted the first option - forking is not a good outcome. But the choice is yours to make, not mine. Stephen
On 2021-09-20 20:56, Stephen Colebourne via tz wrote:
I have always wanted the first option - forking is not a good outcome.
I propose not to fork tzdb and instead revoke the proposed changes until we can agree on a *complete* picture of how the changes can be effected without inducing undue burdens on any downstream user. Fixing singular aspects of the proposed change does not seem to suffice in this situation. We know one possible solution, proposed by Stephen Colebourne (but others might be worth considering as well): ∙ the main line data of tzdb should contain all the historical data we have, and SHOULD be maintained as such; ∙ there are options to reduce the amount of historical data by merging timezones with links and/or by stripping old transitions from timezone data; such options MAY be added if necessary; ∙ using zic with the same options on successive versions of tzdb data SHOULD not lead to major changes in the scope and structure of the TZif results; ∙ similarly, for those using the tzdb data directly, the organization of tzdb data into files SHOULD not change significantly across successive versions. For instance, the proposed merge of timezones agreeing since 1970 can be done quite simply with a new file with links (such as merge1970) without affecting users who cannot use the merge. Michael Deckers.
On Sep 20, 2021, at 17:20, Michael H Deckers via tz <tz@iana.org> wrote:
I propose not to fork tzdb and instead revoke the proposed changes until we can agree on a *complete* picture of how the changes can be effected without inducing undue burdens on any downstream user. Fixing singular aspects of the proposed change does not seem to suffice in this situation.
We know one possible solution, proposed by Stephen Colebourne (but others might be worth considering as well):
∙ the main line data of tzdb should contain all the historical data we have, and SHOULD be maintained as such;
∙ there are options to reduce the amount of historical data by merging timezones with links and/or by stripping old transitions from timezone data; such options MAY be added if necessary;
∙ using zic with the same options on successive versions of tzdb data SHOULD not lead to major changes in the scope and structure of the TZif results;
∙ similarly, for those using the tzdb data directly, the organization of tzdb data into files SHOULD not change significantly across successive versions.
To attempt to summarize succinctly: for tzdb users, changes in data policy should be “opt-in”, not “opt-out”. This seems to me to be well in keeping with the long-standing "principle of least astonishment”. Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | A room without books is like a body without a soul. | | | | -- Cicero | |---------------------------------------------------------------------|
On 9/20/21 1:56 PM, Stephen Colebourne via tz wrote:
where we disagree is the need to nuke what we have now as a first step.
Again, nothing has been nuked. Some data have merely moved from one place to another.
As I've said before, there are only two equitable solutions for the default tzdb files (ie minus backzone). Either they contain full history for every ISO country (even if that history is inaccurate), or they contain no pre-1970 data at all.
There's no need to involve ISO countries, as far as equity is concerned. Suppose someone argued that there should be a full timezone history for every state and province and city in the world, and that otherwise there should be no pre-1970 data at all. Such an argument would be equally invalid, even though these smaller entities were (and in many cases still are) the governmental agencies responsible for setting civil-time rules. For proper timekeeping we needn't model every governmental agency involved in timekeeping; all we need to do is model the timekeeping. Requiring involvement of countries complicates the TZ database for political reasons, not for timekeeping reasons. As such, it complicates maintenance, mostly due to political hassles. We do better by avoiding political entanglements when possible.
Expecting downstream projects to change their build systems with no notice to solve an artificially created crisis isn't good project management.
Yes, it would be better if we came to a consensus and did not have to fork the repository or fork the project in some other way. Any such fork will require some sort of build-system changes for those who doesn't take the default approach. As I've tried to make clear in my recent emails, although the proposed fork is technically feasible (and doesn't even need to rely on a copied repository), it's not a good idea. This is not merely because of the hassle of configuring after a fork; it's because merely going back to 2021a's setup is not something we can or should do, on equity grounds.
Date: Mon, 20 Sep 2021 09:15:58 -0700 From: Paul Eggert via tz <tz@iana.org> Message-ID: <190b4056-9ed0-f876-c32a-acb13185d114@cs.ucla.edu> | Such a fork would arbitrarily discriminate against countries like Angola | and Niger, and in favor of countries like Norway and Sweden. | | A primary goal of the recent patches was to avoid racial or national | preferences that were present in the previous setup. Arguably these | preferences were not intentional, or were apparent and not real; | however, that's not an argument I would want to defend. Whatever the underlying merits of all of this, that argument is pure BS. If the project were actively preventing some data from being included for any spurious reasons, there would be a problem - but as best I can tell that has never happened. Further, this is a volunteer (ie: zero resources) project - we don't have the ability (in general) to go find missing data, no matter why it is missing. We rely upon someone being interested, and sending us the data. If they do that, and we have no reason to believe it is bogus data, we install it - it doesn't need to be perfect, just someone's honest belief that some region had some particular time behaviour sometime in the past, ideally supported by some kind of research. If more accurate data comes to light later, we improve things. If someone in (or from) Angola or Niger wants th research their timezone history, and send it, it should be included, just like that for the US, France, or Norway (etc). There is no discrimination here, and there never has been, and (aside from perhaps sometimes on what the name of a zone should be) I've never even heard of a suggestion that there might be. That argument is simply absurd. Personally, I've always believed that there should be at least one zone for every time zone authority on the planet - it simply makes things easier for everyone. For the users as once they have picked the zone that applies to them, they're unlikely to need a change, and when that might happen, the event will be very well known inside the zone (areas of a region that used to share the same timezone being split between two times). For the project, it means that we don't need to keep creating new zones every time some authority decides to alter things in a way that makes them different than other nearby regions - it really would not take much time at all for authorities to realise that if they were decide to more the clocks forward for 5 seconds of summer time between 03:00 and 04:00 (local time) some Sunday in mid winter, they'd then be entitled to their own private timezone, according to our rules. This isn't something that we ought to be encouraging. The only real cost is a little extra processing time generating the zones, and a trivial amount of zone data files. kre
On 9/20/21 2:58 PM, Robert Elz wrote:
If someone in (or from) Angola or Niger wants th research their timezone history, and send it, it should be included, just like that for the US,
Yes, and there's a place for that in 'backzone'. We're not refusing timezone history because it's unnecessary due to the alike-since-1970 rule. It's put into 'backzone' and is welcome there. Perhaps some people have the misimpression that we have many potential contributors whose data are being refused. On the contrary, almost nobody contributes historical data, because it's boring unpaid work and honestly, it's not that important. I know that, because I contributed the vast majority of 'backzone' (a contribution that I'm coming more and more to regret :-). This disagreement is not about whether the data in question are available; it's only about which file they're in. Nothing is being "wiped out" or refused.
I've always believed that there should be at least one zone for every time zone authority on the planet
That would be ideal, and I hope something like that would be doable for current and future timestamps. But it'd be a stretch to try to do it for all timestamps since 1970, and it'd be completely impractical for pre-1970 timestamps. We don't know who the historical time zone authorities were, much less what rules they used.
The only real cost is a little extra processing time generating the zones, and a trivial amount of zone data files.
I think this greatly underestimate the effort to do a complete job. We'd eventually need thousands of Zones. I expect we'd have to revamp the underlying technology too, and do something more like what Shanks evidently did (though I haven't seen his source code). I would not consider that trivial, and I think our collective effort would be better spent elsewhere.
On Tue, 21 Sept 2021 at 17:48, Paul Eggert via tz <tz@iana.org> wrote:
This disagreement is not about whether the data in question are available; it's only about which file they're in. Nothing is being "wiped out" or refused.
It is about more than that. The current unreleased state of tzdb is that it favours some countries over others, such as Germany over Sweden/Norway. It does this to a much greater extent than before. From my perspective, that is the root of the problem here.
I think this greatly underestimate the effort to do a complete job. We'd eventually need thousands of Zones.
I would imagine that most readers of this list would not want thousands of zones. The goal of including every time-zone that has ever been is a straw man. I'm perfectly happy with the post-1970 rule for determining what IDs we have so long as it results in one ID per ISO country. In addition, at least one ID per ISO country is entitled to have full history back to LMT, **even if it is innaccurate**.
[I] am UCLA teaching professor who has institutional obligations in the areas of equity, diversity and inclusion ... merely going back to 2021a's setup is not something we can or should do, on equity grounds.
The patch makes the equity/diversity/inclusion position of tzdb far, far worse. Appreciating this is the first step necessary to resolving this. Stephen
On 9/21/21 12:33 PM, Stephen Colebourne via tz wrote:
The current unreleased state of tzdb is that it favours some countries over others, such as Germany over Sweden/Norway.
It also "favors" some states over others, such as Arizona (which has the America/Phoenix Zone) over Utah (which has no Zone). It's the same argument, and it's an equally invalid argument as far as timekeeping goes. That sort of argument would be valid only if one accepted as an axiom that countries (or states) are an inherent part of timekeeping. But we don't need to take that as an axiom, and tzdb has done well to avoid nation-state political entanglements as much as possible.
<<On Tue, 21 Sep 2021 12:44:22 -0700, Paul Eggert via tz <tz@iana.org> said:
That sort of argument would be valid only if one accepted as an axiom that countries (or states) are an inherent part of timekeeping.
Governments self-evidently are: they are why we even have time zones in the first place. Everything in this database is _inherently_ political, and your continued attempts to "avoid politics" do nothing of the sort. -GAWollman
On 9/21/21 12:47 PM, Garrett Wollman wrote:
That sort of argument would be valid only if one accepted as an axiom that countries (or states) are an inherent part of timekeeping. Governments self-evidently are: they are why we even have time zones in the first place.
Of course governments determine civil-time rules, and these rules determine what goes into tzdb. But that does not mean that tzdb must include Zones for Utah, Idaho, Montana etc. merely because Arizona has an Zone. All it means is that we need to record governmental rules accurately and concisely. tzdb will continue to be easier to maintain and use if it focuses on timekeeping, not politics.
On Tue, 21 Sept 2021 at 20:54, Paul Eggert via tz <tz@iana.org> wrote:
On 9/21/21 12:47 PM, Garrett Wollman wrote:
That sort of argument would be valid only if one accepted as an axiom that countries (or states) are an inherent part of timekeeping. Governments self-evidently are: they are why we even have time zones in the first place.
Of course governments determine civil-time rules, and these rules determine what goes into tzdb. But that does not mean that tzdb must include Zones for Utah, Idaho, Montana etc. merely because Arizona has an Zone.
Straw man. Literally no one has suggested this. If you don't like ISO countries as the basis of a rule, feel free to suggest something else. If you want to resolve the issue you need a rule that doesn't end up with users in Norway or Sweden requesting the time-zone for 1950 and getting the zone information of Germany instead. It is this aspect of the patch that is so utterly repulsive that months of wasted energy has been spent to get it reverted. IMO the defining characteristic of what is or is not acceptable to merge zones across is ISO country boundaries. But I'd love to hear an alternate idea. Stephen
On 9/21/21 1:20 PM, Stephen Colebourne via tz wrote:
Straw man. Literally no one has suggested this.
Yes, of course it's a straw argument. That's the point. Although it'd be technically possible to require a Zone for each state and province in the world, it's not necessary, it would consume maintenance resources that are better spent elsewhere, and Zone proliferation would make things unnecessarily complicated for users. The situation for countries is similar. Admittedly there are few countries than states+provinces, but logically there's no difference between the two cases.
If you don't like ISO countries as the basis of a rule, feel free to suggest something else.
My suggestion is to stick with the rules already in theory.html. They've worked well over the years.
If you want to resolve the issue you need a rule that doesn't end up with users in Norway or Sweden requesting the time-zone for 1950 and getting the zone information of Germany instead.
If that's the idea, then I also need a rule that doesn't end up with users in Munich requesting the time zone for September 20, 1945 and getting the UTC offset of Berlin instead. The case for a Munich Zone is even stronger than the case for an Oslo Zone, as Munich has a lot more people (and more tzdb users) than Oslo does. Of course this is a straw argument too; that's the point. All this stuff about pre-1970 timestamps is out of scope for tzdb proper, and it's always been out of scope. Only politics would require us to have a Zone for every ISO country. Timekeeping needs don't require it. Admittedly we used to have more Zones, but we've never had a Zone for every ISO country and no timekeeping need for such a rule has ever been demonstrated.
On Tue, 21 Sept 2021 at 22:13, Paul Eggert <eggert@cs.ucla.edu> wrote:
Admittedly there are few countries than states+provinces, but logically there's no difference between the two cases.
Of course there is. A 10 year old could explain the difference between a state and a country.
My suggestion is to stick with the rules already in theory.html. They've worked well over the years.
What worked well was when the data was not constantly fiddled with. I don't believe this group ever asked for the tidy ups it has been subjected to over the past few years. All you had to do as TZ Coordinator was keep the data up to date with new info as it arrived.
The case for a Munich Zone is even stronger than the case for an Oslo Zone, as Munich has a lot more people (and more tzdb users) than Oslo does.
I despair at this. It indicates absolutely no comprehension of the fundamental issue here. Stephen
Stephen Colebourne via tz said:
Admittedly there are few countries than states+provinces, but logically there's no difference between the two cases.
Of course there is. A 10 year old could explain the difference between a state and a country.
Go ahead, then. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Perhaps current timekeeping does need it, but historical time based research does, and that is our point.Sent from my Galaxy -------- Original message --------From: Paul Eggert via tz <tz@iana.org> Date: 2021-09-21 17:13 (GMT-05:00) To: Stephen Colebourne <scolebourne@joda.org> Cc: Time Zone Mailing List <tz@iana.org> Subject: Re: [tz] On 9/21/21 1:20 PM, Stephen Colebourne via tz wrote:> Straw man. Literally no one has suggested this.Yes, of course it's a straw argument. That's the point.Although it'd be technically possible to require a Zone for each state and province in the world, it's not necessary, it would consume maintenance resources that are better spent elsewhere, and Zone proliferation would make things unnecessarily complicated for users. The situation for countries is similar. Admittedly there are few countries than states+provinces, but logically there's no difference between the two cases.> If you don't like ISO countries as the basis of a rule, feel free to> suggest something else.My suggestion is to stick with the rules already in theory.html. They've worked well over the years.> If you want to resolve the issue you need a rule that doesn't end up> with users in Norway or Sweden requesting the time-zone for 1950 and> getting the zone information of Germany instead.If that's the idea, then I also need a rule that doesn't end up with users in Munich requesting the time zone for September 20, 1945 and getting the UTC offset of Berlin instead. The case for a Munich Zone is even stronger than the case for an Oslo Zone, as Munich has a lot more people (and more tzdb users) than Oslo does.Of course this is a straw argument too; that's the point. All this stuff about pre-1970 timestamps is out of scope for tzdb proper, and it's always been out of scope.Only politics would require us to have a Zone for every ISO country. Timekeeping needs don't require it. Admittedly we used to have more Zones, but we've never had a Zone for every ISO country and no timekeeping need for such a rule has ever been demonstrated.
dpatte via tz said:
Only politics would require us to have a Zone for every ISO country. Timekeeping needs don't require it. Admittedly we used to have more Zones, but we've never had a Zone for every ISO country and no timekeeping need for such a rule has ever been demonstrated. Perhaps current timekeeping does need it, but historical time based research does, and that is our point.
Which is why you have the choice of 1970-onwards or everything-we-know. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Stephen Colebourne via tz said:
If you want to resolve the issue you need a rule that doesn't end up with users in Norway or Sweden requesting the time-zone for 1950 and getting the zone information of Germany instead.
But previously you wrote: | In addition, | at least one ID per ISO country is entitled to have full history back | to LMT, **even if it is innaccurate**. So *YOU* are saying that Norway is entitled to have full history but it doesn't have to be accurate. So Germany's pre-1970 history *is* fine for Norway. So you need to find a different reason for saying that ISO 3166 is important.
IMO the defining characteristic of what is or is not acceptable to merge zones across is ISO country boundaries. But I'd love to hear an alternate idea.
How about "places with the same time since 1970"? That's objective and non-political. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Sent from my GalaxyEzactly, which is why tz must use iso countries in its specification.Anything less than accepting iso countries, and continuing to use tz countries is nothing but a tz attempt to apply it's own political decisions on others.Iso has that mandate, not tz. tzdb will continue to be easier to maintainand use if it focuses on timekeeping, not politics.
On 2021-09-22 06:32:58 (+0800), dpatte via tz wrote:
Ezactly, which is why tz must use iso countries in its specification.Anything less than accepting iso countries, and continuing to use tz countries is nothing but a tz attempt to apply it's own political decisions on others.Iso has that mandate, not tz. tzdb will continue to be easier to maintainand use if it focuses on timekeeping, not politics.
tzdb regions refer to areas where people agree what time it is (or was). Whether or not those people agree on what country they are in is (or should be) no concern of the tzdb. Having an ISO two-letter code assigned (or not) does not confer any kind of consensus that a named region is a country (or not). Moreover, ISO takes no position on the boundaries of the regions assigned two-letter codes. Finding consensus on a mapping between countries and time zones is axiomatically impossible since no consensus exists on what countries even are. Philip -- Philip Paeps Senior Reality Engineer Alternative Enterprises
On Tue, Sep 21, 2021 at 12:48 PM Garrett Wollman via tz <tz@iana.org> wrote:
<<On Tue, 21 Sep 2021 12:44:22 -0700, Paul Eggert via tz <tz@iana.org> said:
That sort of argument would be valid only if one accepted as an axiom that countries (or states) are an inherent part of timekeeping.
Governments self-evidently are: they are why we even have time zones in the first place. Everything in this database is _inherently_ political, and your continued attempts to "avoid politics" do nothing of the sort.
If what we want is one time zone per country, let's have that conversation instead of having it happen by accident as a side effect of rules that don't mention countries. Sincerely, Watson Ladd
-GAWollman
Watson Ladd via tz said:
If what we want is one time zone per country, let's have that conversation instead of having it happen by accident as a side effect of rules that don't mention countries.
I agree. But, if we want to talk about countries, what would surely be more useful is a list, for each country, of all the time zones that are at least partially in that country. Oh, please define "country". Is England a country? Wales? The Netherlands (as opposed to the Kingdom of the Netherlands)? Sealand? Taiwan? Macau? Bir Tawil? Liberland? Do you really want to open this can of worms? -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
And who gets to decide? The strategy used over the entire existence of this database has obviated the need to answer that question. Eliot On 21.09.21 22:07, Clive D.W. Feather via tz wrote:
Watson Ladd via tz said:
If what we want is one time zone per country, let's have that conversation instead of having it happen by accident as a side effect of rules that don't mention countries. I agree.
But, if we want to talk about countries, what would surely be more useful is a list, for each country, of all the time zones that are at least partially in that country.
Oh, please define "country". Is England a country? Wales? The Netherlands (as opposed to the Kingdom of the Netherlands)? Sealand? Taiwan? Macau? Bir Tawil? Liberland?
Do you really want to open this can of worms?
The strategy used since tz was created has caused many political arguments and decisions in this group instead of deferring the decisions to the ISO that has this mandate.Sent from my Galaxy -------- Original message --------From: Eliot Lear via tz <tz@iana.org> Date: 2021-09-21 16:17 (GMT-05:00) To: "Clive D.W. Feather" <clive@davros.org>, Watson Ladd <watson@cloudflare.com> Cc: tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb And who gets to decide? The strategy used over the entire existence of this database has obviated the need to answer that question.EliotOn 21.09.21 22:07, Clive D.W. Feather via tz wrote:> Watson Ladd via tz said:>> If what we want is one time zone per country, let's have that>> conversation instead of having it happen by accident as a side effect>> of rules that don't mention countries.> I agree.>> But, if we want to talk about countries, what would surely be more useful> is a list, for each country, of all the time zones that are at least> partially in that country.>> Oh, please define "country". Is England a country? Wales? The Netherlands> (as opposed to the Kingdom of the Netherlands)? Sealand? Taiwan? Macau?> Bir Tawil? Liberland?>> Do you really want to open this can of worms?>
On Tue, Sep 21, 2021 at 3:42 PM dpatte <dpatte@relativedata.com> wrote:
The strategy used since tz was created has caused many political arguments and decisions in this group instead of deferring the decisions to the ISO that has this mandate.
ISO countries doesn't solve some of the thorny political issues, because ISO codes don't take a position on boundary disputes or naming disputes. E.g. Crimea. When the invasion happened, the civil time in the region occupied changed. That means a new zone entry needed to be created. I don't know how defering to ISO resolves the naming of that one. The other example would be Palestine. Regardless of what ISO decides, the time in Palestine is what it is, and is different from Israel in funny ways, and the territories that have gone back and forth have gone back and forth and thus need new entries. What ISO countries do solve is the vanity issues of people wanting timezones with their local big cities in them appearing in a list, even if the timezone is the same as one in another country. Sincerely, Watson Ladd
Sent from my Galaxy
-------- Original message -------- From: Eliot Lear via tz <tz@iana.org> Date: 2021-09-21 16:17 (GMT-05:00) To: "Clive D.W. Feather" <clive@davros.org>, Watson Ladd <watson@cloudflare.com> Cc: tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb
And who gets to decide? The strategy used over the entire existence of this database has obviated the need to answer that question.
Eliot
On 21.09.21 22:07, Clive D.W. Feather via tz wrote:
Watson Ladd via tz said:
If what we want is one time zone per country, let's have that conversation instead of having it happen by accident as a side effect of rules that don't mention countries. I agree.
But, if we want to talk about countries, what would surely be more useful is a list, for each country, of all the time zones that are at least partially in that country.
Oh, please define "country". Is England a country? Wales? The Netherlands (as opposed to the Kingdom of the Netherlands)? Sealand? Taiwan? Macau? Bir Tawil? Liberland?
Do you really want to open this can of worms?
Watson Ladd wrote in <CAN2QdAE8RdE0ynaTVi4WZ4F5PGDEMoxr-A6z_g89VtCsEZ632g@mail.gmail.com>: |On Tue, Sep 21, 2021 at 3:42 PM dpatte <dpatte@relativedata.com> wrote: ... |ISO countries doesn't solve some of the thorny political issues, |because ISO codes don't take a position on boundary disputes or naming |disputes. E.g. Crimea. When the invasion happened, the civil time in You mean the russian majority living in the majority of the land invaded the land it is living on for long? Just to get it right. Since NATO will move onward to the east (do not mind a possibly existing contract, as we only fullfill wishes of the people living there), i will surely soon be able to have a look myself. |the region occupied changed. That means a new zone entry needed to be |created. I don't know how defering to ISO resolves the naming of that |one. | |The other example would be Palestine. Regardless of what ISO decides, |the time in Palestine is what it is, and is different from Israel in |funny ways, and the territories that have gone back and forth have |gone back and forth and thus need new entries. Well not much longer and that problem was overcome by enforced exodus for many parts. If the sun rises over the walls and the razor wire, they surely know it is daytime, and it is time to go and work on the fields they do not own. Maybe what you meant. |What ISO countries do solve is the vanity issues of people wanting |timezones with their local big cities in them appearing in a list, |even if the timezone is the same as one in another country. But let aside that radical left wing political targeted drone killing, as we do not want another cold war, didn't the last discussion came to the conclusion that all you need to get back to original state is a slightly adjusted command line when building the data? Wasn't this true? I mean here at [9296ea527d] i still see Zone Europe/Stockholm 1:12:12 - LMT 1879 Jan 1 1:00:14 - SET 1900 Jan 1 # Swedish Time 1:00 - CET 1916 May 14 23:00 1:00 1:00 CEST 1916 Oct 1 1:00 1:00 - CET 1980 1:00 EU CE%sT in backzone. If i understood right TZ went for long the road to actually go Epoch aka 1970 for the data, without requiring @0 at build time. It may even have been me who said something some years ago, but over the years this free time work progressed without anyone really complaining. Any i came to the conclusion this is maybe right, normal ISO C or POSIX time will not really work before this, so you need specific software to do it "better", and then you can very well compile your own database using the rules which still exist in the TZ database, yet only in backzone. My main concern was the splitting, ugly for human consumption, when reading comments etc., but for software it really should not matter. (I personally would no longer use the source files, but surely go and parse the now standardized binary files.) So please let me wonder why software packages which seem to ship copies of TZ releases in their sources do not simply use the data available as a whole, as if the other backzone data would not be worth it? I personally think that it will be very hard to find a maintainer equally destined and sophisticated. I learned better awk as of Kernighan by reading TZ script files. I am total contra. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Incorrect..Iso 3166Ramallah is recognized in the State of Palestine.Sevastopol is recognized as a city as being in Ukraine.If you disagree politically, you can raise your issue with the iso.Sent from my Galaxy -------- Original message --------From: Watson Ladd <watson@cloudflare.com> Date: 2021-09-21 18:49 (GMT-05:00) To: dpatte <dpatte@relativedata.com> Cc: Eliot Lear <lear@lear.ch>, "Clive D.W. Feather" <clive@davros.org>, tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb On Tue, Sep 21, 2021 at 3:42 PM dpatte <dpatte@relativedata.com> wrote:>>> The strategy used since tz was created has caused many political arguments and decisions in this group instead of deferring the decisions to the ISO that has this mandate.ISO countries doesn't solve some of the thorny political issues,because ISO codes don't take a position on boundary disputes or namingdisputes. E.g. Crimea. When the invasion happened, the civil time inthe region occupied changed. That means a new zone entry needed to becreated. I don't know how defering to ISO resolves the naming of thatone.The other example would be Palestine. Regardless of what ISO decides,the time in Palestine is what it is, and is different from Israel infunny ways, and the territories that have gone back and forth havegone back and forth and thus need new entries.What ISO countries do solve is the vanity issues of people wantingtimezones with their local big cities in them appearing in a list,even if the timezone is the same as one in another country.Sincerely,Watson Ladd>>>> Sent from my Galaxy>>> -------- Original message --------> From: Eliot Lear via tz <tz@iana.org>> Date: 2021-09-21 16:17 (GMT-05:00)> To: "Clive D.W. Feather" <clive@davros.org>, Watson Ladd <watson@cloudflare.com>> Cc: tz <tz@iana.org>> Subject: Re: [tz] Preparing to fork tzdb>> And who gets to decide? The strategy used over the entire existence of> this database has obviated the need to answer that question.>> Eliot>> On 21.09.21 22:07, Clive D.W. Feather via tz wrote:> > Watson Ladd via tz said:> >> If what we want is one time zone per country, let's have that> >> conversation instead of having it happen by accident as a side effect> >> of rules that don't mention countries.> > I agree.> >> > But, if we want to talk about countries, what would surely be more useful> > is a list, for each country, of all the time zones that are at least> > partially in that country.> >> > Oh, please define "country". Is England a country? Wales? The Netherlands> > (as opposed to the Kingdom of the Netherlands)? Sealand? Taiwan? Macau?> > Bir Tawil? Liberland?> >> > Do you really want to open this can of worms?> >>>
Quoting dpatte via tz on Tuesday September 21, 2021:
Incorrect..Iso 3166 Ramallah is recognized in the State of Palestine. Sevastopol is recognized as a city as being in Ukraine. If you disagree politically, you can raise your issue with the iso.
ISO 3166 only encodes countries/territories, and subnational divisions of those entities that are supplied by the respective governments. Most of those subnational divisons are administrative regions like states and they are not necessarily comprehenseive. The standard does not specify boundaries for those subdivisions, and I don't think it would be decisive between conflicting claims (e.g. Western Sahara is encoded independently, but also as entities like Laayoune within Morocco) While ISO 3166 may do a heavy lift in this area and move the dispute over what is a country to another entity, it won't solve a lot of geographic coding problems. kim
On Sep 21, 2021, at 4:51 PM, dpatte via tz <tz@iana.org> wrote:
Date: 2021-09-21 18:49 (GMT-05:00) To: dpatte <dpatte@relativedata.com> Cc: Eliot Lear <lear@lear.ch>, "Clive D.W. Feather" <clive@davros.org>, tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb
On Tue, Sep 21, 2021 at 3:42 PM dpatte <dpatte@relativedata.com> wrote:
The strategy used since tz was created has caused many political arguments and decisions in this group instead of deferring the decisions to the ISO that has this mandate.
ISO countries doesn't solve some of the thorny political issues, because ISO codes don't take a position on boundary disputes or naming disputes. E.g. Crimea. When the invasion happened, the civil time in the region occupied changed. That means a new zone entry needed to be created. I don't know how defering to ISO resolves the naming of that one.
The other example would be Palestine. Regardless of what ISO decides, the time in Palestine is what it is, and is different from Israel in funny ways, and the territories that have gone back and forth have gone back and forth and thus need new entries.
Incorrect..
Iso 3166
Ramallah is recognized in the State of Palestine.
And Hebron, presumably - it's the city used for West Bank Palestine.
Sevastopol is recognized as a city as being in Ukraine.
But it's not a city used for a tzdb - Simferopol is. And Kyiv is also a city in Ukraine, but, as far as I know, it keeps different time from Simferopol, and the tzdb recognizes that. However, "deferring to ISO" doesn't mean "providing one *and only one* tzdb region per ISO country"; such a policy would not work very well for the three largest countries listed in the northamerica file, for example. If, indeed, Crimea and the rest of Ukraine do not, in practice, keep the same time, then, hopefully, "deferring to ISO" also wouldn't mean "giving Crimea the same offset and rules as the rest of Ukraine".
I don't believe anyone has suggested that zones should be limited to 'only one zone per country'. I certainly never sugested that. In fact quite the opposite. But I believe there should be AT LEAST one zone per iso country, and it was the removal of the Montreal historical data a few years back that got me started.If it's THE tz database, it should be able to provide not only since-70 history, but be expandable to all tz history, potentially back to the 1800s when most cities implemented local mean time as local standards.I recognize many users of this db dont need pre1970 history, and perhaps they could filter for only recent data. But many apps need UT (gmt) before 1970 as I mentioned. Think astronomy apps, for example.Up until recently some of this data was available in the main tz file. We need that back, with the ability to expand and improve it. This should be possible without expanding the post1970 table.Sent from my Galaxy -------- Original message --------From: Guy Harris <gharris@sonic.net> Date: 2021-09-21 21:02 (GMT-05:00) To: dpatte <dpatte@relativedata.com> Cc: Watson Ladd <watson@cloudflare.com>, tz <tz@iana.org>, Eliot Lear <lear@lear.ch> Subject: Re: [tz] Preparing to fork tzdb On Sep 21, 2021, at 4:51 PM, dpatte via tz <tz@iana.org> wrote:> Date: 2021-09-21 18:49 (GMT-05:00)> To: dpatte <dpatte@relativedata.com>> Cc: Eliot Lear <lear@lear.ch>, "Clive D.W. Feather" <clive@davros.org>, tz <tz@iana.org>> Subject: Re: [tz] Preparing to fork tzdb> >> On Tue, Sep 21, 2021 at 3:42 PM dpatte <dpatte@relativedata.com> wrote:>> >>> The strategy used since tz was created has caused many political arguments and decisions in this group instead of deferring the decisions to the ISO that has this mandate.>> >> ISO countries doesn't solve some of the thorny political issues,>> because ISO codes don't take a position on boundary disputes or naming>> disputes. E.g. Crimea. When the invasion happened, the civil time in>> the region occupied changed. That means a new zone entry needed to be>> created. I don't know how defering to ISO resolves the naming of that>> one.>> >> The other example would be Palestine. Regardless of what ISO decides,>> the time in Palestine is what it is, and is different from Israel in>> funny ways, and the territories that have gone back and forth have>> gone back and forth and thus need new entries.> > Incorrect..> > Iso 3166> > Ramallah is recognized in the State of Palestine.And Hebron, presumably - it's the city used for West Bank Palestine.> Sevastopol is recognized as a city as being in Ukraine.But it's not a city used for a tzdb - Simferopol is.And Kyiv is also a city in Ukraine, but, as far as I know, it keeps different time from Simferopol, and the tzdb recognizes that.However, "deferring to ISO" doesn't mean "providing one *and only one* tzdb region per ISO country"; such a policy would not work very well for the three largest countries listed in the northamerica file, for example.If, indeed, Crimea and the rest of Ukraine do not, in practice, keep the same time, then, hopefully, "deferring to ISO" also wouldn't mean "giving Crimea the same offset and rules as the rest of Ukraine".
dpatte via tz <tz@iana.org> writes:
I certainly never sugested that. In fact quite the opposite. But I believe there should be AT LEAST one zone per iso country, and it was the removal of the Montreal historical data a few years back that got me started.
It's frustrating when people keep saying things like this given that no data has been removed. I am literally looking at it right now in the GitHub repository. It's right there. It becomes hard to tell if people are genuinely confused about this, if "removed" is shorthand for "my operating system stopped including that zone," if "removed" means "I find it difficult to access now," or if it's a rhetorical device to make the changes sound more severe and destructive than they actually were to people who don't understand the true situation. But it's simply not true as stated.
Up until recently some of this data was available in the main tz file. We need that back, with the ability to expand and improve it.
I don't understand what you think is preventing you from expanding and improving that data literally right now. My understanding is that Paul has said multiple times that if someone wants to do the thankless and tedious reserach to find more accurate historical data with references, patches will be reviewed and integrated. Am I misremembering somehow? I completely understand that you are not happy with the default build choices or with the set of data that's shipped with major operating systems, and I'm not saying that you have nothing to be unhappy about. But I think it would lead to more constructive discussion if we could describe the situation accurately and without exaggeration, and not claim that data was deleted when it is still included in every release of tzdata. -- Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>
I am aware that montreal was moved to back zone because its post 1970 data after research turned out to match toronto.But I have yet to get a clear answer as to whether we can request modifications to backzone records even if they only affect pre1970 clocks, and whether we can add new backzones for regions that differ before 1970. Sent from my Galaxy -------- Original message --------From: Russ Allbery via tz <tz@iana.org> Date: 2021-09-22 23:06 (GMT-05:00) To: tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb dpatte via tz <tz@iana.org> writes:> I certainly never sugested that. In fact quite the opposite. But I> believe there should be AT LEAST one zone per iso country, and it was> the removal of the Montreal historical data a few years back that got me> started.It's frustrating when people keep saying things like this given that nodata has been removed. I am literally looking at it right now in theGitHub repository. It's right there.It becomes hard to tell if people are genuinely confused about this, if"removed" is shorthand for "my operating system stopped including thatzone," if "removed" means "I find it difficult to access now," or if it'sa rhetorical device to make the changes sound more severe and destructivethan they actually were to people who don't understand the true situation.But it's simply not true as stated.> Up until recently some of this data was available in the main tz> file. We need that back, with the ability to expand and improve it.I don't understand what you think is preventing you from expanding andimproving that data literally right now. My understanding is that Paulhas said multiple times that if someone wants to do the thankless andtedious reserach to find more accurate historical data with references,patches will be reviewed and integrated. Am I misremembering somehow?I completely understand that you are not happy with the default buildchoices or with the set of data that's shipped with major operatingsystems, and I'm not saying that you have nothing to be unhappy about.But I think it would lead to more constructive discussion if we coulddescribe the situation accurately and without exaggeration, and not claimthat data was deleted when it is still included in every release oftzdata.-- Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>
On 2021-09-23 12:29:06 (+0800), dpatte via tz wrote:
I am aware that montreal was moved to back zone because its post 1970 data after research turned out to match toronto.But I have yet to get a clear answer as to whether we can request modifications to backzone records even if they only affect pre1970 clocks, and whether we can add new backzones for regions that differ before 1970.
Changes are routinely made to backzone, backed up with verifiable documentation and commentary. https://github.com/eggert/tz/commits/main/backzone Philip -- Philip Paeps Senior Reality Engineer Alternative Enterprises
dpatte <dpatte@relativedata.com> writes:
But I have yet to get a clear answer as to whether we can request modifications to backzone records even if they only affect pre1970 clocks, and whether we can add new backzones for regions that differ before 1970.
Obviously, I'm not Paul and can't give you an authoritative answer, but I have read the archives of this mailing list back to its inception and I see absolutely no reason to believe that you can't contribute modifications to backzone records or that those wouldn't be accepted. Other people certainly have contributed such entries in the past and they were accepted. For example, from just last year: https://mm.icann.org/pipermail/tz/2020-July/029167.html Or in 2019: https://mm.icann.org/pipermail/tz/2019-July/028276.html Or in 2017: https://mm.icann.org/pipermail/tz/2017-July/025167.html Of course, they would need to come with supporting references for why they are correct, and there are some time practices that aren't representable or are not reasonable to track, which might be exceptions. For instance, theory.html mentions: In particular, the tz database's LMT offsets should not be considered meaningful, and should not prompt creation of timezones merely because two locations differ in LMT or transitioned to standard time at different dates. so presumably that specific change would not be welcome. But in general, I see no reason why contributions to backzone would not be welcome. I just refreshed my memory by reading the thread in 2014 that originally introduced backzone and the clear intent at the time was, among other things, to capture such contributions, including by adding new zone identifiers when necessary. There does seem to be some reluctance expressed in theory.html to take on the full number of entries that would be required to fully express historical time, and I know there have been some ambiguous past discussions about this, but in practice I don't think this will be relevant. Given the long history of the project and past experience, I think we can predict with a fair amount of confidence that the number of people willing to research and contribute such entries will be modest and are unlikely to overwhelm the project with data. Anyway, obviously Paul (or Tim) would be canonical here; I'm just an observer. But my clear impression from reading the mailing list over the years is that the reason for the low volume of contributions to that data is lack of interest among contributors and the difficulty of the effort, not that the maintainers are declining such submissions. Few people care about such data enough to do the work to track it down and document it, but there doesn't seem to be any reluctance to incorporate it if someone does do that work and provides it in the form of a patch. (It obviously gets somewhat lower priority than changes to times after 1970 if multiple things are competing for maitnainer attention.) -- Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>
Clive D.W. Feather via tz <tz@iana.org> wrote:
Oh, please define "country".
Jon Postel deftly answered that question in the 1980s by deferring to ISO 3166, and so far all the concrete suggestions for the tz database have followed his lead. Tony. -- f.anthony.n.finch <dot@dotat.at> https://dotat.at/ North Utsire: Southwesterly 3 to 5, increasing 6 or 7, veering westerly 3 or 4 later. Moderate or rough, becoming rough later, occasionally very rough later in far north. Rain later. Good, occasionally moderate later.
Tony Finch said:
Oh, please define "country". Jon Postel deftly answered that question in the 1980s by deferring to ISO 3166,
Except that the ISO 3166 list contained Belorussia and Ukraine, both of which were part of the USSR that was also in the list. The only reason they were in the list was a 1940s deal to give the USSR three votes in the UN to counter the fact that there were more "western" countries than Warsaw Pact ones. So there was no genuine logic to that list, just politics. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Clive D.W. Feather <clive@davros.org> wrote:
Tony Finch said:
Oh, please define "country". Jon Postel deftly answered that question in the 1980s by deferring to ISO 3166,
Except that [...]
I don't think anyone is claiming it is perfect, just that it is the best list available: it is familiar, widely used in Internet standards, reasonably well maintained, and other people are responsible so we don't have to fuss about it. Tony. -- f.anthony.n.finch <dot@dotat.at> https://dotat.at/ Humber, Thames: West or southwest 3 or 4, increasing 5 or 6 later. Smooth or slight becoming slight or moderate. Mainly fair. Good.
Tony Finch said:
Oh, please define "country". Jon Postel deftly answered that question in the 1980s by deferring to ISO 3166,
Except that [...]
I don't think anyone is claiming it is perfect, just that it is the best list available: it is familiar, widely used in Internet standards, reasonably well maintained, and other people are responsible so we don't have to fuss about it.
Perhaps. And for fairly uncontentious things, it may well work. But if we start saying "every 'country' with an ISO 3166 code has its own time zone", then how long will it be before (to pick an example) the Welsh start saying "we've got our own country domain, so we want our own time zone"? Time zones should only be in the database if they differ in some way from all other time zones or they are in there for backwards compatibility reasons (so as to not break existing users), though those should be marked as deprecated and users of TZDB encouraged not to use them for new end-users. The meaning of "in some way" appears to be the actual point of contention. "in some way since 1970-01-00" is an arguable choice. "relates loosely to a specific ISO 3166 code" or "has a name in a different country to where I live" isn't. IMAO, of course. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Iso is a living international standard, and it's their mandate, not the mandate of timezone collectors to do the negotiation and diplomacy in order to arrive at and maintain the world standard.I certainly don't believe we have the mandate or skillset to ignore the international standard widely used in modern computer technology.Sent from my Galaxy -------- Original message --------From: "Clive D.W. Feather via tz" <tz@iana.org> Date: 2021-09-22 03:34 (GMT-05:00) To: Tony Finch <dot@dotat.at> Cc: tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb Tony Finch said:>> Oh, please define "country".> Jon Postel deftly answered that question in the 1980s by deferring to ISO> 3166,Except that the ISO 3166 list contained Belorussia and Ukraine, both ofwhich were part of the USSR that was also in the list.The only reason they were in the list was a 1940s deal to give the USSRthree votes in the UN to counter the fact that there were more "western"countries than Warsaw Pact ones.So there was no genuine logic to that list, just politics.-- Clive D.W. Feather | If you lie to the compiler,Email: clive@davros.org | it will get its revenge.Web: http://www.davros.org | - Henry SpencerMobile: +44 7973 377646
On 22.09.21 15:09, dpatte via tz wrote:
Iso is a living international standard, and it's their mandate, not the mandate of timezone collectors to do the negotiation and diplomacy in order to arrive at and maintain the world standard.
Right. And this group has avoided their business for decades.
On 2021-09-22 9:17 AM, Eliot Lear via tz wrote:
On 22.09.21 15:09, dpatte via tz wrote:
Iso is a living international standard, and it's their mandate, not the mandate of timezone collectors to do the negotiation and diplomacy in order to arrive at and maintain the world standard.
Right. And this group has avoided their business for decades.
I agree. There should be only one 'authority' for country codes, and ISO 3166 is it as far as I know. It seems it is the mapping of the country codes to time zone names (tags) that is causing the current difficulties. It seems to me it should not be such a problem. As zone1970.tab (and zone.tab) says: # This table is intended as an aid for users, to help them select timezones # appropriate for their practical needs. It is not intended to take or # endorse any position on legal or territorial claims. It is useful for "practical needs" but cannot be relied upon in all circumstances. Tzdb can use ISO 3166 but should not alter it or fall victim to any political controversies it might imply. I would point out, however, that zone1970.tab also contains the time zone *coordinates*. This is different matter than country codes. In the work I'm doing the coordinates are important. They define the approximate default coordinates of time zones, and this is useful in a number of ways, indicating geographic distances between time zones and distinction of Northern and Southern hemispheres. I think this has significance to the 'merging' controversy. "Europe/Oslo" is not the same time zone as "Europe/Berlin". They may have been using the same rule sets since 1970 resulting in identical local YMDhms representations but they have different time zone names (tags) and distinct coordinates. Oslo is not the same city as Berlin. If time zone "Europe/Oslo" ever existed it still exists today. "Europe/Oslo" could adopt a new set of rules different from "Europe/Berlin" and its possible they might given the new elective rules being suggested by the EU. Olsen's original insight to use towns and cities to name time zones turns out to be extraordinarily useful. Whether or not "Europe/Oslo" is in Norway is entirely beside the point of local time in the "Europe/Oslo" time zone. Time zones really exist only in the time domain relative to the (special) "Etc/UTC" time zone. I'm sure most contributors to the tz list understand this, but there's a natural tendency equate a time zone with a location or place since this is where the general idea comes from to begin with. But I think tzdb must be vigilant to maintain this conceptual separation of time zone v.s. "place", especially in regard to politically named and claimed geographic boundaries such as Norway or Germany. This separation of "time" from "place" may seem inconsistent with my previous point about time zone coordinates but I don't think so. The time zone names are based on cities and the cities have approximate geographical location and this ties the time zone to a location. This is useful for some important purposes and supports the basic idea behind time zones. But it is a separate conceptual and implementation consideration from the local time zone YMDhms representation with respect to "Etc/UTC". The geographic coordinates have nothing to do with the local time because any time zone may adopt any UTC-offset (STDOFF) or DST rules they choose and this decouples the time zone's local time from the city location. I'm sure most tzdb experts recognize this distinction but it seems to have somehow been lost track of recently. I hope participants can find the way back to the cooperation that has characterized the group for so long. Thanks, -Brooks Harris
On Sep 22, 2021, at 11:32 AM, Brooks Harris via tz <tz@iana.org> wrote:
I would point out, however, that zone1970.tab also contains the time zone *coordinates*.
A "time zone" isn't a point, it's a set of points - and, at least if "time zone" means "tzdb region", it's a set of points not guaranteed to be simply connected, so it doesn't have a single curve that represents its border. As such, a "time zone" doesn't have a single coordinate pair, it has a whole bunch of coordinate pairs corresponding to its borders. All that zone.tab and zone1970.tab have is: # 2. Latitude and longitude of the timezone's principal location # in ISO 6709 sign-degrees-minutes-seconds format, # either ±DDMM±DDDMM or ±DDMMSS±DDDMMSS, # first latitude (+ is north), then longitude (+ is east). The "principal location" is, presumably, the city or other location that gives the region its name; I live about 480 km from the "principal location" of the tzdb region I'm in.
This is different matter than country codes. In the work I'm doing the coordinates are important. They define the approximate default coordinates of time zones, and this is useful in a number of ways, indicating geographic distances between time zones and distinction of Northern and Southern hemispheres.
But it's not useful for determining in which tzdb region an arbitrary coordinate pair belongs, which is also important to at least some Darwin-based OSes (macOS, iOS, iPadOS, watchOS - I can't speak for tvOS), as that's how their mechanism for automatically determining your machine's current tzdb region works. For that information, see https://github.com/evansiroky/timezone-boundary-builder which uses OpenStreetMap.
I think this has significance to the 'merging' controversy. "Europe/Oslo" is not the same time zone as "Europe/Berlin". They may have been using the same rule sets since 1970 resulting in identical local YMDhms representations but they have different time zone names (tags) and distinct coordinates. Oslo is not the same city as Berlin.
San Francisco is not the same city as Los Angeles, but they're in the same tzdb region, so "city A is not the same city as city B" is insufficient region to have both Region/A and Region/B in the tzdb. Europe/Oslo and Europe/Berlin are, in 2021a and earlier releases, separate tzdb regions for historical reasons, both in the sense of "they had different rule sets prior to 1970" and in the sense of "the process of setting up tzdb regions happened to give them separate regions". What Paul has noted is that there are cases where a single tzdb region covers locations not all of which have the same rules/offsets prior to 1970, and asks what's special about Norway and Germany. Two ways of changing that are 1) merging tzdb regions so that Europe/Oslo is a link to Europe/Berlin when using the default data and a separate tzdb region when using the backzone data and 2) splitting other zones so that pre-1970 data is available for them as well. Paul is going with 1) - and he notes that the backzone is still available and can be used.
If time zone "Europe/Oslo" ever existed it still exists today. "Europe/Oslo" could adopt a new set of rules different from "Europe/Berlin" and its possible they might given the new elective rules being suggested by the EU.
If tzdb region "Europe/Oslo" had never existed, Norway could adopt a new set of rules different from those in "Europe/Berlin", and a new "Europe/Oslo" region created. I.e., the *current* existence of a "Europe/Oslo" region is not at all necessary to allow a region that includes Oslo to adopt a new and different offset-from-UTC or new and different DST rules.
Olsen's original insight to use towns and cities to name time zones turns out to be extraordinarily useful. Whether or not "Europe/Oslo" is in Norway is entirely beside the point of local time in the "Europe/Oslo" time zone. Time zones really exist only in the time domain relative to the (special) "Etc/UTC" time zone. I'm sure most contributors to the tz list understand this, but there's a natural tendency equate a time zone with a location or place since this is where the general idea comes from to begin with. But I think tzdb must be vigilant to maintain this conceptual separation of time zone v.s. "place", especially in regard to politically named and claimed geographic boundaries such as Norway or Germany.
Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter. And there's no guarantee that tzdb region boundaries correspond exactly to national boundaries. (I live in one of the many countries that contain more than one tzdb region - and some of them differ for reasons other than how far they are from the IERS Reference Meridian, e.g. they differ base don whether they observe DST or not.)
On 2021-09-22 4:41 PM, Guy Harris wrote:
On Sep 22, 2021, at 11:32 AM, Brooks Harris via tz <tz@iana.org> wrote:
I would point out, however, that zone1970.tab also contains the time zone *coordinates*. A "time zone" isn't a point, it's a set of points - and, at least if "time zone" means "tzdb region", it's a set of points not guaranteed to be simply connected, so it doesn't have a single curve that represents its border.
As such, a "time zone" doesn't have a single coordinate pair, it has a whole bunch of coordinate pairs corresponding to its borders.
All that zone.tab and zone1970.tab have is:
# 2. Latitude and longitude of the timezone's principal location # in ISO 6709 sign-degrees-minutes-seconds format, # either ±DDMM±DDDMM or ±DDMMSS±DDDMMSS, # first latitude (+ is north), then longitude (+ is east).
The "principal location" is, presumably, the city or other location that gives the region its name; I live about 480 km from the "principal location" of the tzdb region I'm in. Yes, all true. I think that "principal location" is helpful as a default. But it is not accurate, and it does not define the "region". It's merely an approximate guide.
This is different matter than country codes. In the work I'm doing the coordinates are important. They define the approximate default coordinates of time zones, and this is useful in a number of ways, indicating geographic distances between time zones and distinction of Northern and Southern hemispheres. But it's not useful for determining in which tzdb region an arbitrary coordinate pair belongs, which is also important to at least some Darwin-based OSes (macOS, iOS, iPadOS, watchOS - I can't speak for tvOS), as that's how their mechanism for automatically determining your machine's current tzdb region works.
For that information, see
https://github.com/evansiroky/timezone-boundary-builder
which uses OpenStreetMap.
Yes, true. I don't believe providing information for automatic time zone detection is tzdb's responsibility. Its an application feature downstream from tzdb. But I think that "principal location" is a helpful starting point in some circumstances. Part of my observation is that the zone1970.tab contains *both* country code and coordinates, which couples the tz time zones to the country creating some part of the difficulty of avoiding politics as much as possible. Maybe ideally there should have been a 'coordinates' file and a 'country' file. I'm not suggesting this should change, only pointing out this is where tzdb got connected to country politics.
I think this has significance to the 'merging' controversy. "Europe/Oslo" is not the same time zone as "Europe/Berlin". They may have been using the same rule sets since 1970 resulting in identical local YMDhms representations but they have different time zone names (tags) and distinct coordinates. Oslo is not the same city as Berlin. San Francisco is not the same city as Los Angeles, but they're in the same tzdb region, so "city A is not the same city as city B" is insufficient region to have both Region/A and Region/B in the tzdb.
Europe/Oslo and Europe/Berlin are, in 2021a and earlier releases, separate tzdb regions for historical reasons, both in the sense of "they had different rule sets prior to 1970" and in the sense of "the process of setting up tzdb regions happened to give them separate regions".
Right. I feel retaining them in the main files is the best approach.
What Paul has noted is that there are cases where a single tzdb region covers locations not all of which have the same rules/offsets prior to 1970, and asks what's special about Norway and Germany.
Two ways of changing that are 1) merging tzdb regions so that Europe/Oslo is a link to Europe/Berlin when using the default data and a separate tzdb region when using the backzone data and 2) splitting other zones so that pre-1970 data is available for them as well.
Paul is going with 1) - and he notes that the backzone is still available and can be used.
Right. I think I'd have chosen to retain all time zones in the main well known source files and providing new files that support filtering and/or new features if that's the intention. But Paul has been more often right than wrong, so I guess we'll see.
If time zone "Europe/Oslo" ever existed it still exists today. "Europe/Oslo" could adopt a new set of rules different from "Europe/Berlin" and its possible they might given the new elective rules being suggested by the EU. If tzdb region "Europe/Oslo" had never existed, Norway could adopt a new set of rules different from those in "Europe/Berlin", and a new "Europe/Oslo" region created.
I.e., the *current* existence of a "Europe/Oslo" region is not at all necessary to allow a region that includes Oslo to adopt a new and different offset-from-UTC or new and different DST rules.
Olsen's original insight to use towns and cities to name time zones turns out to be extraordinarily useful. Whether or not "Europe/Oslo" is in Norway is entirely beside the point of local time in the "Europe/Oslo" time zone. Time zones really exist only in the time domain relative to the (special) "Etc/UTC" time zone. I'm sure most contributors to the tz list understand this, but there's a natural tendency equate a time zone with a location or place since this is where the general idea comes from to begin with. But I think tzdb must be vigilant to maintain this conceptual separation of time zone v.s. "place", especially in regard to politically named and claimed geographic boundaries such as Norway or Germany. Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter.
Yes, that's part of my concern about merging Europe/Oslo with Europe/Berlin.
And there's no guarantee that tzdb region boundaries correspond exactly to national boundaries. (I live in one of the many countries that contain more than one tzdb region - and some of them differ for reasons other than how far they are from the IERS Reference Meridian, e.g. they differ base don whether they observe DST or not.)
Determining national boundaries and time zone regions is a difficult problem, probably impossible in cases like disputed boarders. We all understand tzdb cannot be perfect in all respects. It is, nonetheless, the best available collection of time zone information, having established itself as essentially the de facto standard defining civil time. We really don't want to see it broken.
On Sep 22, 2021, at 2:49 PM, Brooks Harris <brooks@edlmax.com> wrote:
On 2021-09-22 4:41 PM, Guy Harris wrote:
On Sep 22, 2021, at 11:32 AM, Brooks Harris via tz <tz@iana.org> wrote:
The "principal location" is, presumably, the city or other location that gives the region its name; I live about 480 km from the "principal location" of the tzdb region I'm in. Yes, all true. I think that "principal location" is helpful as a default.
How so? I'm over 400 km from my tzdb region's "principal location". And what's special about 111 E 1st Street, Los Angeles, that makes it more "principal" than, for example, 2800 E Observatory Road, Los Angeles?
Yes, true. I don't believe providing information for automatic time zone detection is tzdb's responsibility. Its an application feature downstream from tzdb. But I think that "principal location" is a helpful starting point in some circumstances.
In what circumstances are those?
Part of my observation is that the zone1970.tab contains *both* country code and coordinates, which couples the tz time zones to the country creating some part of the difficulty of avoiding politics as much as possible. Maybe ideally there should have been a 'coordinates' file and a 'country' file. I'm not suggesting this should change, only pointing out this is where tzdb got connected to country politics.
One question is "should zone.tab and zone1970.tab officially even be considered part of the tzdb, or are they just "interesting additional data"?
Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter. Yes, that's part of my concern about merging Europe/Oslo with Europe/Berlin.
Right now, the only difference that would make to the mobile machine in question is in applications that handle pre-1970 time stamps in the tzdb region in which they're currently located. For *those* applications, they would require a set of tzdb files that include backzone if they are to handle pre-1970 time stamps while the user is in, for example, Asmara. If they include backzone, they will be unaffected by the merger. Those applications might also require some *additional* pre-1970 region splits in the future.
On 2021-09-22 6:18 PM, Guy Harris wrote:
On Sep 22, 2021, at 2:49 PM, Brooks Harris <brooks@edlmax.com> wrote:
On 2021-09-22 4:41 PM, Guy Harris wrote:
On Sep 22, 2021, at 11:32 AM, Brooks Harris via tz <tz@iana.org> wrote:
The "principal location" is, presumably, the city or other location that gives the region its name; I live about 480 km from the "principal location" of the tzdb region I'm in. Yes, all true. I think that "principal location" is helpful as a default. How so? I'm over 400 km from my tzdb region's "principal location".
And what's special about 111 E 1st Street, Los Angeles, that makes it more "principal" than, for example, 2800 E Observatory Road, Los Angeles? It makes its "principal location" different than another time zone, such as "America/New_York".
Yes, true. I don't believe providing information for automatic time zone detection is tzdb's responsibility. Its an application feature downstream from tzdb. But I think that "principal location" is a helpful starting point in some circumstances. In what circumstances are those? Say we have an interchange format that includes:
a) tz time zone tag b) local tz date and time c) coordinate field The idea here is that some devices will be able to support accurate coordinates from GPS, like a truck, car, or smart phone. But many devices will not have GPS or other means of determining location. So, the tzdb supplied "principal location" coordinates are a much better default than zero-zero or "unknown".
Part of my observation is that the zone1970.tab contains *both* country code and coordinates, which couples the tz time zones to the country creating some part of the difficulty of avoiding politics as much as possible. Maybe ideally there should have been a 'coordinates' file and a 'country' file. I'm not suggesting this should change, only pointing out this is where tzdb got connected to country politics. One question is "should zone.tab and zone1970.tab officially even be considered part of the tzdb, or are they just "interesting additional data"?
I agree that's worth considering. One approach that could dispose of some of the controversy in one stroke would be to simply delete those files. But, once something is plugged in you can't turn it off. I expect such a suggestion would be met with some derision.
Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter. Yes, that's part of my concern about merging Europe/Oslo with Europe/Berlin. Right now, the only difference that would make to the mobile machine in question is in applications that handle pre-1970 time stamps in the tzdb region in which they're currently located.
An application may be calculating times in some other or several other time zones. Its not just where its located.
For *those* applications, they would require a set of tzdb files that include backzone if they are to handle pre-1970 time stamps while the user is in, for example, Asmara. If they include backzone, they will be unaffected by the merger.
Those applications might also require some *additional* pre-1970 region splits in the future.
Yes, there will be ongoing changes no doubt. Tzdb is not easy.
On Sep 22, 2021, at 3:49 PM, Brooks Harris <brooks@edlmax.com> wrote:
On 2021-09-22 6:18 PM, Guy Harris wrote:
On Sep 22, 2021, at 2:49 PM, Brooks Harris <brooks@edlmax.com> wrote:
On 2021-09-22 4:41 PM, Guy Harris wrote:
On Sep 22, 2021, at 11:32 AM, Brooks Harris via tz <tz@iana.org> wrote:
The "principal location" is, presumably, the city or other location that gives the region its name; I live about 480 km from the "principal location" of the tzdb region I'm in. Yes, all true. I think that "principal location" is helpful as a default. How so? I'm over 400 km from my tzdb region's "principal location".
And what's special about 111 E 1st Street, Los Angeles, that makes it more "principal" than, for example, 2800 E Observatory Road, Los Angeles? It makes its "principal location" different than another time zone, such as "America/New_York".
In what fashion does having 111 E 1st Street, Los Angeles be the "principal location" for America/Los_Angeles make that principal location different from another time zone, but having 2800 E Observatory Road, Los Angeles be the "principal location" for America/Los_Angeles not do so? (Did 111 E 1st Street, Los Angeles appear in Rebel Without a Cause, Back to the Future I and II, and La La Land? 2800 E Observatory Road, Los Angeles did.)
Yes, true. I don't believe providing information for automatic time zone detection is tzdb's responsibility. Its an application feature downstream from tzdb. But I think that "principal location" is a helpful starting point in some circumstances. In what circumstances are those? Say we have an interchange format that includes:
a) tz time zone tag b) local tz date and time c) coordinate field
The idea here is that some devices will be able to support accurate coordinates from GPS, like a truck, car, or smart phone. But many devices will not have GPS or other means of determining location. So, the tzdb supplied "principal location" coordinates are a much better default than zero-zero or "unknown".
So in what circumstances is the current tzdb region currently used, in the real world, to determine your location? And in what fashion is "somewhere in Los Angeles" a better location indication for where I am right now than "I don't know"? (The fact that the city I'm in has the same initials as Los Angeles doesn't count; as noted, it's over 400 km away from Los Angeles.)
Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter. Yes, that's part of my concern about merging Europe/Oslo with Europe/Berlin. Right now, the only difference that would make to the mobile machine in question is in applications that handle pre-1970 time stamps in the tzdb region in which they're currently located.
An application may be calculating times in some other or several other time zones. Its not just where its located.
The statement to which you responded iss "Although, when it comes to determining whether to use Europe/Berlin or Europe/Oslo on your laptop/tablet/phone/watch, the geographic boundaries of Europe/Berlin, Europe/Oslo, etc. *do* matter." An application that's using tzdb regions other than the one you're in will not, when using those regions, be affected by whether the zone you're in is Europe/Oslo or Europe/Berlin. The boundaries of the regions matter only if the application is choosing the other locations to use by taking a location on Earth and determining, using those boundaries, which region that location is in. The difference between Europe/Oslo and Europe/Berlin will matter to an application dealing with pre-1970 times even if the region names are coming from a user interface or a configuration file; in the latter case, the boundaries don't matter to the application, although they'd matter to whoever or whatever entered those region names.
Date: Wed, 22 Sep 2021 14:32:01 -0400 From: Brooks Harris via tz <tz@iana.org> Message-ID: <614B76A1.6030000@edlmax.com> | It seems it is the mapping of the country codes to time zone names | (tags) that is causing the current difficulties. No, it's not - the current issues have nothing whatever to do with zone names or country codes (which are two unrelated issues, the latter of which has only peripheral relevance to tzdb in any case). The current issue relates to what data is present in some of the zones as generated by default. That's it in its entirety. kre
On 2021-09-22 6:20 PM, Robert Elz wrote:
Date: Wed, 22 Sep 2021 14:32:01 -0400 From: Brooks Harris via tz <tz@iana.org> Message-ID: <614B76A1.6030000@edlmax.com>
| It seems it is the mapping of the country codes to time zone names | (tags) that is causing the current difficulties.
No, it's not - the current issues have nothing whatever to do with zone names or country codes (which are two unrelated issues, the latter of which has only peripheral relevance to tzdb in any case).
The current issue relates to what data is present in some of the zones as generated by default.
That's it in its entirety.
kre
Your'e right. I should have said its one of the topics that seems to have contributed to the divergence of opinions.
Brooks Harris via tz said:
I agree. There should be only one 'authority' for country codes, and ISO 3166 is it as far as I know.
It seems it is the mapping of the country codes to time zone names (tags) that is causing the current difficulties. It seems to me it should not be such a problem.
Right. So why is "NO -> Europe/Berlin" a problem when "DE -> Europe/Berlin" isn't? I don't follow this argument. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
"Clive D.W. Feather via tz" <tz@iana.org> writes:
Right. So why is "NO -> Europe/Berlin" a problem when "DE -> Europe/Berlin" isn't? I don't follow this argument.
In a vacuum, that'd perhaps be fine (although bad memories of WWII might be a reason to say it isn't). But I think the real reason for concern is that the proposed change causes a default build of tzdb to contain strictly worse (less accurate) data for NO than what was there before. It is very difficult to accept that that's a good change. The fact that we've made similar changes in the past without so much push-back does not make it okay. Rather, we ought to acknowledge that those past changes were also bad ideas, and undo them. regards, tom lane
Tom Lane said:
Right. So why is "NO -> Europe/Berlin" a problem when "DE -> Europe/Berlin" isn't? I don't follow this argument.
In a vacuum, that'd perhaps be fine (although bad memories of WWII might be a reason to say it isn't). But I think the real reason for concern is that the proposed change causes a default build of tzdb to contain strictly worse (less accurate) data for NO than what was there before.
I see that point, though it depends on how you view pre-1970 data. After all, the data is already wrong for much of the area covered by Europe/Berlin and possibly (I wouldn't be surprised either way) much of the area covered by Europe/Oslo. But it's a completely different argument to the "but they're different countries" one that's been going around this list. That's the one that I don't understand. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Date: Fri, 24 Sep 2021 16:05:45 +0100 From: "Clive D.W. Feather via tz" <tz@iana.org> Message-ID: <YU3pSS/8EM0Nqjkd@davros.org> | I see that point, though it depends on how you view pre-1970 data. After | all, the data is already wrong for much of the area covered by | Europe/Berlin and possibly (I wouldn't be surprised either way) much of | the area covered by Europe/Oslo. It isn't "because they're different countries" but because there was data that is now (effectively) deleted (for anyone who doesn't make a private fork - and I don't think anyone really believes that any kind of fork of the data is a good idea). In an earlier message, Paul said: | so far it's been very unlikely, for example, that Sweden would diverge | from Germany. (This may of course change in the future, in which case | we'll of course deal with it.) "deal with it" means, of course, change Europe/Stockholm back from being a Link (assuming it ever actually becomes one) to being a Zone again. In other words, any authority with the ability to make timezone changes who wants to have their own zone, with their own data (including as much of their own historical data as they can manage to collect) simply needs to change their timezone in some small way, and we have no choice but to give them a zone, after which they can submit the historical (pre 1970) data, and it will be included. They cannot get that in how, as historic data for Sweden is probably not going to be correct for Berlin. This is not something we ought to be encouraging - authorities already make bizarre changes for no seemingly good reasons (to outsiders anyway) we should not be giving them another reason to make even weirder changes, just to satisfy some "rule" that we have imposed. To the extent that we're concerned that even though Berlin is the correct choice for Germany according to the rules, its time history is not accurate for much of the rest of Germany, then there's nothing stopping us treating this as a somewhat special case (which it clearly is), and creating Europe/Fankfurt (or Hamburg, Munich, or ..., I have no idea which city would qualify) to contain what would be a better representation of the data for the bulk of what used to be West Germany. Which one people used for post 1970 timestamps wouldn't matter, but people who want more accurate older data can select which is appropriate. If Berlin is also not accurate for Leipzig (and Frankfurt or whatever isn't either) then we could also create a zone for what used to be East Germany, to contain the historic data for them. Someone would need to do the work to collect the data first of course, just saying "it should happen" isn't enough to cause that to occur. Someone who needs it needs to do the work first. That the data doesn't differ post 1970 does not exclude a new zone, it just does not require that one be created, there's a discretion. Anyone who believes that ever was more than a guideline just needs to look at the zones that are being, and have previously been, merged. If that post 1970 "rule" was a requirement, there would never have been all these post 1970 identical zones to merge. If there's a good enough reason, we can create multiple zones that are identical post 1970, and what's more, we should. Of course, there's some discretion involved - decisions need to be made in each case, rather than simply blindly following some rule, but that's what rational people do - weigh the cost/benefit and decide case by case. kre
On 9/24/21 11:53 AM, Robert Elz via tz wrote:
there's nothing stopping us treating this as a somewhat special case (which it clearly is)
It's not that special (unless by "special" you mean "temporarily commanded by a Red Army general" :-). If you're concerned about pre-1970 timestamps Europe/Paris is wrong for most of France, America/New_York is wrong for most of the US Eastern time zone, etc., etc. I doubt whether it'd be possible even in theory to hash that all out, if one wanted accurate timestamps back to the introduction of standard time. Too many of the old records have been lost. But even if it were possible, it'd be a lot of work and nobody has the time and inclination to do it. We need guidelines that we can realistically follow and that result in an equitable database.
Date: Fri, 24 Sep 2021 13:12:06 -0700 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <f80f3f19-0a5c-ca97-d23c-f08867f9bb50@cs.ucla.edu> | It's not that special (unless by "special" you mean "temporarily | commanded by a Red Army general" :-). There are very few countries in recent history (1900 or after) which have been mashed together out of what was previously two separate countries (even less so from what had previously been one country, formed out of a bunch of smaller states not all that much earlier). If the wall had not come down, we'd have no problems having Berlin and Frankfurt as different zones. That's why it is a rather special case. | If you're concerned about pre-1970 | timestamps Europe/Paris is wrong for most of France, America/New_York is | wrong for most of the US Eastern time zone, etc., etc. I suspect a lot of that depends upon just how far back one goes, and how precise one really needs to be. Certainly the variations caused as armies invade and conquer (first in one direction, then in the other) and the time in the affected region changes gets very messy indeed, and we cannot really hope to model that, for WWII vintage events, nor for more recent (post 1970 ones) - no-one expects different zones for the Crimea region as the Russians took control over however many days or weeks that took, for example. Aside from that kind of issue, standard time is standard time, and the railways, telegraph, (and now airlines) etc, wouldn't really work without it. But like I said, much of this ends up being a judgement call, just how significant is some difference, and how likely is it that anyone now really needs to get things that precisely right, as to whether a new zone ought to be created for historic differences. Something truly significant and long lasting probably deserves it, that one town moved its clocks a day later than its neighbour, almost certainly does not. | I doubt whether it'd be possible even in theory to hash that all out, if | one wanted accurate timestamps back to the introduction of standard | time. Too many of the old records have been lost. In many cases that's probably true, though it often turns out that there are more records than one believes at first glance, it turns out to be quite hard to really make information vanish. But certainly this data can be hard to locate. Fortunately, it's not our problem. | But even if it were possible, it'd be a lot of work and nobody has | the time and inclination to do it. As a mass project, no, of course not. No-one expects that someone set out to collect the detailed time history of the world (though if someone has nothing better to do and is looking for a lifetime project...). However there are people who are interested and do collect it for some region (large or small). There's no reason we need to ignore that resource when it appears, rather we should encourage it. We don't do the work, but we accept contributions. | We need guidelines that we can realistically follow and that | result in an equitable database. Equitable does not mean that we have equal data (coverage) for everyone - it just means that the rules for supplying and accepting data are fair and the same for all. If some regions then get better coverage than others, that's not our problem, rather it is an issue for the areas where no-one cared to contribute (or where record keeping was so shoddy that no-one knows any more) - which is not a problem for us, there is just no contribution. To take another example from your teaching - being equitable doesn't mean that you need to give the same grades to everyone, but rather that the basis for grading be understood and fair to all. Then those who do the work well get better grades than those who do not. All fair and equitable. [ Aside: I know this is a much more complex issue than it seems from this 4 line paragraph, it is just to make a point, so there is no need for everyone to jump in. ] kre
Robert Elz said:
| I see that point, though it depends on how you view pre-1970 data. After | all, the data is already wrong for much of the area covered by | Europe/Berlin and possibly (I wouldn't be surprised either way) much of | the area covered by Europe/Oslo.
It isn't "because they're different countries" but because there was data that is now (effectively) deleted
So, at the risk of putting words into your mouth that weren't there, you want to bring everything from backzone into the main distribution and have a policy of not getting rid of any zone that already exists? That's an arguable position (I don't know at this point if I agree with it) that's worth talking about.
In other words, any authority with the ability to make timezone changes who wants to have their own zone, with their own data (including as much of their own historical data as they can manage to collect) simply needs to change their timezone in some small way, and we have no choice but to give them a zone, after which they can submit the historical (pre 1970) data, and it will be included.
That's always been the case. I've not seen any authority bothering to do that, though - given that they don't bother to tell us about changes until a few days before, we clearly aren't high up on their list of important things to worry about. So I don't think this is a significant risk.
To the extent that we're concerned that even though Berlin is the correct choice for Germany according to the rules, its time history is not accurate for much of the rest of Germany, then there's nothing stopping us treating this as a somewhat special case (which it clearly is),
From another email, it's not just Berlin that's a special case. Just about every zone in the USA is a similar special case because, to paraphrase, until the mid-1960s practically every village had its own date for DST changes.
That the data doesn't differ post 1970 does not exclude a new zone, it just does not require that one be created, there's a discretion.
I can accept that. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Date: Sat, 25 Sep 2021 23:21:50 +0100 From: "Clive D.W. Feather" <clive@davros.org> Message-ID: <YU+g/hDEpa0ytrFV@davros.org> | So, at the risk of putting words into your mouth that weren't there, you | want to bring everything from backzone into the main distribution and have | a policy of not getting rid of any zone that already exists? Apart from the exceptions related to simple renaming when the policy allows that to happen, and removing errors (including typos in zone names, mistaken additions, etc) before they've become stable, then yes, that would be about it. | That's always been the case. I've not seen any authority bothering to do | that, though - given that they don't bother to tell us about changes until | a few days before, we clearly aren't high up on their list of important | things to worry about. So I don't think this is a significant risk. Things are becoming move visible. Until recently only a very small fraction of the world cared in the slightest what we do - but now since everyone's phone depends upon this data, things are becoming more visible. kre
On Sep 24, 2021, at 8:05 AM, Clive D.W. Feather via tz <tz@iana.org> wrote:
I see that point, though it depends on how you view pre-1970 data. After all, the data is already wrong for much of the area covered by Europe/Berlin and possibly (I wouldn't be surprised either way) much of the area covered by Europe/Oslo.
The further back you go from 1970, the more likely that the data for the tzdb region you're in - even if all of backzone is included - is likely to be wrong for your current location. Fixing that would probably require a lot of detailed research, at least for some locations and, even if you don't try to give LMT for every location on earth, could involve creating significantly more tzdb regions. And the further you go back, the more research and the more tzdb regions will be necessary. So if the tzdb were to make an effort to improve the correctness of pre-1970 data over and above undoing all the moves to backzone, the cost - in effort required to get better data for existing regions, effort required to get data for any newly-created regions, effort required to maintain those new regions, *and* effort by at least some consumers of the to use the newly-created regions. It may be that the best pre-1970 data you'll ever get from the tzdb will be data for existing regions, including backzone regions, with whatever fixes are supplied by those interested in improving that data. You may never get, for example, finer-grained data for Germany than you have now, if there is any. So, yes, moving regions to backzone may just be replacing incorrect data with incorrect data, although the data moved to backzone might be "less wrong", if that matters to consumers of pre-1970 data. (BTW, macOS uses at least two relegated-to-backzone tzids - if I set my current city to Montreal, it sets the current tzdb region to America/Montreal, not America/Toronto, and if I set my current city to Asmara, it sets the current tzdb region to Africa/Asmara, not Africa/Nairobi. However, they don't appear to use the backzone data, as the America/Montreal and America/Toronto files are byte-for-byte identical, as are the Africa/Asmara and Africa/Nairobi files. I don't know whether, if I were to travel to Montreal or Asmara, they'd set the tzdb region to America/Montreal or Africa/Asmara, respectively. That depends on the map data they use, and whether it has those regions.)
On 9/22/21 11:32 AM, Brooks Harris via tz wrote:
Olsen's original insight to use towns and cities to name time zones turns out to be extraordinarily useful.
As a historical note, the naming convention you're talking about is something I originated and contributed to tzdb in 1993. Although I began with more-complicated names like Pacific/Fiji/Suva and North_America/US/Los_Angeles, the country names (and the "North_") were more trouble than they were worth, so I removed them before sending patches for Arthur to install. One of my early emails about my idea concluded that "It looks like this will take some time." Which it has. https://mm.icann.org/pipermail/tz/1993-October/009233.html
On Sep 26, 2021, at 5:44 PM, Paul Eggert via tz <tz@iana.org> wrote:
On 9/22/21 11:32 AM, Brooks Harris via tz wrote:
Olsen's original insight to use towns and cities to name time zones turns out to be extraordinarily useful.
As a historical note, the naming convention you're talking about is something I originated and contributed to tzdb in 1993.
Yes - for example, back in 1992, the "Europe" file had these zones: $ git show a544b29c474e03e527df186cd9c33049bce31670:europe | egrep '^Zone' Zone GB-Eire 0:00 GB-Eire %s 1968 Oct 27 1:00s Zone WET 0:00 W-Eur WET%s Zone Iceland 0:00 - WET Zone Portugal 0:00 W-Eur WET%s 1992 Sep 27 1:00s Zone MET 1:00 M-Eur MET%s Zone Poland 1:00 W-Eur MET%s Zone EET 2:00 E-Eur EET%s Zone Turkey 3:00 Turkey EET%s Zone W-SU 3:00 M-Eur ???? (Paul, I think, constructed the tz Git repository, in part, from the original SCCS history from when SCCS was being used by Arthur).
On 9/26/21 6:03 PM, Guy Harris wrote:
(Paul, I think, constructed the tz Git repository, in part, from the original SCCS history from when SCCS was being used by Arthur).
Yes, about a decade ago I wrote some one-off software to convert Arthur's entire SCCS repository into a Git repository before tzcode2012c and tzdata2012d were released. Although we never had competing repositories or a fork or anything like that, a handful of changes were "in the air" during the conversion from SCCS to Git. In the Git log, all commits that were converted from SCCS contain something like "SCCS-file: northamerica" and "SCCS-SID: 8.53" in their commit messages; each of these commits affects just one file because that was the SCCS way, and the SCCS SIDs are file-specific. The last SCCS version corresponds to Git commit 07dbc825cb32f2ae9b7f5d980ba31680a8e92b53, authored 2012-03-03 13:26:39 -0500, and the first Git-only commit is the next one (its parent, in Gitese), commit c0f1e5d9d6c7959f73506e992a0e3ba2b0c5c5e8, authored 2012-03-02 05:21:33 UTC (although this timestamp precedes that of the other commit, that's allowed in Git). Some cleanup commits appear soon after the conversion, interspersed with non-cleanup commits.
International standard doesn't mean world standard. International standard only represent consencus reached by governments around the world, and in TZDB it is not only the governments decided time that are being recorded, but instead it record the time actually used by people for timekeeping on the ground. 在 2021年9月22日週三 21:09,dpatte via tz <tz@iana.org> 寫道:
Iso is a living international standard, and it's their mandate, not the mandate of timezone collectors to do the negotiation and diplomacy in order to arrive at and maintain the world standard.
I certainly don't believe we have the mandate or skillset to ignore the international standard widely used in modern computer technology.
Sent from my Galaxy
-------- Original message -------- From: "Clive D.W. Feather via tz" <tz@iana.org> Date: 2021-09-22 03:34 (GMT-05:00) To: Tony Finch <dot@dotat.at> Cc: tz <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb
Tony Finch said:
Oh, please define "country". Jon Postel deftly answered that question in the 1980s by deferring to ISO 3166,
Except that the ISO 3166 list contained Belorussia and Ukraine, both of which were part of the USSR that was also in the list.
The only reason they were in the list was a 1940s deal to give the USSR three votes in the UN to counter the fact that there were more "western" countries than Warsaw Pact ones.
So there was no genuine logic to that list, just politics.
-- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On Wed, Sep 22, 2021 at 6:09 AM dpatte via tz <tz@iana.org> wrote:
Iso is a living international standard, and it's their mandate, not the mandate of timezone collectors to do the negotiation and diplomacy in order to arrive at and maintain the world standard.
I certainly don't believe we have the mandate or skillset to ignore the international standard widely used in modern computer technology.
What we have done is divide the world into regions based on how people keep time. That this doesn't always distinguish countries is not a problem. One could make an argument that the scale of possible future changes might make it sensible to have countries to properly handle future events, but that is not the argument the country people are making, unless I missed it. What the argument seems to be is that users want their country in the list when picking a timezone, which can be handled by mapping more places to a tzdb defined timezone. That's different from squashing based on post-1970. Sincerely, Watson Ladd
On Tue, 21 Sept 2021 at 20:44, Paul Eggert via tz <tz@iana.org> wrote:
On 9/21/21 12:33 PM, Stephen Colebourne via tz wrote:
The current unreleased state of tzdb is that it favours some countries over others, such as Germany over Sweden/Norway.
It also "favors" some states over others, such as Arizona (which has the America/Phoenix Zone) over Utah (which has no Zone). It's the same argument, and it's an equally invalid argument as far as timekeeping goes.
No, this is patent nonsense. A US state is not of the same importance as a country. Stephen
Stephen Colebourne via tz said:
It also "favors" some states over others, such as Arizona (which has the America/Phoenix Zone) over Utah (which has no Zone). It's the same argument, and it's an equally invalid argument as far as timekeeping goes. No, this is patent nonsense. A US state is not of the same importance as a country.
Have you tried saying that in Texas? Certainly it's the impression we get over here. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
I strongly agree with Stephen here. Every iso country needs at least one slot in the db for their own history back to lmt. Applications have a need for this type of historical data. Should this data be captured in tz, or a fork?Sent from my Galaxy -------- Original message --------From: Stephen Colebourne via tz <tz@iana.org> Date: 2021-09-21 15:34 (GMT-05:00) To: Time zone mailing list <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb On Tue, 21 Sept 2021 at 17:48, Paul Eggert via tz <tz@iana.org> wrote:> This disagreement is not about whether the data in question are> available; it's only about which file they're in. Nothing is being> "wiped out" or refused.It is about more than that. The current unreleased state of tzdb isthat it favours some countries over others, such as Germany overSweden/Norway. It does this to a much greater extent than before. Frommy perspective, that is the root of the problem here.> I think this greatly underestimate the effort to do a complete job. We'd> eventually need thousands of Zones.I would imagine that most readers of this list would not wantthousands of zones. The goal of including every time-zone that hasever been is a straw man.I'm perfectly happy with the post-1970 rule for determining what IDswe have so long as it results in one ID per ISO country. In addition,at least one ID per ISO country is entitled to have full history backto LMT, **even if it is innaccurate**.> [I] am UCLA teachingprofessor who has institutional obligations in the areas of equity,diversity and inclusion...> merely going back to2021a's setup is not something we can or should do, on equity grounds.The patch makes the equity/diversity/inclusion position of tzdb far,far worse. Appreciating this is the first step necessary to resolvingthis.Stephen
Paul Eggert via tz <tz@iana.org> writes:
This disagreement is not about whether the data in question are available; it's only about which file they're in. Nothing is being "wiped out" or refused.
The flaw in that assertion is that while the data may be there, it's no longer possible to extract it in a way that matches prior results. Consumers who value data stability are out in the cold. I just re-ran my experiment comparing the zoneinfo tree generated by 2021a to what I get from current tzdb git. I get the same set of zone files, except for a couple of expected recent changes such as Pacific/Kanton. However: * without backzone, there are 65 zone files/links that change contents compared to 2021a * with backzone, there are 95 zone files/links that change content. It looks like 25 of the changes are shared between the two trees and so probably represent actual recent updates, rather than effects of the May rearrangements. But that still leaves me with a need to change the contents of either 40 or 70 zone files for which there is absolutely no justification other than administrative fiat. I'm damned if I do and damned if I don't, and the only thing that's certain is that I'll have unhappy users. The same conundrum is going to face every tzdb repackager who is unwilling to assume that nobody cares about pre-1970 dates. I think there is going to be a fork, and I think a lot of people are going to adopt it rather than choose which subset of their users to annoy. Of those who don't follow the fork, some large fraction are going to start using backzone, reasoning that adding poorly-attested data for some zones is better than removing somewhat-better-attested data for other zones. That's going to leave us with three competing versions of tzdb, which is absolute disaster. But if you're going to assign negligible weight to data stability, that's what's going to happen, because a lot of us *do* value that. regards, tom lane
On 9/21/21 1:49 PM, Tom Lane wrote:
while the data may be there, it's no longer possible to extract it in a way that matches prior results.
It is possible to extract it, if you use the patches I mentioned earlier today <https://mm.icann.org/pipermail/tz/2021-September/030456.html>. Admittedly this is effectively something of a fork and I don't favor this approach, but it is possible.
Consumers who value data stability are out in the cold.
If you value data stability, then the current tzdb repository is better than going to a one-Zone-per-country rule, because the latter would cause more data churn than the former does, due to all the Links that would turned into Zones that would differ before 1970.
I just re-ran my experiment comparing the zoneinfo tree generated by 2021a to what I get from current tzdb git. I get the same set of zone files, except for a couple of expected recent changes such as Pacific/Kanton. However:
* without backzone, there are 65 zone files/links that change contents compared to 2021a
Yes. This mostly affects only timestamps before 1970.
* with backzone, there are 95 zone files/links that change content.
That also sounds about right. That's the extra data churn I mentioned above. In other words, if the goal is data stability, you're better off with the current development repository, than with a fork that goes to a one-Zone-per-country rule.
It looks like 25 of the changes are shared between the two trees and so probably represent actual recent updates, rather than effects of the May rearrangements.
I am not sure how you're counting, but only 11 files represent updates other than the May rearrangements. Here's the list: America/Barbados America/Guyana Atlantic/Azores Atlantic/Madeira Europe/Lisbon Pacific/Apia Pacific/Enderbury Pacific/Niue Pacific/Rarotonga Pacific/Tongatapu Portugal I got this list by using the abovementioned patches.
I think there is going to be a fork
Any fork won't save you from the before-1970 changes you're concerned about, unless you give up on equity goals.
What equity, diversity or inclusion issues were in 2021a? How does the patch solve them or make the situation better? Is there a roadmap? There were multiple requests about Kiev -> Kyiv rename. And the patch can be interpreted as prioritising some time zones over the others. Won't it cause even more requests of that kind? Sorry if it was already answered, I couldn't find answers to these questions in archives. Thanks, Almaz On Mon, 20 Sept 2021 at 17:16, Paul Eggert via tz <tz@iana.org> wrote:
On 9/20/21 1:06 AM, Stephen Colebourne via tz wrote:
The purpose of the fork would initially be to maintain the tzdb data set as it was prior to the dispute.
Such a fork would arbitrarily discriminate against countries like Angola and Niger, and in favor of countries like Norway and Sweden.
A primary goal of the recent patches was to avoid racial or national preferences that were present in the previous setup. Arguably these preferences were not intentional, or were apparent and not real; however, that's not an argument I would want to defend.
Although the problem of discrimination could also be fixed in the fork you suggest, any such fix would take considerable work. It couldn't be done merely by reverting a patch or two. All this work would be need to be done before any such fork could be used by an organization that is committed to equity, diversity and inclusion.
Instead of forking, I suggest that we work together to address the technical issues raised by the change. I've been attempting to do that with the recent "Revert May patch to zone.tab" patch, which Almaz today said fixes the problems Google had with region-timezone mapping. I hope that we can address remaining technical problems with similar patches.
If the "Revert May patch to zone.tab" patch is not enough, one possible path forward would be to add a build-time option along the lines that I suggested to Tom Lane in <https://mm.icann.org/pipermail/tz/2021-June/030197.html>. This would let projects like PostgreSQL generate a tzdata.zi file containing whatever backzone entries they choose, though I would urge these projects to consider equity, diversity and inclusion issues when they make their choices. I have drafted a patch along these lines and could circulate it if there's interest.
The idea is to commit to an approach that accommodates users who need at least one tzdb entry per country, either via the "Revert May patch to zone.tab" or by further patches if necessary.
These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
None of the information has been "wiped out". It has merely moved from one source file to another, and builds with PACKRATDATA=backzone still install all the info in question. This disagreement is not about "wiping out" info; it's about what category the info falls into.
On 9/22/21 2:53 AM, Almaz Mingaleev wrote:
What equity, diversity or inclusion issues were in 2021a?
Locations in countries like Norway and Sweden got special treatment by being Zones, whereas locations in countries like Angola and Ethiopia were only Links.
How does the patch solve them or make the situation better?
It replaces Zones with Links when Links suffice. That way, Norway etc. are treated like Angola etc.
Is there a roadmap?
The recent patches implement the last part of a long-term process that replaced Zones with Links when Links suffice. The process started in 2013. I worked gradually; I lacked time to do it all at once. The job is done now. There's no reason to make further changes like this.
There were multiple requests about Kiev -> Kyiv rename. The recent patches are orthogonal to the Kiev -> Kyiv rename, because that city has a unique timezone history since 1970 and so gets a Zone under the guidelines, independently of how the Zone is spelled.
Won't it cause even more requests of that kind?
I doubt whether the change will affect the number of name-change requests. The patches did not affect names; they merely changed some Zones to Links.
And the patch can be interpreted as prioritising some time zones over the others.
Only in the sense that the currently-prioritized timezones are those that differ since 1970, which is a timekeeping issue that the tzdb guidelines say is our priority. This is in contrast to 2021a, where some timezones were prioritized for reasons unrelated to timekeeping.
Sorry if it was already answered, I couldn't find answers to these questions in archives.
This stuff is scattered all over the place, unfortunately. It would be helpful if I (or someone else) had the time to write it up. I hope this email is enough in the meantime. PS. I apologize for not answering your email earlier, as well as not replying to everyone else's email. Unfortunately I'm swamped with email now, much of it not tzdb-related. It will take time to consider all the suggestions made recently, but because of Samoa we really need to get a new release out soon and so will have to make do with what we have, somehow.
Date: Thu, 23 Sep 2021 01:52:28 -0700 From: Paul Eggert via tz <tz@iana.org> Message-ID: <cd323514-14cd-245f-0979-2edf91f5b59e@cs.ucla.edu> | Locations in countries like Norway and Sweden got special treatment by | being Zones, whereas locations in countries like Angola and Ethiopia | were only Links. Then make the latter zones instead of links. | It replaces Zones with Links when Links suffice. That way, Norway etc. | are treated like Angola etc. Which could be fixed better the other direction. | The job is done now. There's no reason to make further changes like this. Except that every time one of the zones which is now a link decides to change its timezone rules, it needs to be converted back to a zone. That wouldn't be required if they were left zones in the first place. | Only in the sense that the currently-prioritized timezones are those | that differ since 1970, which is a timekeeping issue that the tzdb | guidelines say is our priority. Priority, yes, but that doesn't mean that nothing before then is relevant. | This is in contrast to 2021a, where some | timezones were prioritized for reasons unrelated to timekeeping. If we stepped back to 2001 or so, and made sure that all the zones which were zones then, continued to be zones now, then we would have none of these problems. All this has been inspired by this unnecessary desire to have less zones, which is something I have never understood. One zone (at least) per authority (nb: not country) should be the goal, as I recall it, that's what we used to have. And to answer Clive's question about whether Wales should have its own timezone, the answer entirely depends upon whether the Welsh parliament (such as it is) has the authority to alter the time in Wales, if it does, then yes, there should be a Europe/Cardiff (my guess as to the largest population centre in Wales, not intended to necessarily be correct). If it does not, then no, not unless the British parliament (or whoever it delegates to make this decision) decides to make the time in Wales differ from the rest of the UK. | but because of Samoa we really need to get a new release out | soon and so will have to make do with what we have, somehow. Yes, 2021a, plus those changes (and anything else similar). No more zone merging. kre
Robert Elz via tz said:
One zone (at least) per authority (nb: not country) should be the goal, as I recall it, that's what we used to have.
Do we know who the authorities are? Three examples: (1) I don't know who the authorities in the USA are. I know that the Secretary of Transport has some kind of authority, but I was under the impression that state governments are also involved in changing what zone a state is in. I know that Arizona seems to have at least three separate authorities. And as for the mess in Indiana ... (2) The EU is the authority for the transition rules throughout the EU, but doesn't select what zone a country is in. That's a national matter. (3) Does Eucla have an authority? Or was it just a few people in the pub that decided to change the clocks and everyone else went along with it? (Does Eucla have a pub? Surely it does - it's in Oz.)
And to answer Clive's question about whether Wales should have its own timezone, the answer entirely depends upon whether the Welsh parliament (such as it is) has the authority to alter the time in Wales,
So far it doesn't. And you've missed my point, which is that people (e.g. the Welsh) don't accept 3166 as the authoritative list of countries for domain names, so why would they for time zones. Which is why "at least one zone per 3166 'country'" isn't automatically going to prevent political arguments. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
The Department of Transportation sets the boundaries of the time zones, but states (and tribes) can opt out of daylight savings time. Envoyé de mon iPhone
Le 23 sept. 2021 à 08:20, Clive D.W. Feather via tz <tz@iana.org> a écrit :
Robert Elz via tz said:
One zone (at least) per authority (nb: not country) should be the goal, as I recall it, that's what we used to have.
Do we know who the authorities are?
Three examples:
(1) I don't know who the authorities in the USA are. I know that the Secretary of Transport has some kind of authority, but I was under the impression that state governments are also involved in changing what zone a state is in. I know that Arizona seems to have at least three separate authorities. And as for the mess in Indiana ...
(2) The EU is the authority for the transition rules throughout the EU, but doesn't select what zone a country is in. That's a national matter.
(3) Does Eucla have an authority? Or was it just a few people in the pub that decided to change the clocks and everyone else went along with it?
(Does Eucla have a pub? Surely it does - it's in Oz.)
And to answer Clive's question about whether Wales should have its own timezone, the answer entirely depends upon whether the Welsh parliament (such as it is) has the authority to alter the time in Wales,
So far it doesn't.
And you've missed my point, which is that people (e.g. the Welsh) don't accept 3166 as the authoritative list of countries for domain names, so why would they for time zones. Which is why "at least one zone per 3166 'country'" isn't automatically going to prevent political arguments.
-- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On 2021-09-23 10:47 AM, J Andrew Lipscomb via tz wrote:
The Department of Transportation sets the boundaries of the time zones, but states (and tribes) can opt out of daylight savings time.
Envoyé de mon iPhone U.S. DEPARTMENT OF TRANSPORTATION time zones https://www.transportation.gov/tags/time-zones
Date: Thu, 23 Sep 2021 13:19:15 +0100 From: "Clive D.W. Feather" <clive@davros.org> Message-ID: <YUxwwy90bZHzrtrU@davros.org> | Do we know who the authorities are? No, but we don't need to in advance - when someone demonstrates that some authority is changing the time in some region, that shows the existence of such an authority. | (1) I don't know who the authorities in the USA are. Nor do I, but someone does. Probably many someones. My believe from what I have seen (mostly here) though is that the states don't get to make decisions themselves, but can make requests. | (2) The EU is the authority for the transition rules throughout the EU, but | doesn't select what zone a country is in. That's a national matter. Yes, that's what I understood too, and so each country should have at least one zone. It doesn't matter if (for example) San Marino has never had times different from Rome (even if that goes back hundreds of years). (I also have no idea if San Marino is an EU member, I'm just guessing.) | (3) Does Eucla have an authority? Or was it just a few people in the pub | that decided to change the clocks and everyone else went along with it? That is the authority. No-one ever said it needs a legislative basis, just that it decides what the clocks should be set to, and that actually takes effect. How (or where) they decided to split the difference between SA and WA times I have no idea (though it does make sense). | (Does Eucla have a pub? Surely it does - it's in Oz.) I would assume so, but I have never been there. I know it isn't very big though. | And you've missed my point, which is that people (e.g. the Welsh) don't | accept 3166 as the authoritative list of countries for domain names, No, I didn't miss that, I understood what you were saying, my point was that, IMO anyway, the answer to that is irrelevant to time zones. What is and what is not a country is irrelevant - which is why the exact status of Palestine doesn't matter to us, it has its own time zone, with its own rules, and someone is deciding what they should be. That is enough. Similarly, in AU, it is the states that decide on timezones, summer time rules, etc, not the commonwealth govt - each state is an authority. And to answer a point that Paul made in response to a much earlier message of mine (a few days ago, so hundreds of messages back...) the authorities that matter are those that exist now (and into the future) - that is, those who can decide, now, to alter what the time is. What various towns might have done a couple of hundred years ago is only relevant if they're still doing that today. So Eucla counts, Sacramento doesn't. kre
On Thu, 23 Sept 2021 at 09:52, Paul Eggert via tz <tz@iana.org> wrote:
On 9/22/21 2:53 AM, Almaz Mingaleev wrote:
What equity, diversity or inclusion issues were in 2021a?
Locations in countries like Norway and Sweden got special treatment by being Zones, whereas locations in countries like Angola and Ethiopia were only Links.
Taken at face value, it is easy for anyone reading this to nod their head and agree that Norway should not receive favouritism over Angola. What it fails to state is that the solution adopted (creating Links where previously there were Zones) has created a *different* equity, diversity or inclusion issue. The resulting tzdb state clearly favours Berlin over Oslo or Stockholm, Brussels over Amsterdam. The justification for this new inequality is arcane to anyone who doesn't understand the nuances of the tzdb system. Sure there is a rule involved (single regions for post 1970 data, largest city for pre-1970 data). But the rule simply won't be understood or accepted by most actual users of the data set. Stephen
Stephen Colebourne via tz said:
Taken at face value, it is easy for anyone reading this to nod their head and agree that Norway should not receive favouritism over Angola.
What it fails to state is that the solution adopted (creating Links where previously there were Zones) has created a *different* equity, diversity or inclusion issue. The resulting tzdb state clearly favours Berlin over Oslo or Stockholm, Brussels over Amsterdam.
The justification for this new inequality is arcane to anyone who doesn't understand the nuances of the tzdb system. Sure there is a rule involved (single regions for post 1970 data, largest city for pre-1970 data). But the rule simply won't be understood or accepted by most actual users of the data set.
But TZDB already favours London over Bristol, Paris over Lyons, Amsterdam over Den Haag, Berlin over Frankfurt, Los Angeles over Seattle, and so on. Without using the word "country", please explain the difference. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
"Clive D.W. Feather via tz" <tz@iana.org> writes:
But TZDB already favours London over Bristol, Paris over Lyons, Amsterdam over Den Haag, Berlin over Frankfurt, Los Angeles over Seattle, and so on. Without using the word "country", please explain the difference.
Why exactly should the answer to that not involve the word "country"? Timekeeping laws are generally set at the country level. Ignoring that fact isn't going to help you arrive at sane conclusions about how to manage tzdb. regards, tom lane
On 9/23/21 7:37 AM, Tom Lane via tz wrote:
Timekeeping laws are generally set at the country level.
I wouldn't put it that way. The EU sets timekeeping rules that are binding across member states. Although members have some flexibility, so far it's been very unlikely, for example, that Sweden would diverge from Germany. (This may of course change in the future, in which case we'll of course deal with it.) Conversely, although the US federal government sets overall rules, it devolves DST decisions to states. If the rule were "at least one Zone per political unit that has the legal power to set its own rules", we'd have dozens more Zones than we do now, Zones that would cause more trouble than they'd cure. Although most countries do set timekeeping laws at the country level, counterexamples are so numerous and significant that it's too much to say that they're generally set that way.
Tom Lane said:
"Clive D.W. Feather via tz" <tz@iana.org> writes:
But TZDB already favours London over Bristol, Paris over Lyons, Amsterdam over Den Haag, Berlin over Frankfurt, Los Angeles over Seattle, and so on. Without using the word "country", please explain the difference.
Why exactly should the answer to that not involve the word "country"?
Because countries aren't part of the present TZDB policies.
Timekeeping laws are generally set at the country level.
Though the ones affecting Oslo and Berlin are made at another level, meaning they're both affected by the same authority. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Date: Thu, 23 Sep 2021 14:14:33 +0100 From: "Clive D.W. Feather via tz" <tz@iana.org> Message-ID: <YUx9uVqhCngF4JXv@davros.org> | But TZDB already favours London over Bristol, Bristol doesn't get to decide what time it sets the clocks to, so unless they start doing that (they could, though the British govt probably would not like it) or it is done for them, they're subservient to the UK time authority, and that one we call Europe/London (and would do so even if the ministry that recommends these things happened to have its offices in Birmingham). The Netherlands, Norway, Sweden, etc, all have the authority to choose their own time zone, unless the EU parliament decided to remove that choice from them (and they accepted that), so they ought be separate zones (the names those take are established by the tzdb rules, which aren't of relevance right now). kre
On 9/23/21 6:07 AM, Stephen Colebourne via tz wrote:
the solution adopted (creating Links where previously there were Zones) has created a *different* equity, diversity or inclusion issue.
That's not a new issue. We've dealt with it many times, by dealing with questions like "How come there's no Asia/Beijing Zone?" When the issue comes up we point to the guidelines, which are based on timekeeping rather than traditional political issues. Although we can't expect the guidelines to be familiar in advance to everyone who asks such questions, explaining the guidelines hasn't been much of a problem in practice.
Speaking of Angola: Africa/Luanda was moved to backward in [1]. The commit message says that it's due to "now-abandoned guideline that every ISO 3166 code should have a zone". Ethiopia: Africa/Addis_Ababa was moved to backward in [2]. NEWS file in it tells "Some more zones have been turned into links, when they differed from existing zones only for older time stamps". Using repo history only I get the impression that in 2021 you merge zones because they were merged in 2014. I don't understand how that process is related to (or improves) equity, diversity or inclusion issues. What do I miss here?
It replaces Zones with Links when Links suffice. That way, Norway etc. are treated like Angola etc.
The reasons mentioned in commit messages do not apply to Norway.
The job is done now.
Sorry, I was unclear. What I meant was what is overall goal of this refactoring. I guess steps are not that important now.
The recent patches are orthogonal to the Kiev -> Kyiv rename
Yes, they are. But I think it still might raise concern from users. tl;dr: Sorry, I've started to follow tz mailing list only recently, I don't know all the historic decisions. But looking to commit messages history only, I get a weird impression around recent changes. [1] https://github.com/eggert/tz/commit/2a18a625a0171e5c54249a54179eda3409b1b838... [2] https://github.com/eggert/tz/commit/6f6f20f0bc739ce3dfd503700b793c5e459e309d On Thu, 23 Sept 2021 at 09:52, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 9/22/21 2:53 AM, Almaz Mingaleev wrote:
What equity, diversity or inclusion issues were in 2021a?
Locations in countries like Norway and Sweden got special treatment by being Zones, whereas locations in countries like Angola and Ethiopia were only Links.
How does the patch solve them or make the situation better?
It replaces Zones with Links when Links suffice. That way, Norway etc. are treated like Angola etc.
Is there a roadmap?
The recent patches implement the last part of a long-term process that replaced Zones with Links when Links suffice. The process started in 2013. I worked gradually; I lacked time to do it all at once.
The job is done now. There's no reason to make further changes like this.
There were multiple requests about Kiev -> Kyiv rename. The recent patches are orthogonal to the Kiev -> Kyiv rename, because that city has a unique timezone history since 1970 and so gets a Zone under the guidelines, independently of how the Zone is spelled.
Won't it cause even more requests of that kind?
I doubt whether the change will affect the number of name-change requests. The patches did not affect names; they merely changed some Zones to Links.
And the patch can be interpreted as prioritising some time zones over the others.
Only in the sense that the currently-prioritized timezones are those that differ since 1970, which is a timekeeping issue that the tzdb guidelines say is our priority. This is in contrast to 2021a, where some timezones were prioritized for reasons unrelated to timekeeping.
Sorry if it was already answered, I couldn't find answers to these questions in archives.
This stuff is scattered all over the place, unfortunately. It would be helpful if I (or someone else) had the time to write it up. I hope this email is enough in the meantime.
PS. I apologize for not answering your email earlier, as well as not replying to everyone else's email. Unfortunately I'm swamped with email now, much of it not tzdb-related. It will take time to consider all the suggestions made recently, but because of Samoa we really need to get a new release out soon and so will have to make do with what we have, somehow.
On 9/24/21 9:35 AM, Almaz Mingaleev wrote:
Using repo history only I get the impression that in 2021 you merge zones because they were merged in 2014.
Yes, the 2021 alike-since-1970 merges were part of a process that began in 2013 or so. The reasons for that process were discussed back then; you can dig them out of the mailing list and they should help answer some of the other questions in your email. Those reasons were not primarily based on equity per se (sorry, don't recall details offhand). What's happened since 2013, unfortunately, is that I am both busy and forgetful, and after getting about two-thirds of the way through the process over the next few years, I forgot to finish it. The result is that we have 2021a where about 30 Zones (notably the much-discussed Europe/Oslo and Europe/Stockholm) currently have "preferred" status, whereas the other roughly 350 Zones are following the guidelines. This situation creates the appearance (and some would say the reality) of national or ethnic preferences in tzdb. This equity issue was raised early this year, and my recent alike-since-1970 changes were in response to that. Basically they reply to the complaint by saying, "You're right! I messed up. Here's the fix that will make tzdb fairer again."
On Sep 24, 2021, at 9:35 AM, Almaz Mingaleev via tz <tz@iana.org> wrote:
Speaking of Angola: Africa/Luanda was moved to backward in [1]. The commit message says that it's due to "now-abandoned guideline that every ISO 3166 code should have a zone". Ethiopia: Africa/Addis_Ababa was moved to backward in [2].
Presumably you meant "moved to backzone". "backward" "provides links between current names for timezones and their old names", and contains no data, just Link lines. backward does have Link Africa/Nairobi Africa/Asmera Africa/Asmera would be a link even if we brought all backzone entries back to the main database - it'd be a link to Africa/Asmara. I don't know whether Link works transitively, so I don't know whether Link Africa/Asmara Africa/Asmera would work, given that there's also Link Africa/Asmara Africa/Asmera in the africa file.
On Sep 20, 2021, at 12:15 PM, Paul Eggert via tz <tz@iana.org> wrote:
A primary goal of the recent patches was to avoid racial or national preferences that were present in the previous setup.
I applaud the motivation to make these changes. But is it really ethical to knowingly introduce incorrect information into the default configuration, where previously the information was correct? Even if said information is “historic” or even “not officially supported”? You have billions of consumers. If you ship it, people are using it, even if they represent comparatively small numbers. If I’m understanding things correctly, a non-zero amount of people previously consuming correct information will silently start consuming incorrect information after upgrading to a new tzdb version — and we’re doing that on purpose. Howard
On Sep 22, 2021, at 4:49 PM, Howard Hinnant via tz <tz@iana.org> wrote:
I applaud the motivation to make these changes. But is it really ethical to knowingly introduce incorrect information into the default configuration, where previously the information was correct? Even if said information is “historic” or even “not officially supported”?
As per my mail, is the issue here that the Europe/Oslo information, for example, is believed to be correct, whereas the America/Montreal data, apparently moved to backzone in 2015, is *not* believed to be correct?
On Sep 22, 2021, at 8:00 PM, Guy Harris <gharris@sonic.net> wrote:
On Sep 22, 2021, at 4:49 PM, Howard Hinnant via tz <tz@iana.org> wrote:
I applaud the motivation to make these changes. But is it really ethical to knowingly introduce incorrect information into the default configuration, where previously the information was correct? Even if said information is “historic” or even “not officially supported”?
As per my mail, is the issue here that the Europe/Oslo information, for example, is believed to be correct, whereas the America/Montreal data, apparently moved to backzone in 2015, is *not* believed to be correct?
I believe we’re discussing 2021b, not 2015x. If we made a mistake in 2015, we should fix it, not use it as a justification to make the same mistake again today. Howard
On Sep 22, 2021, at 5:03 PM, Howard Hinnant <howard.hinnant@gmail.com> wrote:
On Sep 22, 2021, at 8:00 PM, Guy Harris <gharris@sonic.net> wrote:
On Sep 22, 2021, at 4:49 PM, Howard Hinnant via tz <tz@iana.org> wrote:
I applaud the motivation to make these changes. But is it really ethical to knowingly introduce incorrect information into the default configuration, where previously the information was correct? Even if said information is “historic” or even “not officially supported”?
As per my mail, is the issue here that the Europe/Oslo information, for example, is believed to be correct, whereas the America/Montreal data, apparently moved to backzone in 2015, is *not* believed to be correct?
I believe we’re discussing 2021b, not 2015x.
I'm *mentioning* 2015c to indicate that it's not as if this is the first time we've moved something to backzone.
If we made a mistake in 2015, we should fix it, not use it as a justification to make the same mistake again today.
That'a an "if". I can see several views here on what should be done: "We should never move anything to backzone, we should never have moved anything to backzone, and we should not only not move Europe/Oslo to backzone now, we should undo all the moves we made to backzone." "We should never move anything to backzone, we should never have moved anything to backzone, but what's done is done, so we should not move Europe/Oslo to backzone but we should leave the existing stuff in backzone there and, perhaps, if it's either shown to be accurate or updated to be accurate, move it from backzone to the main database." "We should only move to backzone data we don't believe to be accurate, so we shouldn't move Europe/Oslo to backzone but we should leave the existing stuff in backzone there and, perhaps, if it's either shown to be accurate or updated to be accurate, move it from backzone to the main database." "We should move to backzone all zones that only differ prior to 1970." There may be more. (I don't hold any of those views at all firmly. My personal preference for what to do in the short term is "put out 2021a plus fixes such as Samoa as 2021b, don't put out the results of the merger yet, and discuss this further to see which of those, if any, should be adopted as the new policy, or pick a policy that's not in the list if that's where we end up".)
On Sep 22, 2021, at 8:18 PM, Guy Harris <gharris@sonic.net> wrote:
(I don't hold any of those views at all firmly. My personal preference for what to do in the short term is "put out 2021a plus fixes such as Samoa as 2021b, don't put out the results of the merger yet, and discuss this further to see which of those, if any, should be adopted as the new policy, or pick a policy that's not in the list if that's where we end up".)
Here we are firmly agreed. I sincerely hope that the TZ Coordinator will take this path. Howard
I also agree this is a first good step. But I still would like to know whether data in back zone can be modified in the future when new data comes in.Sent from my Galaxy -------- Original message --------From: Howard Hinnant via tz <tz@iana.org> Date: 2021-09-22 20:40 (GMT-05:00) To: Guy Harris <gharris@sonic.net> Cc: Stephen Colebourne <scolebourne@joda.org>, Time Zone Mailing List <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb On Sep 22, 2021, at 8:18 PM, Guy Harris <gharris@sonic.net> wrote:> > (I don't hold any of those views at all firmly. My personal preference for what to do in the short term is "put out 2021a plus fixes such as Samoa as 2021b, don't put out the results of the merger yet, and discuss this further to see which of those, if any, should be adopted as the new policy, or pick a policy that's not in the list if that's where we end up".)Here we are firmly agreed. I sincerely hope that the TZ Coordinator will take this path.Howard
Howard Hinnant via tz said:
A primary goal of the recent patches was to avoid racial or national preferences that were present in the previous setup.
I applaud the motivation to make these changes. But is it really ethical to knowingly introduce incorrect information into the default configuration, where previously the information was correct? Even if said information is ???historic??? or even ???not officially supported????
My understanding of the default setup is: * If two places had different times at any point since 1970, they must have separate zones. * If two places had the same time at all points since 1970, they do not have to have the same zone even if they differed before 1970. * The data for a zone applies to the whole zone for times after 1970, but might only apply to part of the zone before 1970. That might not be the best arrangement but, if I'm understanding correctly, it's what we've got. Based on that, then an alien observer would say that the area currently covered by the nation-states of Germany, Norway, and Sweden is allowed to be a single zone. And, if we were creating the TZDB from new right now, then under those rules plus the desire to minimize the number of zones, they would be a single zone. The data in them is *not* incorrect; it meets the policy described above. Therefore any argument that they should be two or three zones instead needs to be based on some *other* objective requirement. Which, I suspect, needs to be *added* to the present policies rather than just handwaved. Such requirements could be: * Never abolish a zone once it's got into the default setup. * At least one zone per 3166 "country". * At least one zone per "authority". * Use separate zones if we have reliable data going back to 1945 (or pick some other year). -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On Thu, Sep 23, 2021 at 8:30 AM Clive D.W. Feather via tz <tz@iana.org> wrote:
Therefore any argument that they should be two or three zones instead needs to be based on some *other* objective requirement. Which, I suspect, needs to be *added* to the present policies rather than just handwaved. Such requirements could be: * Never abolish a zone once it's got into the default setup.
I could live with this. My chief concerns are (a) we should not arbitrarily make data that are believed to be reliable more difficult to access; (b) we should not surprise users by suddenly making them see different names for their home time zones. (For a Norwegian suddenly to have 'Berlin' instead of 'Oslo' appear is not acceptable.) Could we establish the rule, "don't abolish zones once released, even if they are subsequently discovered to be duplicative"? I understand the desire to get rid of data that are known to be incorrect or at least unreliable. I have no objection, for instance, to removing pre-1970 data from Montréal if it appears that Shanks compiled incorrect information. While that might change the interpretation of historical dates to be a proleptic version of whatever the situation was in 1970, that at least keeps things compatible, and is no different in effect from a correction to the historical information.
* At least one zone per 3166 "country".
I don't see this as necessary, but it's Mostly Harmless. Links are cheap. It still would run afoul of similar problems to the one we are facing, because it's entirely possible that a zone with deleted data will turn out to be in a country with multiple zones.
* At least one zone per "authority".
This one is almost certainly unworkable, since it puts whoever maintains tzdb in the position of determining who is a competent authority. We have known cases where a region with an ethnic, religious or political minority maintains a different time zone in defiance of a central authority, or where there is a time zone maintained by a government whose legitimacy is disputed. Moreover, there is a definite problem with multiplication of authorities (see below).
* Use separate zones if we have reliable data going back to 1945 (or pick some other year).
From a US-centric perspective, the choice of 1970 was not exactly an accident. If you go back much farther, you run into tremendous Balkanization of local authorities. I can recall one archivist that I knew regaling me with the story of the time in the late 1960's that he had to endure no fewer than seven time zone changes in the course of a 45-minute bus journey, because individual towns and counties in West Virginia and Ohio had decreed different dates for the Daylight Saving Time transition. The date could readily be pushed back to 1967. Prior to that, what we would have would be valid only for individual municipalities; :America/New_York would not have the same rules as a hypothetical :America/West_Virginia/Charleston. (Charleston in turn would not have the same rules as Wheeling, Moundsville or Morgantown. It was an unbelievable mess back then.
-- 73 de ke9tv/2, Kevin
Kevin Kenny via tz <tz@iana.org> writes:
I could live with this. My chief concerns are (a) we should not arbitrarily make data that are believed to be reliable more difficult to access; (b) we should not surprise users by suddenly making them see different names for their home time zones. (For a Norwegian suddenly to have 'Berlin' instead of 'Oslo' appear is not acceptable.) Could we establish the rule, "don't abolish zones once released, even if they are subsequently discovered to be duplicative"?
I think that rule already exists --- the problem is that instead of removing zone names altogether, we're willing to make them misleading by linking them to data for some nearby location (for possibly large values of "nearby"). It's that practice that needs to go, in one way or another. regards, tom lane
Kevin Kenny said:
On Thu, Sep 23, 2021 at 8:30 AM Clive D.W. Feather via tz <[1]tz@iana.org> wrote:
Therefore any argument that they should be two or three zones instead needs to be based on some *other* objective requirement. Which, I suspect, needs to be *added* to the present policies rather than just handwaved. Such requirements could be: * Never abolish a zone once it's got into the default setup.
I could live with this. [...]
Just to be clear, I'm not arguing for or against any of the possible requirements I listed. I was just making the point that they're *new* requirements and need to have group consensus, or at least solid group discussion, before being adopted. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone#general-1 I plea for stability. That doesn’t mean no changes. But it does mean no drastic changes. Howard
On Mon, Sep 20, 2021 at 9:51 AM Howard Hinnant via tz <tz@iana.org> wrote:
The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone#general-1
I plea for stability. That doesn’t mean no changes. But it does mean no drastic changes.
Unfortunately you are held hostage to the wishes of British Canadian chicken farmers, Indiana counties, etc, etc. in determining what the zones will be. The set of zones may increase due to post-1970 changes in ways that we cannot predict. And the C++20 standard thankfully does not include the list of zones in the API. The objections to removal of the pre-1970 data (which affects very few users on computers) seem to be about ancillary effects on the number of zones, and I think the discussion should be about the zone names themselves. The TZ database has never had a one name per country at least policy. Sincerely, Watson Ladd
Howard
On Sep 20, 2021, at 7:23 PM, Watson Ladd <watson@cloudflare.com> wrote:
On Mon, Sep 20, 2021 at 9:51 AM Howard Hinnant via tz <tz@iana.org> wrote:
The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone#general-1
I plea for stability. That doesn’t mean no changes. But it does mean no drastic changes.
Unfortunately you are held hostage to the wishes of British Canadian chicken farmers, Indiana counties, etc, etc. in determining what the zones will be.
Adding new zones is not problematic. Removing or changing the names of existing ones is.
The set of zones may increase due to post-1970 changes in ways that we cannot predict. And the C++20 standard thankfully does not include the list of zones in the API.
A C++20 programmer can easily obtain the list of time zone names and link names from the already standardized API: #include <chrono> #include <iostream> int main() { for (auto& tz : std::chrono::get_tzdb().zones) std::cout << tz.name() << '\n'; for (auto& link : std::chrono::get_tzdb().links) std::cout << link.name() << '\n'; } Current output: Africa/Abidjan Africa/Accra Africa/Algiers Africa/Bissau ... Howard
Maybe you are late to the game, but there was no reliable source of historical timezone data except for tz. This source of historical data is unfortunately, step by step, being scrubbed in tz.Numerous applications for historical research (astronomy, astrology, family history, general recent history since the 1800s) depend on having historical tz data, even pre-1970 data.If tz is not going to be the repository for this data, then tz must be formed so that this historical data has a place to be captured.When I read an astronomical journal and read of a bolide sighting in 1957 at 7:30 PM in haiti, it is important that I can attach that to a ut in 1957. But if it's now in the 'merged data' I am now out of luck.This is my biggest issue (of several) with the most recent changes.Sent from my Galaxy -------- Original message --------From: Watson Ladd via tz <tz@iana.org> Date: 2021-09-20 19:23 (GMT-05:00) To: Howard Hinnant <howard.hinnant@gmail.com> Cc: Time Zone Mailing List <tz@iana.org> Subject: Re: [tz] Preparing to fork tzdb On Mon, Sep 20, 2021 at 9:51 AM Howard Hinnant via tz <tz@iana.org> wrote:>> The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone#general-1>> I plea for stability. That doesn’t mean no changes. But it does mean no drastic changes.Unfortunately you are held hostage to the wishes of British Canadianchicken farmers, Indiana counties, etc, etc. in determining what thezones will be. The set of zones may increase due to post-1970 changesin ways that we cannot predict. And the C++20 standard thankfully doesnot include the list of zones in the API.The objections to removal of the pre-1970 data (which affects very fewusers on computers) seem to be about ancillary effects on the numberof zones, and I think the discussion should be about the zone namesthemselves. The TZ database has never had a one name per country atleast policy.Sincerely,Watson Ladd>> Howard>
On Sep 20, 2021, at 10:57 PM, dpatte <dpatte@relativedata.com> wrote:
When I read an astronomical journal and read of a bolide sighting in 1957 at 7:30 PM in haiti, it is important that I can attach that to a ut in 1957. But if it's now in the 'merged data' I am now out of luck.
In C++-speak: #include <chrono> #include <iostream> int main() { using namespace std; using namespace std::chrono; zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n'; } Output: 1957-06-06 00:30:10 TAI Note that programmers are hard-coding the name "America/Port-au-Prince”. Howard
On Mon 2021-09-20T23:34:25-0400 Howard Hinnant via tz hath writ:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
Output:
1957-06-06 00:30:10 TAI
anything that produces a value of TAI for a date in 1957 has created a fantasy world that does not correspond to reality. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m
On Sep 21, 2021, at 1:49 AM, Steve Allen via tz <tz@iana.org> wrote:
On Mon 2021-09-20T23:34:25-0400 Howard Hinnant via tz hath writ:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
Output:
1957-06-06 00:30:10 TAI
anything that produces a value of TAI for a date in 1957 has created a fantasy world that does not correspond to reality.
After I hit send I thought I may have gone a step too far in converting to TAI. Leaving that conversion out simplifies things and does not negate my point: that IANA time zone names are part of the C++20 API, and pre-1970 data is expected (best effort of course): zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", zt.get_sys_time()) << '\n'; Output: 1957-06-06 00:30:00 UTC
On Tuesday 2021-09-21 15:20, Howard Hinnant via tz wrote:
On Mon 2021-09-20T23:34:25-0400 Howard Hinnant via tz hath writ:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
[...] IANA time zone names are part of the C++20 API, and pre-1970 data is expected (best effort of course):
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", zt.get_sys_time()) << '\n';
I can't really find substance to this claim. The API [speaking of the locate_zone() function] wants a string, but it does not specify which particular string value(s) is/are needed to get any particular zone. locate_zone can succeed or it can fail. In that regard, it is similar to openssl's EVP_getdigestbyname("MD5") Speaking of openssl, if the C++ standard had wanted to encode timezones in the API, they would have probably done it like openssl, as a function or object: EVP_DigestInit(somectx, EVP_md5());
On Sep 21, 2021, at 10:29 AM, Jan Engelhardt <jengelh@inai.de> wrote:
On Tuesday 2021-09-21 15:20, Howard Hinnant via tz wrote:
On Mon 2021-09-20T23:34:25-0400 Howard Hinnant via tz hath writ:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
[...] IANA time zone names are part of the C++20 API, and pre-1970 data is expected (best effort of course):
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", zt.get_sys_time()) << '\n';
I can't really find substance to this claim.
The API [speaking of the locate_zone() function] wants a string, but it does not specify which particular string value(s) is/are needed to get any particular zone. locate_zone can succeed or it can fail.
This issue was discussed in committee, and is a most delicate one (to get the wording right). In the end we believed that this sentence was the best way to describe the set of time zone names: http://eel.is/c++draft/time.zone#general-1
[time.zone] describes an interface for accessing the IANA Time Zone Database that interoperates with sys_time and local_time. This interface provides time zone support to both the civil calendar types ([time.cal]) and to user-defined calendars.
We also added a link the Bibliography section, which is unfortunately not included in the online document:
* IANA Time Zone Database. Available from: https://www.iana.org/time-zones
I was there, and I authored and championed this proposal through the standardization process. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0355r7.html Visual Studio is already shipping it. Howard
On Tuesday 2021-09-21 16:45, Howard Hinnant wrote:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
[...] IANA time zone names are part of the C++20 API, and pre-1970 data is expected (best effort of course)
I can't really find substance to this claim. The API [speaking of the locate_zone() function] wants a string, but it does not specify which particular string value(s)
In the end we believed that this sentence was the best way to describe the set of time zone names:
http://eel.is/c++draft/time.zone#general-1
[time.zone] describes an interface for accessing the IANA Time Zone Database [...]
With a standard wording so loose, the best interpretation I can give you is that the minimum set of known names in C++20 is that of tzdata1996l. Which may be lacking America/Port-au-Prince. (Probably not, but humour me.)
On Sep 21, 2021, at 11:15 AM, Jan Engelhardt <jengelh@inai.de> wrote:
On Tuesday 2021-09-21 16:45, Howard Hinnant wrote:
zoned_time zt{"America/Port-au-Prince", local_days{1957y/June/5} + 19h + 30min}; cout << format("%F %T %Z", clock_cast<tai_clock>(zt.get_sys_time())) << '\n';
[...] IANA time zone names are part of the C++20 API, and pre-1970 data is expected (best effort of course)
I can't really find substance to this claim. The API [speaking of the locate_zone() function] wants a string, but it does not specify which particular string value(s)
In the end we believed that this sentence was the best way to describe the set of time zone names:
http://eel.is/c++draft/time.zone#general-1
[time.zone] describes an interface for accessing the IANA Time Zone Database [...]
With a standard wording so loose, the best interpretation I can give you is that the minimum set of known names in C++20 is that of tzdata1996l.
Here are detailed instructions for making suggestions to improve the wording of the C++ standard. Everyone is welcome to contribute: https://cplusplus.github.io/LWG/lwg-active.html#submit_issue
Which may be lacking America/Port-au-Prince. (Probably not, but humour me.)
The C++20 spec also has API to inspect the version of the time zone database. Howard
On Sep 20, 2021, at 7:57 PM, dpatte via tz <tz@iana.org> wrote:
When I read an astronomical journal and read of a bolide sighting in 1957 at 7:30 PM in haiti, it is important that I can attach that to a ut in 1957. But if it's now in the 'merged data' I am now out of luck.
As long as it's well established that Haiti didn't observe DST in 1957, as the current northamerica file claims is the case, that works. If it's not, well.... (I.e., how "reliable" data for a given tzdb region and a given time in the past is probably depends on how far back in the past it is and how well offset and time-shifting rule changes are documented - and what sources people have checked.)
The current tzdb only have separate zone for data up to 1970, other than some exceptions Forking to include data from earlier times need to considering willingness of general data user to switch over with a larger-sized database and less stable[due to foreseeable upcoming contribution to historical timezone] as well as less accurate data from the past, especially when the number of computer applications and OS in modern time need to handle files with timestamps before 1970 is probably relatively less. It was not without precedent that data that were same after 1970 being moved away from the main tz file in tz db, with examples including Chinese timezones except Xinjiang, as well as Japanese Southwestern Islands timezone. 在 2021年9月21日週二 13:43,Guy Harris via tz <tz@iana.org> 寫道: > On Sep 20, 2021, at 7:57 PM, dpatte via tz <tz@iana.org> wrote: > > > When I read an astronomical journal and read of a bolide sighting in > 1957 at 7:30 PM in haiti, it is important that I can attach that to a ut in > 1957. But if it's now in the 'merged data' I am now out of luck. > > As long as it's well established that Haiti didn't observe DST in 1957, as > the current northamerica file claims is the case, that works. > > If it's not, well.... > > (I.e., how "reliable" data for a given tzdb region and a given time in the > past is probably depends on how far back in the past it is and how well > offset and time-shifting rule changes are documented - and what sources > people have checked.) >
On 9/20/21 9:51 AM, Howard Hinnant via tz wrote:
The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone [Retitling since the C++20 time library is a different topic.]
Here are comments about that draft. Hope they help. Unfortunately I got only about halfway thru the draft before running out of time. * Although the draft distinguishes Zones from Links, such a distinction is not available on platforms that use a filesystem's hard links to implement Links. For example, on Solaris 10: $ cd /usr/share/lib/zoneinfo $ ls -li Asia/Chongqing Asia/Chungking Asia/Harbin Asia/Shanghai PRC 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Chongqing 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Chungking 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Harbin 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Shanghai 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 PRC with no indication that Asia/Shanghai is the Zone and the other names Links. If there's a reason that the draft distinguishes Zones from Links, I suggest explaining the reason, and saying what implementations should do when the distinction between Zones and Links is not available. Another possibility is to remove the distinction between Zones and Links. * Minor wording nit: in the "remote_version" description, change "The latest remote database version" to "The current remote database version". It's possible that the current remote version is a downgrade or sidegrade from the local database version; the API still works in this case. * Exception classes. I see a problem with leap seconds here. These classes and their related operations seem to be designed only for local timestamps that are ambiguous and/or missing due to UTC offset changes, such as switching from standard to daylight-saging time. However, timestamps can also be ambiguous and/or missing due to leap seconds, and this is true regardless of whether the timestamp is local time or UTC. For example, if there is a negative leap second at the end of 2023, the timestamp 2023-12-31 23:59:59 UTC will be missing, and similarly for simultaneous localtime timestamps. Conversely, if there is a positive leap second at the end of 2025 and local time is not an integral number of minutes away from UTC, some (but not all) implementations have ambiguous localtime timestamps; see <https://mm.icann.org/pipermail/tz/2021-September/030377.html> for an example. Also, perhaps a local timestamp's UTC-offset ambiguity could interact with the same timestamp's leap-second ambiguity, though such cases should be rare.
On Sep 21, 2021, at 11:58 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 9/20/21 9:51 AM, Howard Hinnant via tz wrote:
The C++20 standard has adopted the IANA timezone database (not the code) as the basis for a major part of its API: http://eel.is/c++draft/time.zone [Retitling since the C++20 time library is a different topic.]
Here are comments about that draft. Hope they help. Unfortunately I got only about halfway thru the draft before running out of time.
Thanks for taking the time for this constructive criticism.
* Although the draft distinguishes Zones from Links, such a distinction is not available on platforms that use a filesystem's hard links to implement Links. For example, on Solaris 10:
$ cd /usr/share/lib/zoneinfo $ ls -li Asia/Chongqing Asia/Chungking Asia/Harbin Asia/Shanghai PRC 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Chongqing 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Chungking 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Harbin 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 Asia/Shanghai 303178 -rw-r--r-- 5 root bin 148 2018-09-24 11:26 PRC
with no indication that Asia/Shanghai is the Zone and the other names Links. If there's a reason that the draft distinguishes Zones from Links, I suggest explaining the reason, and saying what implementations should do when the distinction between Zones and Links is not available. Another possibility is to remove the distinction between Zones and Links.
Agreed that the link part of the database may be zero-sized on some platforms. The API minimizes the distinction between zones and links by having the lookup of all names search both the zones and the links. The interested programmer can still ferret out a distinction. But the casual programmer will see the union of zones and links as just one big set.
* Minor wording nit: in the "remote_version" description, change "The latest remote database version" to "The current remote database version". It's possible that the current remote version is a downgrade or sidegrade from the local database version; the API still works in this case.
Good point, thanks.
* Exception classes. I see a problem with leap seconds here. These classes and their related operations seem to be designed only for local timestamps that are ambiguous and/or missing due to UTC offset changes, such as switching from standard to daylight-saging time. However, timestamps can also be ambiguous and/or missing due to leap seconds, and this is true regardless of whether the timestamp is local time or UTC.
For example, if there is a negative leap second at the end of 2023, the timestamp 2023-12-31 23:59:59 UTC will be missing, and similarly for simultaneous localtime timestamps. Conversely, if there is a positive leap second at the end of 2025 and local time is not an integral number of minutes away from UTC, some (but not all) implementations have ambiguous localtime timestamps; see <https://mm.icann.org/pipermail/tz/2021-September/030377.html> for an example. Also, perhaps a local timestamp's UTC-offset ambiguity could interact with the same timestamp's leap-second ambiguity, though such cases should be rare.
This part of the library is specified to be interoperable with Unix Time (http://eel.is/c++draft/time.clock.system#overview-1). Thus leap seconds are excluded by design. There does exist a separate facility, utc_clock (http://eel.is/c++draft/time.clock.utc) that is specified to include leap second information. However to interoperate with the time zone facility, a conversion from utc_time to sys_time is necessary. This conversion specifies exactly how the leap second is dealt with in the event that the source time_point points within a leap second. The utc_clock/utc_time facility is mainly useful if you need the duration between two time_points that potentially includes one or more leap seconds. It is also useful for communicating with external API’s that include leap seconds in their data structures (e.g. https://github.com/HowardHinnant/date/wiki/Examples-and-Recipes#ccsds). Howard
On 9/21/21 9:29 AM, Howard Hinnant wrote:
Agreed that the link part of the database may be zero-sized on some platforms.
That's fine, but could a note or footnote be added to that effect? For example, the text should make it clear that even given the same tzdb version, the same entry could be a Zone on one platform and a Link on another.
This part of the library is specified to be interoperable with Unix Time (http://eel.is/c++draft/time.clock.system#overview-1). Thus leap seconds are excluded by design.
Ah, OK. Unfortunately that's not obvious from the text, which has an extensive discussion of leap seconds. I suggest adding some text that makes it clear that the leap second interfaces are only for interrogating tzdb, and that leap seconds are not used by the conversion functions. For example, it should be clear what conversion functions do when given a tzdb entry that has leap seconds - do they ignore the leap seconds, or report an error, or what?
On Sep 21, 2021, at 12:59 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 9/21/21 9:29 AM, Howard Hinnant wrote:
Agreed that the link part of the database may be zero-sized on some platforms.
That's fine, but could a note or footnote be added to that effect? For example, the text should make it clear that even given the same tzdb version, the same entry could be a Zone on one platform and a Link on another.
I’ll see what I can do.
This part of the library is specified to be interoperable with Unix Time (http://eel.is/c++draft/time.clock.system#overview-1). Thus leap seconds are excluded by design.
Ah, OK. Unfortunately that's not obvious from the text, which has an extensive discussion of leap seconds.
I suggest adding some text that makes it clear that the leap second interfaces are only for interrogating tzdb, and that leap seconds are not used by the conversion functions. For example, it should be clear what conversion functions do when given a tzdb entry that has leap seconds - do they ignore the leap seconds, or report an error, or what?
The “right” form of the tzdb should not be used for C++. Howard
One other thing. Internet RFC 8536 discusses TZif file truncation, and says that TZif files should not be used to convert timestamps outside of the truncated range. This discussion is improved in the latest draft for RFC 8536's successor, here: https://datatracker.ietf.org/doc/draft-murchison-rfc8536bis/ which also talks about expiration of leap second tables. Should the C++20 time library address the issues of TZif truncation and leap second table expiration, or treat these issues as being out of scope? Either way, some text would be advisable.
On Sep 21, 2021, at 1:19 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
One other thing. Internet RFC 8536 discusses TZif file truncation, and says that TZif files should not be used to convert timestamps outside of the truncated range. This discussion is improved in the latest draft for RFC 8536's successor, here:
https://datatracker.ietf.org/doc/draft-murchison-rfc8536bis/
which also talks about expiration of leap second tables.
Should the C++20 time library address the issues of TZif truncation and leap second table expiration, or treat these issues as being out of scope? Either way, some text would be advisable.
The standard does not specify that the source form of the database is TZif, though this is likely on platforms which already deploy TZif. Leap second table expiration is a possible feature for a future standard. You are correct that it currently does not exist. Howard
On Sep 20, 2021, at 1:06 AM, Stephen Colebourne via tz <tz@iana.org> wrote:
As most of you probably know, there is a dispute about the tzdb maintainer's recent changes to merge large numbers of time-zones [1][2]. These have the effect of wiping out historic time-zone information on many locations where the data has been in tzdb for many years.
The comment at the beginning of 2021a's backzone says: # From Paul Eggert (2014-10-31): # This file contains data outside the normal scope of the tz database, # in that its zones do not differ from normal tz zones after 1970. # Links in this file point to zones in this file, superseding links in # the file 'backward'. # Although zones in this file may be of some use for analyzing # pre-1970 timestamps, they are less reliable, cover only a tiny # sliver of the pre-1970 era, and cannot feasibly be improved to cover # most of the era. Because the zones are out of normal scope for the # database, less effort is put into maintaining this file. Many of # the zones were formerly in other source files, but were removed or # replaced by links as their data entries were questionable and/or they # differed from other zones only in pre-1970 timestamps. so it's not as if this is the first time that historic information has been moved to backzone, which, although not "wiping it out" in the sense of making it no longer exist in any form in the tzdb repository or releases, removes it from any collection of compiled tzdb files not build using backzone, and removes it from any data set that applications get by reading tzdb source files and not reading backzone. One example of data that's in backzone in 2021a is the data for America/Montreal, the comment for which in 2021a backzone is: # Canada # # From Paul Eggert (2015-03-24): # Since 1970 most of Quebec has been like Toronto; see # America/Toronto. However, earlier versions of the tz database # mistakenly relied on data from Shanks & Pottenger saying that Quebec # differed from Ontario after 1970, and the following rules and zone # were created for most of Quebec from the incorrect Shanks & # Pottenger data. The post-1970 entries have been corrected, but the # pre-1970 entries are unchecked and probably have errors. # The example that's being cited as a problem with the merge is Europe/Oslo, the comment for which in 2021a europe is: # Norway # http://met.no/met/met_lex/q_u/sommertid.html (2004-01) agrees with Shanks & # Pottenger. So one difference here appears to be that the pre-1970 data for Europe/Oslo may be accurate while the pre-1970 data for America/Montreal may be inaccurate. Another difference is that Toronto and Montreal are in the same country, while Oslo and Berlin aren't. So: Were there any complaints when America/Montreal was turned into a link in 2015c (if the NEWS file is to be believed)? Is the concern that there's software, processes, etc. that depend on Europe/Oslo having the correct pre-1970 data, because that data appears to match what The Norwegian Meteorological Institute says, whereas there may be less software, processes, etc. that depend on America/Montreal having pre-1970 data because that data might be bogus?
Guy Harris wrote in <8C6BA6DA-AC7D-4FCE-97F0-1B277A981B55@sonic.net>: ... |So one difference here appears to be that the pre-1970 data for Europe/O\ |slo may be accurate while the pre-1970 data for America/Montreal may \ |be inaccurate. But isn't the drive to backzone going on for longer, yes hasn't it been introduced for exactly the purpose in 2014? ac5bf48519 New data file 'backzone' for out-of-scope and/or poorly-sourced data. where NEWS says A new file 'backzone' contains data which may appeal to connoisseurs of old time stamps, although it is out of scope for the tz database and is often poorly sourced. The new file is not recommended for ordinary use and its entries are not installed by default. (Thanks to Lester Caine for the Guernsey, Jersey, and Isle of Man entries in 'backzone'.) Ever since i am on this ML (at least a decade i would say) over and over again it is said to everybody that the IDs are just strings and the data is the thing, that selection is a downstream thing etc etc etc. I surely hated it, but that is seven years! |Another difference is that Toronto and Montreal are in the same country, \ |while Oslo and Berlin aren't. The thing is that _i_ never treated links as links, even my dumb brain did manage to get by to separate data from names. (I posted some code snippets many years ago. Yes, negative leaps i failed for. It was just a single binary DB file with the zone data and at the end a TOC of names pointing to offsets of their data. I was prowd by then, i was stupid. Whatever.) Ever since i am on this ML it was discussed often to possibly replace the identifiers with anonymized strings, i wonder how anyone could create an interface where one name maps to another name. And what for? This is not just "super correct" and "i give my users the full knowledge of the real thing, of anything", that is just broken. It is anyway a programmer-only thing, maybe also a UNIX command line power user thing -- but to get it _really_ right for _users_, you had to use ICU data and use their approach of time zones, that includes translation then also, does it? You know, as a human german speaking being i would _never_ assume Europe/Vienna and do "$ TZ=Europe/Vienna date" really, that is a nerd assumption. Maybe "$ date --country=AUstrIA" or "$ date --city Würzburg" or "$ date --state Baden-Württemberg", and i only write this for case-messing out-of-ASCII. Sorry for getting bold, but for one i really did not make that error (!), and then i also find it absurd to claim to offer a timezone interface that covers all the date and time data, and then to exclude backzone. That is schizophrenic, that is fascistic, because of course bad quality data that has been collected from an uncounted number of sources (and Mr. Eggert seems to be very interested in the topic, and actively sleuths in libraries and other sources to improve his knowledge on the topic, which i cannot claim from myself) is much better than simply excluding the few traces that exist, and asserting "this is the best i can offer". I do not think Joda would do that. (iirc) On the other hand the announced release strategy felt offending to me. Even though many hours of talking over several months were spent where possibly some code fixes etc would have been due. Also, for JAVA in particular, i seem to remember that there was discussion that it really would be better to use custom JAVA tarballs for downloading already years ago, but it seems nothing changed. If it would, then a simple sed(1) that postprocesses any Link would have been put in place, maybe. I also remember that in the last discussion it seemed at least one of my mails was forwarded behind the scenes before the administrator (whoever it is) waved it through. I really do not know what for. One approach could for example be to _require_ a new build flag, so that downstream consumers aka packagers which do not read the list but only have robots sitting around that watch out for releases will get build failures, and thus have to adjust their packaging recipe. Then the release announcement could also prominently reassert the IANA TZ direction of going post 1970 by default, but to not actively cut zone data at 1970, ie, that backzone is needed for the full picture if you want to get it right. This does not satisfy data-only downstream consumers, but then again these should really know what they are doing. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
On Sep 23, 2021, at 5:41 AM, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
Guy Harris wrote in <8C6BA6DA-AC7D-4FCE-97F0-1B277A981B55@sonic.net>: ... |So one difference here appears to be that the pre-1970 data for Europe/O\ |slo may be accurate while the pre-1970 data for America/Montreal may \ |be inaccurate.
But isn't the drive to backzone going on for longer, yes hasn't it been introduced for exactly the purpose in 2014?
I.e., one to two years longer, given that America/Montreal was introduced in 2015 (as I believe I noted when I first brought up America/Montreal in this context).
Ever since i am on this ML it was discussed often to possibly replace the identifiers with anonymized strings, i wonder how anyone could create an interface where one name maps to another name.
If I open up System Preferences on my machine, select the "Date and Time" entry, select the "Time Zone" tab, enable editing via a couple of steps, and type "Oxnard" into the "Closest City" box, under the hood it maps "Oxnard, CA" to "America/Los_Angeles", setting the current system tzdb region to the latter. (And then I go back to "figure it out automatically based on where I'm currently located", but I digress....) Is that what you mean by "one name maps to another name"? There's no "America/Oxnard", so it needs to figure out that Oxnard is in the tzdb region that's named "America/Los_Angeles".
And what for? This is not just "super correct" and "i give my users the full knowledge of the real thing, of anything", that is just broken. It is anyway a programmer-only thing, maybe also a UNIX command line power user thing -- but to get it _really_ right for _users_, you had to use ICU data and use their approach of time zones, that includes translation then also, does it?
macOS *does* include ICU, but the string "Oxnard" appears nowhere in CLDR as of cldr-28, so I suspect it uses more than just ICU data.
You know, as a human german speaking being i would _never_ assume Europe/Vienna and do "$ TZ=Europe/Vienna date" really, that is a nerd assumption. Maybe "$ date --country=AUstrIA" or "$ date --city Würzburg" or "$ date --state Baden-Württemberg", and i only write this for case-messing out-of-ASCII.
Sounds good. That would require data of the sort that macOS uses; could OpenStreetMaps data be used? Anyway:
Sorry for getting bold, but for one i really did not make that error (!), and then i also find it absurd to claim to offer a timezone interface that covers all the date and time data, and then to exclude backzone.
I think we need, as some have suggested, to figure out what to do about pre-1970 data. I think that includes somehow making it clear that people who expect complete, accurate, and unchanging pre-1970 data are expecting something that 1) is currently self-contradictory (Do you want "accurate" or do you want "unchanging"? Some of the data we have is probably inaccurate, so that making it accurate will involve change!) 2) would involve a lot of effort (sometimes it's even hard to get information about upcoming time changes from governments; finding historical information might be a lot harder, especially to people who aren't familiar with the language and culture of the location for which you're trying to get historical time information). It also includes deciding whether to offer all the data by default (as in "not in backzone"), to provide a way to offer both sets of data separately, or whatever.
On the other hand the announced release strategy felt offending to me.
I think that given the current level of controversy, the next release should be "2021a+Samoa and any other updates" rather than "current tip of the main branch", so that we get something out that includes the Samoa updates but doesn't include the controversial changes.
Guy Harris wrote in <B97CABBF-BFEA-4CE5-9D05-9663170ABBE0@sonic.net>: |On Sep 23, 2021, at 5:41 AM, Steffen Nurpmeso <steffen@sdaoden.eu> wrote: |> Guy Harris wrote in |> <8C6BA6DA-AC7D-4FCE-97F0-1B277A981B55@sonic.net>: |> ... |>|So one difference here appears to be that the pre-1970 data for Europe/O\ |>|slo may be accurate while the pre-1970 data for America/Montreal may \ |>|be inaccurate. And the README even says This database of historical local time information has several goals: * Provide a compendium of data about the history of civil time that is useful even if not 100% accurate. |> But isn't the drive to backzone going on for longer, yes hasn't it |> been introduced for exactly the purpose in 2014? | |I.e., one to two years longer, given that America/Montreal was introduced \ |in 2015 (as I believe I noted when I first brought up America/Montreal \ |in this context). | |> Ever since i am on this ML it was discussed often to possibly |> replace the identifiers with anonymized strings, i wonder how |> anyone could create an interface where one name maps to another |> name. | |If I open up System Preferences on my machine, select the "Date and \ |Time" entry, select the "Time Zone" tab, enable editing via a couple \ |of steps, and type "Oxnard" into the "Closest City" box, under the \ |hood it maps "Oxnard, CA" to "America/Los_Angeles", setting the current \ |system tzdb region to the latter. (And then I go back to "figure it \ |out automatically based on where I'm currently located", but I digress....) | |Is that what you mean by "one name maps to another name"? There's \ |no "America/Oxnard", so it needs to figure out that Oxnard is in the \ |tzdb region that's named "America/Los_Angeles". This is malicous hairsplitting for nothing. If a programming interface gives you one TZ name for another TZ name where both effectively indistinguishable select the same data. The first name had to be selected first, why would you want it to be mapped? backward is not timing out, i think. |> And what for? This is not just "super correct" and "i give |> my users the full knowledge of the real thing, of anything", that |> is just broken. It is anyway a programmer-only thing, maybe also |> a UNIX command line power user thing -- but to get it _really_ |> right for _users_, you had to use ICU data and use their approach |> of time zones, that includes translation then also, does it? | |macOS *does* include ICU, but the string "Oxnard" appears nowhere in \ |CLDR as of cldr-28, so I suspect it uses more than just ICU data. | |> You know, as a human german speaking being i would _never_ assume |> Europe/Vienna and do "$ TZ=Europe/Vienna date" really, that is |> a nerd assumption. Maybe "$ date --country=AUstrIA" or |> "$ date --city Würzburg" or "$ date --state Baden-Württemberg", |> and i only write this for case-messing out-of-ASCII. | |Sounds good. That would require data of the sort that macOS uses; \ |could OpenStreetMaps data be used? Well _i_ do not think that sounds good, i think reality is you go to a mega companies app store, download one, and then input or even speak a name. |Anyway: Yes, please! |> Sorry for getting bold, but for one i really did not make that |> error (!), and then i also find it absurd to claim to offer |> a timezone interface that covers all the date and time data, and |> then to exclude backzone. | |I think we need, as some have suggested, to figure out what to do about \ |pre-1970 data. I think that includes somehow making it clear that \ |people who expect complete, accurate, and unchanging pre-1970 data \ |are expecting something that | | 1) is currently self-contradictory (Do you want "accurate" or do you \ | want "unchanging"? Some of the data we have is probably inaccurate, \ | so that making it accurate will involve change!) | | 2) would involve a lot of effort (sometimes it's even hard to get \ | information about upcoming time changes from governments; finding \ | historical information might be a lot harder, especially to people \ | who aren't familiar with the language and culture of the location \ | for which you're trying to get historical time information). | |It also includes deciding whether to offer all the data by default \ |(as in "not in backzone"), to provide a way to offer both sets of data \ |separately, or whatever. My personal opinion is that data should not be splitted at all. Instead tooling should be used to cut where the line, if any, is drawn, the make file is capable of doing this. Without being able to quote now, IANA TZ is supposed to offer post-1970 data by default. |> On the other hand the announced release strategy felt offending to |> me. | |I think that given the current level of controversy, the next release \ |should be "2021a+Samoa and any other updates" rather than "current \ |tip of the main branch", so that we get something out that includes \ |the Samoa updates but doesn't include the controversial changes. Or the build should be changed to either cut the data by default, or to include backzone by default. In both cases all but a surely unused broken interface of two downstream libraries should not recognize a difference, .. unless they want to use date pre 1970. It is just sad that for black and other coloured noone spoke up against splitting to backzone, but suddenly it is shaking. Given the length of the thread, maybe your way is the best. But if the year long and now standardized way is continued, including the release, then downstream consumers should be made aware of changes, like letting an unchanged build recipe fail. --End of <B97CABBF-BFEA-4CE5-9D05-9663170ABBE0@sonic.net> --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
On Sep 23, 2021, at 3:26 PM, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
Guy Harris wrote in <B97CABBF-BFEA-4CE5-9D05-9663170ABBE0@sonic.net>: |On Sep 23, 2021, at 5:41 AM, Steffen Nurpmeso <steffen@sdaoden.eu> wrote
|> Ever since i am on this ML it was discussed often to possibly |> replace the identifiers with anonymized strings, i wonder how |> anyone could create an interface where one name maps to another |> name. | |If I open up System Preferences on my machine, select the "Date and \ |Time" entry, select the "Time Zone" tab, enable editing via a couple \ |of steps, and type "Oxnard" into the "Closest City" box, under the \ |hood it maps "Oxnard, CA" to "America/Los_Angeles", setting the current \ |system tzdb region to the latter. (And then I go back to "figure it \ |out automatically based on where I'm currently located", but I digress....) | |Is that what you mean by "one name maps to another name"? There's \ |no "America/Oxnard", so it needs to figure out that Oxnard is in the \ |tzdb region that's named "America/Los_Angeles".
This is malicous hairsplitting for nothing.
No, it's attempting to determine what you mean by "an interface where one name maps to another name". It sounds as if you are not referring to interfaces where a city name maps to an identifier for the tzdb region in which that city resides, in which case I wonder how anyone could wonder how anyone could create such an interface, given that, at minimum, they exist in macOS and Ubuntu. So what *do* you mean here.
If a programming interface gives you one TZ name for another TZ name where both effectively indistinguishable select the same data.
OK, so is *that* the type of mapping to which you're referring? Mapping tzids to other tzids (which is *not* what I was referring to when describing the macOS time zone selector)? That's different from "[replacing] the identifiers with anonymized strings".
| |> You know, as a human german speaking being i would _never_ assume |> Europe/Vienna and do "$ TZ=Europe/Vienna date" really, that is |> a nerd assumption. Maybe "$ date --country=AUstrIA" or |> "$ date --city Würzburg" or "$ date --state Baden-Württemberg", |> and i only write this for case-messing out-of-ASCII. | |Sounds good. That would require data of the sort that macOS uses; \ |could OpenStreetMaps data be used?
Well _i_ do not think that sounds good, i think reality is you go to a mega companies app store, download one, and then input or even speak a name.
What doesn't sound good - using OpenStreetMaps? Or allowing "date --country=AUstrIA", "date --city Würzburg", or "--state Baden-Württemberg"? "date --country=Canada" doesn't work, for obvious reasons (the same applies to "United States", "Australia", "Russia", etc.). "date --state=Arizona" doesn't work, because you aren't indicating which part of Arizona you're in. But those can print an error message.
My personal opinion is that data should not be splitted at all. Instead tooling should be used to cut where the line, if any, is drawn, the make file is capable of doing this. Without being able to quote now, IANA TZ is supposed to offer post-1970 data by default.
Preferably with a big red "THIS IS NOT GUARANTEED TO BE CORRECT AND IS SUBJECT TO CHANGE OVER TIME AS A RESULT OF ATTEMPTS TO MAKE IT MORE CORRECT" warning. Those who find those caveats unacceptable are more than welcome to solve the problem themselves.
It is just sad that for black and other coloured noone spoke up against splitting to backzone, but suddenly it is shaking.
Which is part of the point that Paul was making - why is Europe/Oslo currently not in backzone but Africa/Asmara is currently in backzone? There is at least one wypipo region where data was moved to backzone, America/Montreal; I don't know whether that received more pushback than, say, the movement of Africa/Asmara to backzone.
Given the length of the thread, maybe your way is the best.
"My" way? As I indicated in another message, I have no strongly-held ideas about whether pre-1970 data should be relegated to backzone. (And I've also indicated that I think that, *regardless* of the merits or demerits of moving Europe/Oslo to backzone, the right thing to do for 2021b is not to move anything to backzone, just because it's proven, at least to my eyes, to be sufficiently controversial that it shouldn't be done *right now*.) I also haven't weighed in on the way to handle backzone in the build process if we keep it around and move more stuff to it over time.
participants (31)
-
Almaz Mingaleev -
Brooks Harris -
Clive D.W. Feather -
Derick Rethans -
dpatte -
Eliot Lear -
Eliot Lear -
Emily Crandall Fleischman -
Fred Gleason -
Garrett Wollman -
Guy Harris -
Howard Hinnant -
J Andrew Lipscomb -
Jan Engelhardt -
Kevin Kenny -
Kim Davies -
Michael Douglass -
Michael H Deckers -
Murray S. Kucherawy -
Paul Eggert -
Paul Goyette -
Phake Nick -
Philip Paeps -
Robert Elz -
Russ Allbery -
Steffen Nurpmeso -
Stephen Colebourne -
Steve Allen -
Tom Lane -
Tony Finch -
Watson Ladd