A partly-baked idea: would it make sense to allow something like a link in a Zone record? Let's say that, if the first character in the GMTOFF column is not a decimal digit or a '-', then GMTOFF is the name of another Zone. The only other column in such a record would be [UNTIL]. As an example, let's say that America/Shiprock becomes a Zone, and that the Navajo people followed Arizona time until that state opted out of DST. That Zone might look something like (I'm just making this up): # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone America/Shiprock -6:59:56 - LMT 1883 Nov 18 12:00:04 America/Phoenix 1967 America/Denver That would be easy to parse. The "interesting" bit would be figuring out which row in the linked Zone we should start with. It would be the row following the one whose [UNTIL] is the greatest lower bound of the date of interest. Downstream systems that use the source files directly would need access to files with the older format for a while to give them time to change their parsers. (Maybe the first column in a new Zone record could be something other than "Zone".) --Bill Seymour On Sun, Jun 6, 2021 at 10:03 AM Tom Lane via tz <tz@iana.org> wrote:

Stephen Colebourne via tz <tz@iana.org> writes:

...

[ assorted proposals ]

One other issue that I think deserves more attention than it has gotten lately is that tzdb has become a de facto standard and users rely on its stability. I would like to see some sort of principle adopted that minimizes changes in historical data. In particular, I think it's past time to prohibit data changes adopted for essentially-administrative reasons (as opposed to new findings of historical fact). I'd put the recent reorganization under the heading of things that would be forbidden by this principle, and also the changes a few years ago that removed "made up" zone abbreviations. Whatever the justification for those abbreviations originally, some people had come to depend on them, and little was to be gained by removing them.

Looking at Stephen's list with this in mind, one thing I'd vote against is redefining the LMT offsets. Yeah, I suggested that myself a few days ago, but that was in the context of what seemed to be a fait accompli that would largely destroy tzdb's backward compatibility anyway. If we're going to reverse that choice and try to preserve the existing pre-1970 data, then preserving the existing LMT data goes along with that.

The idea of having at least one zone per ISO-3166-1 country does seem like a good one, though. Aside from possibly deflecting politically-based complaints, this seems basically like future proofing: even if two countries have shared clocks since 1970, they could diverge at any time. Being prepared with an appropriate zone name should minimize the pain to users. Also notice that splitting an existing zone creates no compatibility problems, since no one is obligated to switch to the new zone name immediately.

regards, tom lane

Fred Gleason via tz <tz@iana.org> writes:

On Jun 6, 2021, at 11:02, Tom Lane via tz <tz@iana.org> wrote:

...

The idea of having at least one zone per ISO-3166-1 country does seem like a good one, though.

These two points in particular are synergistic; stability of historical data is just a Good Thing, but no one here wants to see the TZDB maintainer receiving ‘vitriolic’ e-mail, nor getting sued [again]. Hence, politically ‘future-proofing' the DB seems a prudent move.

I had a further thought about this: if we want to have both of these principles (zone-per-country and stability of old data), then it would make sense to insist that we don't create new per-country zones until someone has done the research to fill in plausible old data back to the LMT era for the proposed zone name. If that initial data later proves wrong, well, fixing it falls within longstanding tzdb practice. But we shouldn't start out a new zone with known-bogus old data. As an example that relates to one of the current complaints, if Sweden had been part of the Europe/Berlin zone all along, we'd not split out Europe/Stockholm without first reconstructing plausible historical data for Sweden. The advantage of this rule is that it would encourage an incremental approach to getting to zone-per-country. If someone comes along and whines that $wherever should have its own zone, they can be told "Sure. Come back when you've done the research." There's no reason that Paul and Tim should be expected to make that happen on their own. regards, tom lane

John Hawkinson <jhawk@alum.mit.edu> writes:

Tom Lane via tz <tz@iana.org> wrote on Sun, 6 Jun 2021 at 13:31:06 EDT in <655997.1623000666@sss.pgh.pa.us>:

...

I had a further thought about this: if we want to have both of these principles (zone-per-country and stability of old data), then it would make sense to insist that we don't create new per-country zones until someone has done the research to fill in plausible old data back to the LMT era for the proposed zone name.

I don't think it's fair because the tz database needs to turn on a dime to reflect political changes that happen on rapid timescales. If a zone into two countries (whether by one of two previously-aligned countries changing their time zone rules, or by other political or even military mechanism) overnight, we need to push a new release out ASAP, we can't say, "oh, sorry, you can't have working time on your computers, we have to research the history, just suffer along for 6 months."

You're misunderstanding the context, I think. If country X actually changes their DST rules with minimal notice, then yeah, we'd have to create a new zone and worry about correcting its old data later. What I'm thinking about is how to handle the situation where X should have its own zone according to the newly formulated zone-per-country rule, but there is no post-1970 data divergence that would make it necessary to have a separate zone according to other rules. I do not think that there need be any urgency about making that new zone come into existence, especially not if that would certainly lead to the need to change its pre-1970 data later. So I don't buy that there's any "fairness" argument. What's unfair about asking somebody who wants a quick change to do the legwork to support it? regards, tom lane

I would agree to both of these. I also think it would be preferable to restore (with corrections if necessary) all the distinct zones per ISO country that have varied since 1970. Russia and Canada have numerous timezones, just as the USA has, and it would simplify access, as well as providing more historical data per country. Overall my preference is to make historical data available for as many cities as possible / per country, in one place; or for at least all zones per country that have differed since 1970. If they link is to generic 'noname' zones which crosses political boundaries, then I could also be happy with that, as long as no historical data is lost. In the long run, app developers will also want historical data for separate zones that merged before 1970, and they could exist in a secondary table, managed by tz, or even managed by someone else as a separate project. On 2021-06-06 12:44, Fred Gleason via tz wrote:

On Jun 6, 2021, at 11:02, Tom Lane via tz <tz@iana.org <mailto:tz@iana.org>> wrote:

...

One other issue that I think deserves more attention than it has gotten lately is that tzdb has become a de facto standard and users rely on its stability.  I would like to see some sort of principle adopted that minimizes changes in historical data.  In particular, I think it's past time to prohibit data changes adopted for essentially-administrative reasons (as opposed to new findings of historical fact).  I'd put the recent reorganization under the heading of things that would be forbidden by this principle, and also the changes a few years ago that removed "made up" zone abbreviations. Whatever the justification for those abbreviations originally, some people had come to depend on them, and little was to be gained by removing them.

[…]

The idea of having at least one zone per ISO-3166-1 country does seem like a good one, though.  Aside from possibly deflecting politically-based complaints, this seems basically like future proofing: even if two countries have shared clocks since 1970, they could diverge at any time.  Being prepared with an appropriate zone name should minimize the pain to users.  Also notice that splitting an existing zone creates no compatibility problems, since no one is obligated to switch to the new zone name immediately.

These two points in particular are synergistic; stability of historical data is just a Good Thing, but no one here wants to see the TZDB maintainer receiving ‘vitriolic’ e-mail, nor getting sued [again]. Hence, politically ‘future-proofing' the DB seems a prudent move.

Cheers!

|---------------------------------------------------------------------| | Frederick F. Gleason, Jr. |             Chief Developer         | |                           |             Paravel Systems         | |---------------------------------------------------------------------| |         A room without books is like a body without a soul.         | |         | |                                                         -- Cicero   | |---------------------------------------------------------------------|

Tom Lane via tz said:

One other issue that I think deserves more attention than it has gotten lately is that tzdb has become a de facto standard and users rely on its stability. I would like to see some sort of principle adopted that minimizes changes in historical data. [...] and also the changes a few years ago that removed "made up" zone abbreviations. Whatever the justification for those abbreviations originally, some people had come to depend on them, and little was to be gained by removing them.

But I'd like another principle to be "truth". We should not be making up data - that goes both for lying about LMT offsets and lying about the abbreviations people use. Remember that the basic principle was "what's used on the ground".

The idea of having at least one zone per ISO-3166-1 country does seem like a good one, though. Aside from possibly deflecting politically-based complaints,

And adding other ones.

this seems basically like future proofing: even if two countries have shared clocks since 1970, they could diverge at any time.

So can a country and a dependent territory. Or even a non-dependent one: to pick a fanciful example, as part of the upcoming EU changes, Italy could decide to stick to UTC+2 but France to UTC+1, then Corsica decide to go to UTC+2.

Being prepared with an appropriate zone name should minimize the pain to users.

Why not wait until it's needed? We're likely to get more notice than we get for some countries' Ramadan changes. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

Derick Rethans via tz said:

...

2) Negative DST Negative DST in the source files continues to be a problem, but we should address other issues first.

When I first highlighted this Irish case (https://mm.icann.org/pipermail/tz/2017-December/025621.html), it was never my intention that the tz data was changed to negative DST - I was originally only pointing out that IST isn't only Iran Standard Time, but also Irish Standard Time. I don't mind how it is recorded, as long as it produces the right output (in the form of transitions).

If negative DST represents what actually happens in the real world, then that's what we should be using. If implementations can't cope with negative DST, then they have bugs in them and/or don't match the real world. They need to change; we should not be lying to everyone just because some people won't face reality. (I don't object to a mechanism for people to do a build with the lies in, provided they have to explicitly accept it. Lies shouldn't be the default.) The other thing I think we need to discuss in this debate is the intent of pre-1970 data. It should be clear to people what this data is meant to cover. I was under the impression that pre-1970 data represents the time in at least some part of the zone. So, at present, the Berlin zone has pre-1970 data that represents at least part of Germany and the Stockholm zone has pre-1970 data that represents at least part of Sweden. If these are merged, the resulting zone has two names but both only contain pre-1970 data for part of Germany. In this case, it needs to be made very clear that the area covered by Stockholm is "Germany and Sweden", not "Sweden". The problem appears, to me, to be two zones that have the same contents but different descriptions. That would need to be fixed. (Personally, I'd like all the pre-1970 data that we used to have to still be available. But I understand that Paul doesn't want to have to change dozens of zones that last differed in the 19th century just because of a change in the 21st. There ought to be a solution to this.) -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

On Mon, 7 Jun 2021 at 10:28, Guy Harris via tz <tz@iana.org> wrote:

Note that theory.html says "don't do that":

In short, many, perhaps most, of the tz database's pre-1970 and future timestamps are either wrong or misleading. Any attempt to pass the tz database off as the definition of time should be unacceptable to anybody who cares about the facts. In particular, the tz database's LMT offsets should not be considered meaningful, and should not prompt creation of timezones merely because two locations differ in LMT or transitioned to standard time at different dates.

If you ask a Java library "what was the offset on 1st Jan 1600 at 12:36 pm" they will all provide an answer. My understanding is that the same is true of libraries in many other environments. Most systems will provide LMT as the answer. Some (eg. the older Java API) will provide a non-LMT offset. Those of us on this list know that the question being asked is pretty stupid, but it is asked by end users. And when it is asked, people are surprised by the LMT answer. I is worthwhile considering if LMT is the best solution to the far past problem. The second point is not relevant - the proposal is to determine which zones get a LMT and history based on post-1970 data with an ISO country minimum. There is no proposal to create timezones solely for LMT. Stephen

On 6/7/21 7:57 AM, David Patte via tz wrote:

In my own code I use 1800 as the transition date between LST and LMT when it's not specified elsewhere.

1800 is surely a bit early, for all but the most technologically-advanced locations. In most of the world, solar time maintained its supremacy over local mean time well after 1800. The "when it's not specified elsewhere" intrigues me, though. What sort of specification do you have elsewhere? One can imagine a tzdb extension containing when local mean time came into effect at each location. Unfortunately if we added something along these lines to tzdb, I expect we'd have to invent nearly every data item. It'd be like a good chunk of the pre-1970 data we already have, only worse. Part of the problem is that people in the early 19th century didn't much care whether they were using local solar time or local mean time, and many towns actually observed a mean-time approximation to solar time. Here's a quote from page 15 of Francis Abbott's book "A Treatise on the Management of Public Clocks", 3rd ed. (1839): "Nothing is more common than to place the management and regulating of church clocks in the hands of the sexton, without keeping any check upon him or allowing him a salary to stimulate him in this important duty; the consequence is, that the village clocks throughout the country are kept by chance, and generally speaking vary from one quarter to three quarters of an hour from mean time." Abbott also wrote (page 16) that when a sexton periodically checked and set a church clock's time, "the usual mode of ascertaining time is by the sundial", i.e., the clock was considered to be a good-enough approximation to solar time rather to local mean time. This state of affairs didn't change until the telegraph made it feasible for timestamps to be communicated more accurately, and railroads needed more-accurate time.

On Mon 2021-06-07T20:14:00+0100 Clive D.W. Feather via tz hath writ:

I was wandering round the Fitzwilliam Museum in Cambridge the other day and noticed a Thomas Tompion (1639-1713) clock. It had two hands and a dial with Roman numerals I to XII twice going round it. Outside that, in Arabic numerals, were 5, 10, ..., 60, also twice. At the top it said "EQVAL 60 TIME".

So 1800 is perhaps a bit late, at least for some places.

A clock of this quality would be sold as a "regulator" and priced accordingly. Not even most observatories had clocks accurate to one second until around 1800. If one second is the standard then 1800 is a good guess at the year. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m

On Jun 7, 2021, at 10:53, Stephen Colebourne via tz <tz@iana.org> wrote:

"what people in a region think" is essentially unknowable. What governmental authorities define is generally well recorded. Where we can accurately fnd data to indicate that a region is not following the lead of the governmental authorities then I agree we need to make sure that those people's experience can be expressed in some form by TZDB. But I think that governmental authorities so dominate the field of timezones, and what our downstream users perceive of timezones, that we need to reflect it. Putting our heads in the sand and pretending that governmental authorities don't exist is not helpful.

Other than Asia/Urumqi, I’m hard pressed to call to mind a region where a significant section of the population has basically decided to defy all governmental authority. However, there are areas where *which* governmental authority is relevant is a very live question (the Crimea comes to mind immediately). The “what people in a region think” standard allows TZDB to avoid taking sides about which political entity is the “legitimate” one. Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | A room without books is like a body without a soul. | | | | -- Cicero | |---------------------------------------------------------------------|