November 2021 - tz - lists.iana.org

Pre-1970 data
by Stephen Colebourne Nov. 12, 2021

Nov. 12, 2021

How can this project successfully handle pre-1970 data? Why have I focussed so much on IDs and their value to tzdb? Intro ------- In previous threads I've focussed on IDs, and laid out different ways to describe the groups of IDs we have. For now, lets focus solely on the two main groups - region IDs, which represent abstract regions where clocks have been the same since 1970 - non-region IDs, which represent locations where tzdb has, at some point, added an ID The key observation is that the segregation between these two groups of IDs *did not exist* until around 2014. It was only in 2018 that the ISO country rule was removed. It has only been since 2014 or so that IDs have been merged. What tzdb previously offered was a set of IDs, based on a simple rule - "ID as needed for post-1970 data, with at least one per ISO country". Full history was available for each of these (whether accurate or not). What has happened since is a split, where no split previously existed. A split which favours some locations over others. The recent mailing list debates merely represent the cumulation of this favouritism. The irony is that the merges are being done in the name of equity and fairness, when the outcome has actually been the exact opposite - picking favourites, and denigrating everywhere else. That the approach to picking favourites is according to a standardised largest city rule isn't really that relevant here - it is the outcome that is unfair, not the process. If region IDs were of different appearance (eg. numeric or textually different) then the issues would not have arisen. The mistake was taking a fully functional and fully integral set of IDs, and bifurcating it into two groups. The split was actually a huge change in the policy of tzdb, which has been added drip by drip, rather than something that was ever fully appreciated up front. FWIW, it is clear to me that there is an aspect of imposing a US-centric timezone system on other parts of the world. The recent tzdb approach of focussing entirely on timezone regions makes perfect sense for the US, where region boundaries do not follow state lines, and ordinary members of the public need to be aware of whether they are in US/Mountain or US/Central. This simply isn't the timezone model in many other parts of the world. In places like Europe and Asia, the timezone is driven primarily by the country you live in - an ordinary member of the public in Iceland is never going to associate with some abstract timezone region stretching down the Atlantic that is not named, not legally defined and is little more than a random outcome based on tzdb's choice of 1970. Even in somewhere like Norway, an ordinary member of the public will understand that although they follow CET, their timezone is actually driven by their Government in Oslo. The brilliance of the original rule - "ID as needed for post-1970 data, with at least one per ISO country" - was that it seamlessly handled *both* models of timezone in one unified set of IDs. Removal of the ISO country part has completely destabilised that balance. As a constraint to this thread, tzdb really needs to offer one standard view of data, not command line flags that allow different views. If downstream projects end up with different views of the data, it makes tzdb a much less reliable source. (tzdb can be packaged in different ways on the same machine, for example it is undesirable for Postgres' internal tzdb and the OS tzdb to diverge for the same version). Given this, what needs to be nailed down is what is the default data set that tzdb publishes - there isn't really much point in talking about compile time flags, or that the contents of backzone could be used by someone. Lets look at seven options for pre-1970 data: Pre-1970 data for regions only ----------------------------------------- 1) Pre-1970 data for regions only - Despite looking identical to other IDs, region IDs are treated as special/favoured - Pre-1970 data for non-region IDs is of no importance whatsoever, thus most get pre-1970 data from another country/continent - The split between region and non-region locations is fully completed - the US-centric timezone model is dominant - An end user in Iceland is supposed to use Africa/Abidjan, Europe/Reykjavik is treated as a historical mistake of tzdb kept around only for backwards compatibility Pre-1970 data for most IDs ------------------------------------ 2) Pre-1970 data for each ID meeting the rule "ID as needed for post-1970 data, with at least one per ISO country" - The split between region and non-region locations is healed - High quality data from places like Iceland and Norway is retained, but low quality data from elsewhere is restored - The pre-1970 data is simply viewed as the best available data for the each location which can be improved over time - Most end-users in Iceland would expect tzdb to provide pre-1970 data from Iceland, not the Ivory Coast 3) Pre-1970 data for for all IDs that currently exist except true aliases - This is very similar to #2, but would include something like Montreal which was effectively mistakenly added to tzdb - This doesn't seem as desirable as #2 4) Pre-1970 data for any ID where the pre-1970 data is high quality - Subjective on quality, which doesn't seem like a great idea - It does avoid bringing bad quality data back into the main tzdb distribution - On balance, #2 has fewer places for debate to arise 5) Pre-1970 data for all IDs that currently exist except true aliases, plus a *new* set of IDs representing regions - As per #2, but adding IDs like "Region/12345" or "Region/Berlin" - Regions should contain post-1970 data only, as the region is by definition only meaningful post-1970 - It is not entirely clear what this solves over just going with #2 - This option is connected to Russ' proposals [1], although he suggests a bigger split between timekeeping data and ID naming - Perhaps it makes sense if the new region IDs were internal and not normally seen by end users? Remove pre-1970 data from general use ------------------------------------------------------- 6) No pre-1970 data whatsoever, all IDs are post-1970 only - Project policy is that tzdb is focussed on post-1970 data only - Paul repeatedly tells us that pre-1970 data is unreliable, and people shouldn't use it - Pre-1970 data would not be deleted, but would not be available in most downstreams - The split between region and non-region locations is effectively healed - End users lose access to pre-1970 data, which is particularly notable in some locations where that data is reliable, eg London - It is unknown at this point what the user impact is of removing pre-1970 data from major financial/business centres (no major location has yet been merged) 7) No existing IDs get pre-1970 data, but a *new* set of IDs are created containing it - As per #5, the existing IDs get no pre-1970 data and the split between region and non-region locations is healed - New IDs, such as "Historic/Europe/London" or "Historic/Africa/Abidjan" get created for each of the main IDs - The historic IDs include pre-1970 data, the standard IDs do not - There is a mechanical transformation between the historic and non-historic IDs - Whether downstreams do or do not include the historic IDs is their choice, potentially based on available space - The data provided by the standard non-historic IDs remains the same whether the downstream includes the historic IDs or not (a Good Thing) - Users have to deliberately opt-in to get pre-1970 data, which might make them think about the accuracy issue Notes -------- - I'm not discussing adding new IDs simply to represent locations whose clocks differ only before 1970. I don't personally think that is a job for tzdb, and even if it is, it is a job for a different thread. - I'm not discussing what value is returned prior to 1970 when pre-1970 data is removed. That would be a job for a different thread. - Link vs Zone is not important for this discussion. Summary ------------- After a few weeks of thinking, these are the options I've come up with. Feel free to suggest another option or variant if you think I've missed anything obvious. I believe that was a disaster that the brilliant "ID as needed for post-1970 data, with at least one per ISO country" rule was removed. It has created needless division, bifurcating a unified set of ID into region and non-region IDs and creating backwards compatibility issues in the data of many locations. IMO, there are two basic models of timezones in the world, and moving tzdb from one that supported both to one that only supports the US-centric model is simply a mistake that needs correcting. As such, my preference would be to adopt option 2. Option 6 or 7 could work, and might be the best choices if we had a clean slate or if there was some hidden pressure that the list is unaware of to remove pre-1970 data, but they are risky options given we do not know the impact on end-users in major financial/business centres. Stephen [1] https://mm.icann.org/pipermail/tz/2021-September/030518.html

23 59

The link-name, America/Virgin, is linked to two zones in 'backzone'
by Yan-Jie Wang Nov. 12, 2021

Nov. 12, 2021

A link-name (the second column of the Link line) in a file should be unique. However, 'America/Virgin' is linked to two zones (America/Port_of_Spain and America/St_Thomas) in 'backzone'. The error is in the latest commit as well as in the archive tzdata2021e.tar.gz. The link to the file: https://github.com/eggert/tz/blob/6c15079/backzone Yan-Jie Wang

2 1

Re: [tz] What are the interfaces to the TZDB?
by Brian Park Nov. 10, 2021

Nov. 10, 2021

(Replying back to the list with permission) On Tue, Nov 9, 2021 at 11:45 AM Eliot Lear <lear(a)lear.ch> wrote: > On 09.11.21 20:38, Brian Park wrote: > > Is the TZDB project intended to support only POSIX systems? > > It's a fair question. Before we say, “no”, I would like to understand > what sort systems use the database that themselves are *NOT* POSIX > compliant. But also, as Paul pointed out, the binary format itself is > standardized at the IETF, the sponsor of this shindig. > > We could go up the stack, but before we do so, we should be clear on our > reasons as to why we are doing so. And so when you say: > > My downstream usage does not have a file system, or even an operating > system. For me, the API into the TZDB project are the raw files, but I > understand that my usage is unusual. > > Would you mind being a bit more specific? Are you printing a book? ;-) > Not that primitive... My libraries run on bare-metal, embedded microcontrollers, with no operating system. My first version is in C++, supporting all Zones and Links in about 36kB of flash (8-bit controllers) to 44kB of flash (32-bit controllers), without using 64-bit integers. I have another version in Python that I hope to convert to MicroPython soon. I'm hoping to write another version in Go lang, which will be converted into TinyGo. Brian

2 1

Re: [tz] Pre-1970 data
by dpatte Nov. 5, 2021

Nov. 5, 2021

I was be contributing a longer message on this whole subject within a few days, but I want to state upfront that country/city is insufficient in many countries for a fully expandable location database.Off the top of my head, I can think of three different Richmonds in Canada. One in Quebec, one in Ontario, and one in BC. I know there are many Springfields in many USA states, and let's not even talk about the potential USA/Columbus.The key of a database should be unique and recognizable. For many countries, multiple levels of government are the only way to properly differentiate cities.Canada/British_Columbia/Richmond would be more appropriate.This allows the pre-1970 db to be both infinitely expandable, as well as user friendly to the point of being fully obvious to all users.Sent from my Galaxy -------- Original message --------From: Brian Park via tz <tz(a)iana.org> Date: 2021-11-05 11:27 (GMT-05:00) To: Philip Paeps <philip(a)trouble.is> Cc: Stephen Colebourne <scolebourne(a)joda.org>, Time Zone Mailing List <tz(a)iana.org> Subject: Re: [tz] Pre-1970 data On Thu, Nov 4, 2021 at 10:11 PM Philip Paeps <philip(a)trouble.is> wrote:On 2021-11-05 12:17:34 (+0800), Brian Park via tz wrote: > I get the impression that this debate is caused by the existence of 2 > different schools of thought: [...] > > I want to suggest that it may be possible for these 2 views to > coexist. They de facto coexist right now. The overwhelming majority of the data are descriptive. Only recent efforts have made some of the post-1970 data appear more prescriptive.They coexist in an ad hoc manner right now, and that seems to be one of the causes for the contention. I am suggesting that we formalize the separation, so that both groups are happier. > We > could create a new file, e.g. call it 'countryzone', which contains a > set > of Links organized in a hierarchical tree by country, pointing to the > Core > zones. I strongly believe we should continue to carefully avoid attempting to group data by country. [I would even avoid using the word "country" wherever possible.]Can you explain why? Because it will cause arguments about disputed places? I think only a small minority of places around the world are disputed. By separating these ISO-country timezones into a 'countryzone' file, perhaps we can confine the debate into a smaller section of the TZDB. We could create duplicate entries (i.e. Country1/City, Country2/City), or create a pseudo-country called "Disputed" (i.e. Disputed/City). The point is, we can create policies that govern these disputed regions.Could we move 'countryzone' into a separate project? Probably, but some amount of initial coordination and refactoring would be required to resolve conflicting zone identifiers.Overall, I feel like the TZDB data should lean a bit more towards matching how end-users think about timezones in the real world (Prescriptive), and lean slightly less on treating timezones as a clustering problem (Descriptive). But I can see pros and cons of both approaches. Which is why I am suggesting ways to make the 2 approaches interoperate better. > For the pre-1970 data, it is my understanding that the 'backzone' file > contains Zone records which should replace ONLY the LinkMerged records > found in the other files. I propose that all LinkMerged records be > extracted into a separate file (let's call it 'mergedzone') so that > there > is a clear symmetry between 'backzone' and 'mergedzone', which allows > them > to be substituted for each other. The dependency diagram looks > something > like this: As I've suggested before in another thread, I think we should consider undoing the split into backzone. I really liked Stephen's phrasing earlier in this thread: acceptably accurate, not outrageously wrong. We started moving data to backzone to limit the scope of 'active' maintenance to post-1970 data. That artificial split led us towards a more prescriptive worldview. It seems clear that prescriptive simply does not work for a real world with people on it.I think Paul Eggert has made it clear that he does not want to maintain this data. My proposed refactoring of this info into the 'backzone' / 'mergedzone' pair makes it easy for downstream libraries to add back the 'backzone' data if they want. The 'make PACKRATDATA=backzone' hack does not help downstream libraries which do not use TZif or the Makefile. > If there is any chance that this will result in being able to type > "Canada/Toronto" instead of "America/Toronto", that would resolve an > annoyance that has lasted some 30-35 years. In this context, America refers to the landmass, not to the political entity occupying a large chunk of it. [Canada/Eastern etc moved to backward around 1993, as far as I can tell.]Virtual no one in the world thinks of "America" as referring to all of "North America" and "South America".Brian

2 1

Ref: Tz on social media
by nadir altinbas Nov. 4, 2021

Nov. 4, 2021

is it possible to add 'news' from TZ Coordiation web site share button to social media such as; Linkedin,FaceBook(meta),Instagram,Twitteretc.... There may be occur an anouncment to publicize for sharing via social media? Nadir Altinbas Estanbul /TR, 2021 Nov,2 nadiraltinbas(a)yahoo.com n.altinbas

2 2