Hi Paul thanks a lot. One question are these tables also the one that each OS will include? I mean I need to know how the OS patches itself having the new rules. Did they use your tables? thanks -----Messaggio originale----- Da: Paul Eggert <eggert@cs.ucla.edu> Inviato: lunedì 27 maggio 2019 20:03 A: DONATELLA SABELLICO <DONATELLA.SABELLICO@hcl.com>; kim.davies@icann.org; michelle.cotton@icann.org Cc: Time zone mailing list <tz@iana.org>; Anawesha Khuntia <AnaweshaK@in.ibm.com>; Java Timezone Utility Support <JTZU@uk.ibm.com>; Sbriccoli's L3 only <sbriccolisl3only@hcl.com>; SILVANO LUTRI <silvano.lutri@hcl.com>; Ramanand Patil <ramanand.patil@oracle.com>; Sean Coffey <sean.coffey@oracle.com>; Peter Dempsey <Peter.Dempsey@verint.com> Oggetto: need testing for new tzdb release for Brazil changes DONATELLA SABELLICO wrote:

Do you know when the Timezone package will be released'

We plan to get a new release out reasonably soon, and well before November. However, there is no fixed schedule or specific plan. In the meantime I suggest that you test your software by using the tzdb development version and reporting your results back. You can currently retrieve it by running this command: git clone https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feggert%2Ftz&data=02%7C01%7CDONATELLA.SABELLICO%40hcl.com%7C58c7cd4cfc104d2f1bb108d6e2cd8c2d%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636945769795584444&sdata=KR2zaaXc2xCxFQVFl3KzNUj%2FxwNI7Uf5NBW8E9NATlo%3D&reserved=0 I would like more testing by people other than myself. It's quite possible that the next release will have problems and I want to hear bug reports and "it's all OK!" reports back. I'm not hearing anything, which leads me to suspect that a new release must not be that urgent. Here is one longstanding issue that I would like to see progress on: https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.oracle.com%2Ftechnetwork%2Fjava%2Fjavase%2Ftzupdater-readme-136440.html%23issues&data=02%7C01%7CDONATELLA.SABELLICO%40hcl.com%7C58c7cd4cfc104d2f1bb108d6e2cd8c2d%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636945769795594439&sdata=CxlIH1WAbvCiBj1VyWipQWzVzAPlGF9u%2FJGM8LywiKE%3D&reserved=0 That is, Oracle TZUpdater fails to handle the forms of DST currently used in Ireland and Morocco, and historically used in Czechoslovakia, Japan and Namibia. Although this was reported as a P3 (major loss of function) bug in January 2018 <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8195595&data=02%7C01%7CDONATELLA.SABELLICO%40hcl.com%7C58c7cd4cfc104d2f1bb108d6e2cd8c2d%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636945769795594439&sdata=VJ4Hynids%2FqCqIF8EiioNWZuTBhpIborj54n45cSFqc%3D&reserved=0>, I am not plugged into the JDK development effort and don't know what sort of progress is being made here. Ramanand, you're assigned that bug report. What's the status of the fix for this issue, and of testing the fix? Have you tested with data taken from the current tzdb development version? Although I doubt whether the Brazil changes themselves will cause problems with TZUpdater I've been surprised in the past, and also the next release will contain changes not related to Brazil, so I'd appreciate some feedback here. ::DISCLAIMER:: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

On 2019-05-27 12:02, Paul Eggert wrote:

Here is one longstanding issue that I would like to see progress on: https://www.oracle.com/technetwork/java/javase/tzupdater-readme-136440.html#... That is, Oracle TZUpdater fails to handle the forms of DST currently used in Ireland and Morocco, and historically used in Czechoslovakia, Japan and Namibia. Although this was reported as a P3 (major loss of function) bug in January 2018 <https://bugs.openjdk.java.net/browse/JDK-8195595>, I am not plugged into the JDK development effort and don't know what sort of progress is being made here.

Ramanand, you're assigned that bug report. What's the status of the fix for this issue, and of testing the fix? Have you tested with data taken from the current tzdb development version? Although I doubt whether the Brazil changes themselves will cause problems with TZUpdater I've been surprised in the past, and also the next release will contain changes not related to Brazil, so I'd appreciate some feedback here.

Nothing has happened in the last 12 months, and recent advice in the bug report is still to use the rearguard workaround. Businesses are unlikely to spend money on a risky software update which requires expensive testing in house and by their downstream customers, while docs for a process workaround will suffice for the subset of customers who want to perform their own Java tz updates, perhaps from the dev repo. Some customer account holder, who pays a lot to Oracle for Java support, would have to push their Oracle account manager to have Oracle development work on this. [If Oracle open sourced their tzupdater, some customer volunteer with the need could probably provide a patch quickly.] -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

On 5/27/19 1:48 PM, Paul Goyette wrote:

Is the current rearguard format expected to live "forever"?

No, the idea is only that it's rearguard "enough". That is, even the rearguard is on the march, and every announcement of the publication of the rearguard-format tarballs has said that they're intended to be temporary. The next release (2019b, which should be reasonably soon) may be a good time to skip my publication of a rearguard-format tarball. The main use of the rearguard format seems to be last-minute updates for people using TZUpdater and the like, and we'll almost surely have a release 2019c before October anyway so people can update then. People who need rearguard format tarballs before then can make them by running "make rearguard_tarballs".

On Tue, 28 May 2019 at 17:19, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 5/27/19 1:48 PM, Paul Goyette wrote:

...

Is the current rearguard format expected to live "forever"?

No, the idea is only that it's rearguard "enough". That is, even the rearguard is on the march, and every announcement of the publication of the rearguard-format tarballs has said that they're intended to be temporary.

The next release (2019b, which should be reasonably soon) may be a good time to skip my publication of a rearguard-format tarball. The main use of the rearguard format seems to be last-minute updates for people using TZUpdater and the like, and we'll almost surely have a release 2019c before October anyway so people can update then. People who need rearguard format tarballs before then can make them by running "make rearguard_tarballs".

So, the needs of Java-based libraries have not fundamentally changed since tzdb first started making changes. It is still the case that Java libraries do not want negative DST, and my expectation is that this will always be true. If this means using rearguard forever and shouting about attempts to remove it then so be it. A hack was added to Joda-Time that reverses negative SAVE values which works so long as a rule set does not mix positive and negative save values. This is pretty awful as an approach because it is unreliable. The same hack has not yet been added to ThreeTen-Backport or OpenJDK itself, with both still using rearguard. I can't comment directly on other Java libraries. Joda-Time and ThreeTen-Backport now parse and handle 25:00 as a cutover time. I don't think the associated OpenJDK issue has been fixed. --- Perhaps a discussion could be had on a way to represent the data that Java needs alongside the main data, so that Java parsers can avoid the nasty hack? And remove the need for both hacks and rearguard? Proposal: Add an additional file with legacy versions of the rules that cause problems to downstream parsers. Thus, add a new file "legacy" that contains an alternate version of "Eire" and "Europe/Dublin" (ones with positive DST). Note that the complete set of Rule and Zone lines would be added to legacy, so it could be parsed in its own right as a tzdb source file. I would expect that most parsers would simply ignore the extra file (the main reason for doing it this way). But Java (and other) parsers could choose to parse it and overwrite the negative DST definitions (a much simpler and safer process than the hack that has been added so far). Stephen

On 5/29/19 9:05 AM, Paul Ganssle wrote:

Why exactly does Java not want to implement a parser that supports the current version of the tz database? It seems like they should either make the case for why negative DST is not a good feature to support, or they should adjust to the fact that it is a feature that will be present in the time zone database.

As I vaguely understand it, the difficulty is not merely in fixing the TZUpdater parser, it's that the Java library code internally mishandles negative DST offsets. So, if TZUpdater were changed simply to pass negative DST offsets through, this would cause trouble in production instances of Java relying on current or older libraries, just as setting TZ='IST-1GMT0,M10.5.0,M3.5.0/1' in the environment presumably causes trouble in production instances of Java. (That TZ setting exercises the support for negative DST that's been standardized by POSIX since 1988.) If I'm right, one workaround would be for TZUpdater to compute the most negative DST offset in use for a zone, and treat that offset as standard time while treating the other DST offsets as daylight-saving time. That way, TZUpdater's output would contain only nonnegative DST offsets and the Java library code would be happy. I don't see anything in the current Java standard API that prohibits negative DST offsets, and a cursory look at the code for the Java standard library doesn't reveal any problems either, so I guess the problem with negative offsets is deep in the Java library implementation. Or perhaps the problem doesn't exist in the current API and/or library, but does exist in older versions. I could well be wrong about all this, though, as I don't use the Java timezone API or code.

On 5/30/19 2:14 AM, Stephen Colebourne wrote:

It is a compatibility requirement of the main Java libraries (including icu4j IIUC) that DST is always positive.

Where is this documented? And what do the main Java libraries do in an environment where the DST offset is negative? For example, if I run the following shell command on a POSIX-compatible host: TZ='IST-1GMT0,M10.5.0,M3.5.0/1' java [arguments] can you give an example of what goes wrong and why? This command gives Java the rules that Ireland has used since 1996, with Irish Standard Time (1 hour ahead of GMT) used in the summer and GMT (a "negative daylight saving time") in the winter.

Unfortunately I don't run on Linux, so can't check this. I've tried looking for the relevant source code in OpenJDK, but haven't seen anything obvious that might parse this format. This was the closest I got: http://hg.openjdk.java.net/jdk/jdk/file/f91999057a5a/src/java.base/share/nat... http://hg.openjdk.java.net/jdk/jdk/file/f91999057a5a/src/java.base/unix/nati... Stephen On Fri, 31 May 2019 at 03:08, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 5/30/19 2:14 AM, Stephen Colebourne wrote:

...

It is a compatibility requirement of the main Java libraries (including icu4j IIUC) that DST is always positive.

Where is this documented? And what do the main Java libraries do in an environment where the DST offset is negative? For example, if I run the following shell command on a POSIX-compatible host:

TZ='IST-1GMT0,M10.5.0,M3.5.0/1' java [arguments]

can you give an example of what goes wrong and why? This command gives Java the rules that Ireland has used since 1996, with Irish Standard Time (1 hour ahead of GMT) used in the summer and GMT (a "negative daylight saving time") in the winter.

On 2019-05-31 10:53, Guy Harris wrote:

On May 31, 2019, at 8:31 AM, Stephen Colebourne wrote:

...

Unfortunately I don't run on Linux, so can't check this. Do you mean "don't run on Linux" or do you mean "don't run on any flavor of UN*X"? Paul said "if I run the following shell command on a POSIX-compatible host", which is inequivalent to "on Linux" (especially given that few if any Linux distributions have been run through the Single UNIX Specification test suite to validate them as POSIX-compliant). Is there something special here about Linux that means that some other UN*X won't suffice?

You don't need Linux, any POSIX compatible shell *sh on Windows or Mac will handle the TZ env var, with the capability also of launching the platform java, assuming one is available, like OpenJDK. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

Stephen Colebourne wrote:

Unfortunately I don't run on Linux, so can't check this. I looked into this and found that on Fedora 30 x86-64, OpenJDK 12.0.1 mishandles not only *negative* DST specified via a POSIX TZ string, it also mishandles *positive* DST specified via a POSIX TZ string. On the other hand, this Java works fine with TZ='Europe/Dublin' even though Europe/Dublin has negative DST on my platform. See attached shell transcript. I am using OpenJDK 12.0.1 linux-x64 which I got from download.java.net.

So what I'm observing on this platform is: * Java supports negative DST when it comes from a TZif file. * Java does not support POSIX TZ strings that specify DST, regardless of whether the DST is positive or negative. Although this is just one Java platform, it appears that I may have been worrying overmuch about Java and tzdb's rearguard format, in the sense that the problem appears to have been fixed on Fedora without modifying the Java libraries themselves.

Stephen Colebourne wrote:

JDK only really handles identifiers like Europe/Dublin.

JDK does handle the POSIX-format TZ setting TZ='MST7', even though MST7 is not a tzdb Zone name, so it appears that JDK already supports POSIX-format TZ settings that lack daylight saving rules.

I have't see much evidence that Java developers think the lack of support of the POSIX string is a problem. Perhaps they'll think it's a problem after reviewing this discussion thread. Had Java been upward-compatible with POSIX-format TZ settings all along, then we would likely have avoided this compatibility mess. When POSIX came out in 1988 tzdb was not 100% upward-compatible with POSIX, but we changed tzdb long ago to make it compatible and Java surely can do the same. What differs is the value returned by ZoneRules:

Yes, it appears that Fedora does the following: 1. Install tzdb's main-format .zi and TZif files, so that most programs think Ireland has negative DST. 2. Install openjdk's time zone data generated from rearguard-format .zi files, so that Java programs think Ireland has positive DST. The two methods agree everywhere, in that all Irish timestamps have the same UTC offset, the same "IST"/"Irish Standard Time" names, etc.; *except* that the two methods disagree about the DST flag (tm_isdst in the POSIX model, isDaylightSavings in the Java model). This discrepancy is so minor that it apparently doesn't matter. As Robert Elz writes, programmers shouldn't rely on the DST flag anyway (for other reasons), and we can simply add the Ireland glitch to the list of reasons that programmers should avoid the DST flag. Of course it would be nice for Java and tzdb to agree about the DST flag warts and all, and I suggest that the Java folks do so when they fix Java's mishandling of POSIX-format TZ strings. But this apparently isn't crucial since we've known about the problem for some time and systems are being delivered with this minor DST-flag discrepancy without major problems.

[The Java documentation] doesn't explicitly claim that the savings is positive

Yes, that was a real eye-opener to me. It means the Java developers could add negative-DST support to Java without changing Java's documented API. From my point of view it's a bug fix that's needed for POSIX compatibility anyway.

The rearguard format is needed as the input to TZUpdater

If that's the problem, it can be addressed by fixing TZUpdater. Even if the Java developers want to outlaw negative DST (which in my view would be a mistake, because Java would be incompatible with POSIX), they could change TZUpdater to do the equivalent of ziguard.awk to translate the input into rearguard form before continuing to process the input. Better, though, would be to be upward-compatible with POSIX and allow negative DST; i.e., the Java developers could modify TZUpdater to support negative DST consistently with Java's longstanding API and documentation.

The purpose of this thread from my perspective is mainly to highlight that the data we extract from rearguard is going to be needed forever.

It'll be needed "forever" by TZUpdater only if the Java developers adopt the hack where TZUpdater generates the rearguard format internally, and at that point it's TZUpdater's responsibility to keep the rearguard format going. I suggest, though, that TZUpdater be modified to support negative DST since negative DST is needed for POSIX compatibility anyway.

On Jun 2, 2019, at 10:10 AM, Stephen Colebourne <scolebourne@joda.org> wrote:

On Sat, 1 Jun 2019 at 18:59, Paul Eggert <eggert@cs.ucla.edu> wrote:

...

...

[The Java documentation] doesn't explicitly claim that the savings is positive

Yes, that was a real eye-opener to me. It means the Java developers could add negative-DST support to Java without changing Java's documented API. From my point of view it's a bug fix that's needed for POSIX compatibility anyway.

...

I've also indicated that there is no desire to support full POSIX: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4263805

Presumably meaning "in theory, Java could be run, on a system that supports the full POSIX syntax for TZ, with TZ set to arbitrary POSIX-compliant values, including values in which clocks are turned *backwards* from standard time in the autumn and winter, but, in practice, that's insufficiently likely that there's any reason to make Java support it".

What is unfortunate is the lack of desire to engage on the substantive point: * that both ways of describing the data are valid

"Valid" in what sense? Yes, one could, when defining an API, either say: "standard time" is the time in effect in autumn and winter, even if that's not what some country's law defines as standard time, and clocks are turned forward from standard time for the spring and summer and turned back for autumn and winter; "standard time" is the time that the country's law defines as standard time, and clocks might be turned forward from standard time for the spring or summer, or might be turned backward from standard time for the autumn or winter. For the tzdb, there is at least one API for which some implementations use the tzdb and for which it's not entirely clear which of those two interpretations is intended - namely the API for which the tzdb was originally created, and which remains an API for several OSes using the tzdb, such as most Linux distributions, the *BSDs, macOS, and Solaris, i.e. the POSIX API. What the Single UNIX Specification page for <time.h>: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/time.h.html says is The value of tm_isdst shall be positive if Daylight Savings Time is in effect, 0 if Daylight Savings Time is not in effect, and negative if the information is not available. On the other hand, what the same specification's page for Environment Variables: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html says, of the TZ environment variable, is: The expanded format (for all TZ s whose value does not have a <colon> as the first character) is as follows: stdoffset[dst[offset][,start[/time],end[/time]]] Where: std and dst Indicate no less than three, nor more than {TZNAME_MAX}, bytes that are the designation for the standard (std) or the alternative (dst -such as Daylight Savings Time) timezone. Only std is required; if dst is missing, then the alternative time does not apply in this locale. suggesting that there's "standard time" and "alternative time", where "Daylight Savings Time" is an example of an "alternative time", but isn't necessarily the *definition* of "alternative time". Before we conclude that ""standard time" is the time in effect in autumn and winter" is the POSIX definition, we should try to get this resolved by the Open Group, so we know which of the two definitions of "standard time" are valid in POSIX, and whether "tm_isdst" means "clocks have been turned forward" or "clocks are have been turned forward or backward from the legally-defined standard time". If the decision is that the valid definition of "standard time" is "standard time as specified by law", and "tm_isdst" means "clocks have been turned forward or backward from the legally-defined standard time", then the tzdb needs to support that in some fashion, so that, for Europe/Dublin, "tm_isdst" is set for times during the autumn and winter" and that the "timezone" variable: https://pubs.opengroup.org/onlinepubs/9699919799/functions/daylight.html be set to one hour in advance of UTC. This would mean that it would need to distinguish between the way clocks are adjusted in most countries that do DST and the way they're adjusted in the Republic of Ireland - even though that might cause inconvenience for software using the "standard time is the time in effect in autumn and winter" view. Paul, how should we go about requesting an interpretation from TOG here? Presumably that would be done through the "Austin Group": https://www.opengroup.org/austin/

On Jun 2, 2019, at 11:39 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

...

Paul, how should we go about requesting an interpretation from TOG here?

I could do that, but I don't see what sort of interpretation is needed.

An interpretation to indicate whether "tm_isdst" means "is Daylight Saving{s} Time", for "Daylight Saving{s} Time" presumably meaning "the clocks are turned forwards for late spring and summer", or means "is the alternative timezone", in the sense of "alternative timezone" from the "Environment Variables" page section on the TZ environment variable, is needed.

POSIX clearly requires support for a TZ setting like 'IST-1GMT0,M10.5.0,M3.5.0/1' that specifies negative DST, and that is true regardless of what Ireland happens to be doing right now.

But, with that TZ setting, does it: 1) require that tm_isdst be 1 from 1AM Sunday in the last week of October until 1AM Sunday in the last week of March, because that's the "alternative timezone"; 2) require that tm_isdst be 1 from 1AM Sunday in the last week of March until 1AM Sunday in the last week of October, because that's when the clocks are turned forward - or, rather, *aren't* turned *backward* from standard time; 3) say it's implementation-specified which of those is the case?

Stephen Colebourne said:

...

Yes, that was a real eye-opener to me. It means the Java developers could add negative-DST support to Java without changing Java's documented API. From my point of view it's a bug fix that's needed for POSIX compatibility anyway. With Java, it is wrong to assume that the documented API is the only issue when considering a change.

Then how do I, as a developer, know what I am and am not promised by the API? That's the whole point of having an API and documenting it - it's a contract between the two parties and you can only rely on what is in the contract. That's the basis on which we wrote the C Standard. There may be ambiguities but if it's not mentioned at all then you can't rely on it. (The classic example in C is that a "byte" is not required to be 8 bits, just at least 8 bits. I've programmed on a machine where a byte - the smallest addressable object - was 16 bits.) -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

On Sun, 2019-06-02 at 18:10 +0100, Stephen Colebourne wrote:

As I, and others, have said, I don't believe there is any appetite to change the meaning of DST in Java - the pain would be far too great with no or negative benefits.

Perhaps it would be helpful to review what happened in the POSIX world in a similar situation. The man page for gettimeofday(3) on Linux has contained this narrative for as long as I can remember (mid/late 1990s): *** snip snip *** NOTES Traditionally, the fields of struct timeval were of type long. The tz_dsttime field has never been used under Linux. Thus, the following is purely of historic interest. On old systems, the field tz_dsttime contains a symbolic constant (values are given below) that indicates in which part of the year Daylight Saving Time is in force. (Note: this value is constant throughout the year: it does not indicate that DST is in force, it just selects an algorithm.) The daylight saving time algorithms defined are as follows: DST_NONE /* not on DST */ DST_USA /* USA style DST */ DST_AUST /* Australian style DST */ DST_WET /* Western European DST */ DST_MET /* Middle European DST */ DST_EET /* Eastern European DST */ DST_CAN /* Canada */ DST_GB /* Great Britain and Eire */ DST_RUM /* Romania */ DST_TUR /* Turkey */ DST_AUSTALT /* Australian style with shift in 1986 */ Of course it turned out that the period in which Daylight Saving Time is in force cannot be given by a simple algorithm, one per country; indeed, this period is determined by unpredictable political decisions. So this method of representing timezones has been abandoned. *** Snip snip *** I find it significant that POSIX apparently ended up ditching an entire API element in favor of adding a model capabale of handling the full complexities of civil timekeeping (as they were understood at the time); a move which surely must have caused heartburn for many POSIX- based applications in use at that time. Can anyone here recall the context and complexities surrounding this transition? Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | A room without books is like a body without a soul. | | | | -- Cicero | |---------------------------------------------------------------------|

On Jun 3, 2019, at 1:19 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 6/3/19 9:44 AM, Fred Gleason wrote:

...

I find it significant that POSIX apparently ended up ditching an entire API element in favor of adding a model capabale of handling the full complexities of civil timekeeping (as they were understood at the time); a move which surely must have caused heartburn for many POSIX- based applications in use at that time.

I suppose it must have caused some problems, although the applications world was smaller then. I don't recall any problems being publicized.

As per my previous "more than you ever wanted to know about the history of time zone handling in UNIX since V7" email 1) it was less "ditched" than "not adopted", as not all systems had adopted it; 2) at least some, if not all, of the systems using it had adopted tzcode, making that API element obsolete; 3) that API element was largely used only by localtime() and mktime(), so invisible to most applications doing conversion between UNIX time and local time; so I suspect the problems were more for OS vendors then for applications - and even those were minimal for vendors who had adopted tzcode - or independently implemented it.

The precise road to the current spec for POSIX TZ strings is a bit obscure to me. I vaguely recall that in 7th Edition Unix the time zone was a kernel setting (and perhaps you had to rebuild the kernel?);

The standard time offset from UTC was a kernel setting, and you had to rebuild the kernel and reboot or, if you're ambitious, fire up adb, as root, against /unix and /dev/kmem and patch the variable. The time zone rules were compiled into ctime(), and you had to change it, rebuild it, relink programs that needed the new localtime(), and reinstall the new library and executables.

presumably this inspired the gettimeofday API you mention, which was introduced some time between 4BSD (1980-10) and 4.1cBSD (1982-12),

Probably in one of the 4.1[a-z]BSD releases, as it was, as I remember, present in 4.2BSD.

and was declared obsolete sometime between 4.3BSD Reno (1990-06) and 4.4BSD (1993-06);

...because they'd picked up tzcode.

presumably Linux (released before 4.4BSD came out) picked up the API from 4.3BSD.

Many OSes picked up gettimeofday(), mainly to be able to get time stamps with finer-than-one-second resolution. As I remember, Roland McGrath put an independent implementation of code to use the tzdb into GNU libc (there are probably emails about it in the timezone mailing list archives), so Linux would have had a tzcode-equivalent implementation of localtime()/mktime() since the first "GNU/Linux" was available, and would have had no need for the second argument to gettimeofday().

At some point the time zone was put into the user environment via the TZ variable, but only the forms TZ=XXXn and TZ-XXXnYYY were supported, with US daylight-saving rules being hardcoded in the C library; I don't know when and where this was first done,

That dates back to System III, if not earlier.

but it made the kernel setting (and thus the 2nd argument of gettimeofday) obsolete.

System Roman Numeral never had gettimeofday() (until SVR4); I don't know whether it picked up ftime() from V7 and then dropped it in favor of the TZ environment variable or whether whatever AT&T group was handling the internal "product" versions of UNIX decided, either before V7 added ftime() or when they found out about ftime(), to go with an environment-variable based system instead.

However, this didn't work for non-US rules and vendors had nonportable ways to address this. In July 1986 the US changed its daylight-saving rules (effective April 1987) and by then it surely had became clear to everybody that something better had to be done. The POSIX committee settled on something close to the current TZ design in the first POSIX edition in 1988, where they purposely did not standardize gettimeofday and suggested that something would be standardized later; by the time gettimeofday made it into POSIX its timezone argument was completely vestigial.

It's possible the POSIX TZ design was a committee invention, and did not reflect any specific implementation at the time.

It seems to reflect the SVR3.1 implementation; the current Single UNIX Specification rationale doesn't seem to mention SVR3.1 or any other existing practice in the "Environment Variables" section.

Guy Harris and Arthur David Olson would likely know the history better, as they were active in this area then. The rationale of POSIX 1003.1-1988 cites the early tzdb work as the "Olson/Harris method"

And I eventually got people to stop mentioning my name - I guess if you want to call it the Olson/Elz/Harris/... method (apologies to others subsumed into the "..."), you can go ahead, but as a non-originator I don't deserve any more credit than any other non-originator (Arthur being the originator, with the initial code and announcement).

and says it would be a conforming extension to POSIX if it were fixed up a bit (which it was, eventually).

":Europe/Berlin" is a TZ setting allowed by the SUS (and maybe by 1988 POSIX, which I *think* had the : prefix); its behavior is implementation-defined by the SUS. So, yes, in that sense it's conforming.

On Jun 3, 2019, at 9:44 AM, Fred Gleason <fredg@paravelsystems.com> wrote:

On Sun, 2019-06-02 at 18:10 +0100, Stephen Colebourne wrote:

...

As I, and others, have said, I don't believe there is any appetite to change the meaning of DST in Java - the pain would be far too great with no or negative benefits.

Perhaps it would be helpful to review what happened in the POSIX world in a similar situation. The man page for gettimeofday(3) on Linux has contained this narrative for as long as I can remember (mid/late 1990s):

*** snip snip *** NOTES Traditionally, the fields of struct timeval were of type long.

The tz_dsttime field has never been used under Linux. Thus, the following is purely of historic interest.

On old systems, the field tz_dsttime contains a symbolic constant (values are given below) that indicates in which part of the year Daylight Saving Time is in force. (Note: this value is constant throughout the year: it does not indicate that DST is in force, it just selects an algorithm.) The daylight saving time algorithms defined are as follows:

DST_NONE /* not on DST */ DST_USA /* USA style DST */ DST_AUST /* Australian style DST */ DST_WET /* Western European DST */ DST_MET /* Middle European DST */ DST_EET /* Eastern European DST */ DST_CAN /* Canada */ DST_GB /* Great Britain and Eire */ DST_RUM /* Romania */ DST_TUR /* Turkey */ DST_AUSTALT /* Australian style with shift in 1986 */

Whoever wrote that part of the Linux man page used a definition of "old systems" that means "systems starting with 4.2BSD" rather than, for example, "systems starting with Seventh Edition UNIX". The "tz_dstime" page was introduced in 4.2BSD; it was not present in V7 UNIX, or in any of the System Roman Numeral versions of UNIX from AT&T.

I find it significant that POSIX apparently ended up ditching an entire API element in favor of adding a model capabale of handling the full complexities of civil timekeeping (as they were understood at the time); a move which surely must have caused heartburn for many POSIX- based applications in use at that time.

V7 UNIX had a system that, to handle the full complexities of civil timekeeping in any given location, would require a source code change to ctime.c: https://www.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/libc/gen/ctime.c and a recompile of the C library and all programs using ctime()/localtime() to handle any location in which the rules for when DST begins and ends don't match those of the US in effect from 1976 to 1986, or in which the time change is anything other than "turn the clocks forward by one hour when it begins and back one hour when it ends". The standard-time offset from UTC could be obtained with an ftime() system call; changing it required recompiling the kernel, installing the new kernel, and rebooting. 4.2BSD improved this slightly. It allowed the standard-time offset from UTC to be set with a system call, so that no kernel recompile and reboot was required, and allowed selection from a small set of Daylight Saving Time rules, with the rule also settable with a system call. The rules for each set, however, were still compiled into ctime.c, so changing the rules would still require changing ctime.c and recompiling the C library and all programs using ctime()/localtime(), and adding a new rule set would, in addition to that, involve changing the header that defines the values for the types of DST rules, adding a new type for use by ctime.c. Meanwhile, at least according to a manual of System III vintage (it appears to have been an internal AT&T manual for "UNIX 3.0" rather than the documentation issued for the public release of UNIX 3.0/System III, but it's probably close enough): http://bitsavers.org/pdf/att/unix/System_III/UNIX_Users_Manual_Release_3_Jun... As per ctime(3), TZ could be set to specify the abbreviations for standard and Daylight Saving Time, as well as the offset from UTC of standard time in hours, but could not specify non-hour offsets or DST rules or change offsets other than the standard US ones (see previous comment about "change ctime.c and do a bunch of recompiling"). A purported System V Release 1 manual: http://bitsavers.org/pdf/att/unix/System_V_Release_1/301-905_UNIX_System_V_R... says pretty much the same thing in ctime(3), as does a purported SVR2 manual: http://bitsavers.org/pdf/att/unix/System_V_Release_2/UNIX_Programmers_Manual... and a purported SVR3 manual: http://bitsavers.org/pdf/att/unix/System_V_Release_3/UNIX_System_V_Programme... An SVR3.1 manual for the 32-bit x86 port: http://bitsavers.org/pdf/att/unix/System_V_386_Release_3.1/386_UNIX_System_V... first shows a TZ syntax capable of handling non-US rules; SVR3.1 came out sometime in the late 1980's, probably 1987-1988. That syntax was probably the basis of the POSIX syntax; the first POSIX release came out in 1988. The "Olson database" effort from which came the tzdb and the tzdb code was started sometime in 1986: https://tz.iana.narkive.com/dGZADdCn/zoneinfo-questions so it predated the release of the first POSIX specification. The POSIX committee didn't, I think, want to standardize the tzdb, so as not to overly constrain implementations, but they allowed TZ to begin with a colon, followed by implementation-dependent text, to allow alternative ways of specifying the time zone, and I think that was done primary if not entirely to allow tzdb IDs to be provided. By 1988, whatever BSD-based systems had the time zone stuff in gettimeofday()/settimeofday() had, I think, switched to using the tzdb code and data, so that the 4.2BSD-style way of specifying the time zone was provided only for backwards compatibility. So it was 1) less capable than the SVR3.1 scheme and 2) not really being used any more, so there wasn't any reason for POSIX to standardize on it. I don't remember whether tzcode ever paid any attention to the 4.2BSD-style time zone setting on systems where it was available, e.g. mapping the values for various regions to tzdb IDs and using that ID. It may not have. Eventually, SVR4 also picked up tzcode and tzdata via the Sun/AT&T work. I'm not sure what complexities, if any, the transition from the 4.2BSD scheme, on systems providing it, to the tzdb scheme added. I forget when I added the initial support for POSIX-style TZ settings in tzcode; that pretty much made support for POSIX-style TZ settings Just Work on systems using tzcode. I don't know whether any systems supported the 4.2BSD scheme and didn't end up picking up the tzcode. Systems *not* using the 4.2BSD scheme just had to pick up later SV code to get support for the SVR3.1 scheme; I don't know whether SVR3.2 added support for whatever in the POSIX scheme wasn't already in the SVR3.1 scheme, but, if so, picking up that code would have added POSIX compatibility for TZ, and the vendors could also have added that themselves.

On Jun 3, 2019, at 4:43 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

Thanks for that history, much of which I was not aware of. Would it be OK to put that text into the tz distribution somewhere? Perhaps a new file 'tzdb-history.html'? The idea is we should record a more-complete early history of the project than what's in NEWS.

If combined with Robert Elz' response, and with any disclaimers I put in about incomplete knowledge or other forms of uncertainty included, yes, it'd be OK.

Wow. Thanks for all that detail, Guy. For anyone at risk of drowning in it, though, the simpler answer (as I think someone else already pointed out) is that the second argument to gettimeofday wasn't sufficient and wasn't relevant to most application code anyway. The vast majority of application programs just want to convert UTC to local time, or local back to UTC. They don't really care precisely where the C localtime() function gets its timezone data from. Putting that data into an environment variable (as SysV and Posix did) or into a database keyed by an environment variable (as the ado tzdb does) offered huge advantages, and moreover the mechanism was more or less invisible to most application code. So abandoning gettimeofday's second argument was easy, almost natural. But I'm not sure that's the right question to ask. The more relevant question for this thread, I think, is "Why has the C/Unix world mostly abandoned the tz_isdst field of struct tm?" And the answer to that question is that tm_isdst isn't usually necessary or sufficient, either. On output from localtime(), anyway, tm_isdst doesn't tell you anything you need to know. What you may be more interested in is a name or abbreviation for the local time zone, or the time delta between the local time and UTC. tm_isdst is of no use (at least, not by itself) in answering those questions; what you really want is the tm_zone and tm_gmtoff fields (which are, sadly, not official, but that's another story). So the analogous question for me is, "What are Java programs using isDaylightSavings for? What problems would it cause them if isDaylightSavings's output changed, or if it disappeared? Would some other differently-defined function serve the needs of those programs better?" I don't know much about Java (and nothing about its standardization process), so I take Stephen Colebourne at his word when he says that isDaylightSavings is not going to change for the foreseeable future. But I do still wonder what programs are using it for. * If they're using it as part of their own homebrew conversion between UTC and local time, well, they're doing it wrong, and they really ought to change. * If they know about time zones and the details of time zones (perhaps as part of a user interface that presents these details to users), they definitely need to have a good understanding of the situation in a region that observes DST or some similar switchover. But as Ireland and this thread have shown us, a model based on a simple binary distinction between "standard time" and "DST" doesn't really work. I wonder if, for some of those programs at least, slavishly presenting Ireland as standard=GMT,DST=UTC+1=IST doesn't cause those programs to do precisely the wrong thing? I wonder if a different choice for "DST" mapping (or a different API) wouldn't serve those programs' needs better? * If they're doing something else, I'd really like to know what it is. I guess my bottom line is, again, to wonder what preserving the status quo for Java DST indications actually accomplishes. Yes, it upholds (some part of) a standardized API, and interface stability is obviously good. But in the last analysis, does a Java program that cares abut DST *really* want to be told that a timestamp in the summer in Ireland does use it, or would it rather be told something else?

On Tuesday, June 4 2019, "Guy Harris" wrote to "Steve Summit, tz@iana.org" saying:

BTW, what happens on a system that *does* have them if you:

call localtime()/localtime_r();

set the TZ environment variable in your program to a new value and call tzset();

try to use the tm_zone string from the previous localtime()/localtime_r() call?

Is the tm_zone string guaranteed to survive the tzset() call?

No, its value becomes undefined. https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/strft... : If a struct tm broken-down time structure is created by localtime() or localtime_r(), or modified by mktime(), and the value of TZ is subsequently modified, the results of the %Z and %z strftime() conversion specifiers are undefined, when strftime() is called with such a broken-down time structure. This was specifically written to allow the behavior where strftime's %Z substitutes tm->tm_zone. -- Jonathan Lennox lennox@cs.columbia.edu

On Tue, 4 Jun 2019 at 17:49, Steve Summit <scs@eskimo.com> wrote:

So the analogous question for me is, "What are Java programs using isDaylightSavings for? What problems would it cause them if isDaylightSavings's output changed, or if it disappeared? Would some other differently-defined function serve the needs of those programs better?"

As discussed when the issue started a couple of years ago, the flag is used to select the name to display: public String getDisplayName(boolean daylight, int style, Locale locale) https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/TimeZ...) The assumption is that daylight=summer/advanced, and this goes a long way down the stack, notably into CLDR. No one in Java-land is saying this API is perfect. Its absoluely clear that a single boolean is insufficient to lookup a name accurately in cases like Ramadan. But it has been this way for 20+ years and isn't going to change any time soon. It also produces an answer that as correct as the underlying data (CLDR) where there are only "standard" and "daylight" names. Note that CLDR treats "standard" as winter and "daylight" as summer/advanced universally. https://www.unicode.org/reports/tr35/tr35-dates.html#Time_Zone_Names https://www.unicode.org/cldr/charts/35/verify/zones/en.html Remember that the offsets from UTC in Java are always right, and this covers the vast majority of use cases for time-zones. A much smaller percentage of end-users care about the name of the time-zone, ie. the difference between Irish Time (all year round), IST or Irish Summer Time (summer), and GMT or Greenwich Mean Time (winter). But what those people care about is that the text "Irish Summer Time" is returned in summer. When tzdb changed its format, end-users started getting "Irish Summer Time" in winter, because the boolean flag had been reversed - thats why rearguard or a parser hack has to be used to keep the boolean flag as daylight=summer. What almost no end-user cares about is what Irish legislation says (and I don't think Java should expose that knowledge). As a downstream consumer, I don't care what the legal status of a period is - I care whether it is winter or summer/advanced so I can lookup the name in CLDR. Stephen

Robert Elz said:

What are you planning to do when some jurisdiction does summer/retarted ?

And before anyone claims that's absurd, it isn't really. Queensland [...]

Didn't Palm Springs used to do winter/advanced because of a large mountain? -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

Robert Elz wrote:

Stephen Colebourne wrote: | The assumption is that daylight=summer/advanced, and this goes a long | way down the stack, notably into CLDR.

What are you planning to do when some jurisdiction does summer/retarted ?

I'm not Stephen, but I assume the answer would be, "We can cross that bridge when we get to it; we don't have to solve it now", and I don't think that's necessarily a completely indefensible position. You're basically trading off a huge amount of pain now to solve a problem that *might* happen later, versus the exact same amount of pain later if/when you actually do have the problem, except by then, you and everyone else who has to endure the pain are much more motivated to do so.

On Jun 5, 2019, at 3:51 AM, Stephen Colebourne <scolebourne@joda.org> wrote:

A much smaller percentage of end-users care about the name of the time-zone, ie. the difference between Irish Time (all year round), IST or Irish Summer Time (summer), and GMT or Greenwich Mean Time (winter). But what those people care about is that the text "Irish Summer Time" is returned in summer. When tzdb changed its format, end-users started getting "Irish Summer Time" in winter, because the boolean flag had been reversed - thats why rearguard or a parser hack has to be used to keep the boolean flag as daylight=summer.

For what it's worth, a Google search of "Irish Summer Time" site:.ie gets "About 921 results" and a Google search of "Irish Standard Time" site:.ie gets "About 1,060 results", so I'm not sure whether, in Ireland, the correct term for time during the summer is "Irish Summer Time" or "Irish Standard Time", or whether either one are considered valid and both are used interchangeably. If I further limit it with "site:.gov.ie", I see one reference to "Irish Summer Time": https://data.gov.ie/dataset/opw-hydrometric-network-real-time-water-level-da... which is a report of water levels that says Times are reported in UTC/GMT. Irish Summer Time is one hour ahead of this. and three references to "Irish Standard Time": https://assets.gov.ie/4900/181218122604-060b342e59ea48a09953a94bd4301cbd.pdf which says Closing Date for Applications: Friday, 25 May at 18:00 (Irish Standard Time – UTC+1) and https://www.dfa.ie/irish-consulate/sao-paulo/news-and-events/2017/goiies-sch... which says Please note that the closing date, for receipt of proposals from Irish HEIs, is 17.00 (Irish Standard Time) on 26 May 2017. and https://www.dfa.ie/media/embassycyprus/newsandevents/One-Minute-for-One-Mill... which says The closing date for submissions is 18.00 (Irish Standard Time) Monday 7 September 2015. (and a fourth reference to the same event, but I'm not counting it separately). (Some of those aren't the sites directly pointed to by the Google links - in one case, the page moved, and I dug it up elsewhere, and, in other cases, I de-HTTPSed the URL scheme to get Safari not to warn me that the site wasn't secure, and got redirected to another HTTPS site that didn't provoke the warning.) So my guess is "in Ireland it's both Irish Summer Time *and* Irish Standard Time, either name's OK" is the answer.

What almost no end-user cares about is what Irish legislation says (and I don't think Java should expose that knowledge).

I.e., either 1) Java shouldn't offer a single unchanging per-zone offset from UTC or 2) if it does, it shouldn't offer any guarantee that it's the standard offset from UTC as defined by law. I would vote for 1) here, given that a tzdb region can change what time zone it's in (which is why I don't like calling tzdb regions "time zones").

On 6/5/19 3:51 AM, Stephen Colebourne wrote:

But it has been this way for 20+ years and isn't going to change any time soon. It also produces an answer that as correct as the underlying data (CLDR)

I'm not sure it's fair to pass this buck to CLDR. CLDR is trying to maintain timezone strings that are suitable for Java (as almost nobody else uses them) and so if Java is messed up, CLDR will likely be equally messed up, to stay compatible with Java. Even now (that is, with rearguard format to pacify Java) there are still problems with time in Ireland due to Java's confusion on this issue (as mimicked by CLDR). See the example at the end of this email, where OpenJDK 12 says that local time should be called "GMT" / "Greenwich Mean Time" in Ireland on January 1, 1970, which is obviously wrong as Irish clocks were one hour ahead of UTC then (and Java gets the UTC offset right) and nobody on the ground then would have called that time "GMT". That being said, I'm not that pessimistic about whether these problems will be fixed. They are obviously fixable; all we need is for Oracle to do the fix in OpenJDK, and I'm sure that they can do it if they want to. In the meantime the problems do not seem to be that big a deal in practice. $ jshell |  Welcome to JShell -- Version 12.0.1 |  For an introduction type: /help intro jshell> var epoch = java.time.Instant.EPOCH epoch ==> 1970-01-01T00:00:00Z jshell> var zone = java.time.ZoneId.of("Europe/Dublin") zone ==> Europe/Dublin jshell> var dtf = java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss Z z (zzzz)") dtf ==> Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOf ... RT)' ''('ZoneText(FULL)')' jshell> epoch.atZone(zone).format(dtf) $4 ==> "1970-01-01 01:00:00 +0100 GMT (Greenwich Mean Time)"

On 6/6/19 2:25 AM, Stephen Colebourne wrote:

Java has never provided accurate time zone names for historic times. I doubt any system does

Quite true, in large part because accurate time zone names do not exist for most historic times. That being said, there's no doubt that for Ireland in 1970, tzdb gets it right whereas Java gets it wrong.

Mark Davis ☕️ wrote:

CLDR has misleading IDs for the different variants of a zone name:

generic, standard, daylight.

While these are invisible to users, they can be misleading to programmers. Names that would have been more reflective of the intended use would have been something like:

generic, offset0, offset1

Although that would be an improvement, it's not enough because it's not historically correct to couple UTC offsets to zone names. For example, tzdb says America/Los_Angeles historically has had three offsets (-07:52:58, -08, -07) but has had four time zone abbreviations because -07 has usually been abbreviated PDT but was abbreviated PWT during 1942-5. And if California moves to permanent -07 as it is threatening to do, -07 may be called MST or PST soon in Los Angeles with tm_isdst equal to 0. This all works fine in tzdb, but it appears that CLDR/Java cannot handle it. CLDR/Java has gotten away so far with these glitches because they have affected only timestamps from decades ago (Ireland circa 1970, US 1942-5, etc.) so hardly anybody cares. However, the US and Europe are quite likely to change time zone names and/or rules in the near future, in such a way that these problems will happen with contemporaneous timestamps. When this happens, lots of people will rightly complain (and many people will mistakenly blame tzdb :-). Now is a good time for Java and CLDR to fix this. Better now than six months from now.

On 2019-06-10 13:27, Matt Johnson wrote:

I'd just like to point out the obvious - LOTS of other things use the CLDR time zone names data.  Some go through ICU and some use CLDR directly.

   Yes. The PHP date() function has the format specifier I which yields             "1 if Daylight Saving Time, 0 otherwise",    and also "1 si está en horario de verano, 0 si no" in Spanish.    I am sure that nobody in Ireland would ever say "daylight-saving time"    when they meant winter time, so that the (new meaning of the) tzdb dst bit    is of no use for PHP.    Michael Deckers.

Stephen Colebourne wrote:

On Tue, 4 Jun 2019 at 17:49, Steve Summit <scs@eskimo.com> wrote:

...

So the analogous question for me is, "What are Java programs using isDaylightSavings for?

As discussed when the issue started a couple of years ago, the flag is used to select the name to display:

public String getDisplayName(boolean daylight, int style, Locale locale) https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/TimeZ...)

Thanks. So if I understand correctly, most/all of these calls to isDaylightSavings have nothing to do with Daylight Saving Time per se; they're just getting a more or less opaque token to use to fetch a CLDR string. And I think that sort of thing happens all over the place; it's basically one of the widespread assumptions which, for better or worse, lots of programmers make. Ideally we'd all be thinking about one or two arbitrarily-different kinds of time per year, but it's so easy to assume that one of them is called "dst", and that it happens in the summer, and that it's always an hour ahead of the other kind. So perhaps there's an opportunity here to help everyone out by trying to promulgate a vocabulary which emphasizes the arbitrariness. Instead of having a "dst" flag which is either true or false, what if we had a "kind" field which was a small integer from 0 to N-1? (Of course, internal to tz files, that's exactly what we have already.) I'm not sure what I mean by "what if we had", here, but if it's our contention that "standard versus DST" is an insufficient and/or obsolete distinction, that we'd like people to move beyond, there might be better ways of motivating that evolution than by setting isdst to 1 during the winter in Ireland and busting people's assumptions. (As an aside: Is it my imagination, or was there a time when C had an array of 1, 2, or maybe more than 2 time zone strings, and you used tm_isdst as an index into them, such that tm_isdst functioned as a sort of an opaque token even though it had "dst" in its name?)

On Jun 06 2019, Stephen Colebourne <scolebourne@joda.org> wrote:

In effect, tzdb used to provide (0, 1) for Ireland, now it provides (1, 0). This was an incompatible API change for the downstream projects of Java and CLDR.

And it was completely uncalled-for. Just because the Ireland law calls it standard time doesn't mean it can't be summer time as well. The EU summer-time directive defines summer time as "the period of the year during which clocks are put forward by 60 minutes compared with the rest of the year". Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."

Paul wrote:

It's not unreasonable to use standard time in POSIX to represent standard time in Ireland.

Indeed. But I don't think it's as obviously reasonable to be setting isdst for GMT in winter.

On 6/6/19 2:25 AM, Stephen Colebourne wrote:

...

Java has never provided accurate time zone names for historic times. I doubt any system does

Quite true, in large part because accurate time zone names do not exist for most historic times. That being said, there's no doubt that for Ireland in 1970, tzdb gets it right whereas Java gets it wrong.

I'm not sure that's an entirely fair challenge, though. Given that (as I understand it) Java and ICU/CLDR use tt_isdst to decide whether to display their equivalents of "GMT" or "IST", I don't think they *can* get the right answer near 1970, because tzdb changed its mind about the mapping at that point. Now, it's true, isdst might not be the best key to use for this sort of thing any more. Do we have recommendations for what projects like Java and ICU/CLDR ought to be keying off of, if not isdst? (I suppose tt_abbrind, or more likely the actual string it indexes, might be better.)

On 08/06/2019 09:47, Lester Caine wrote:

prior to 1970 that of cause should have been POST 1970

-- Lester Caine - G8HFL ----------------------------- Contact - https://lsces.uk/wiki/Contact L.S.Caine Electronic Services - https://lsces.uk Model Engineers Digital Workshop - https://medw.co.uk Rainbow Digital Media - https://rainbowdigitalmedia.co.uk

On 2019-06-08 06:59, Paul Goyette wrote:

Just FYI, a reasonably current version of Firefox reports a security issue with one of the web-sites noted in your .sig

This is what PM is designed for folks, please don't spam the list ;^> -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

On 08/06/2019 19:24, Paul Eggert wrote:

Lester Caine wrote:

...

And Ireland is a case where pre-1970 data is different to the set of data currently being used!

I didn't follow everything you said, but above statement may give other readers the misimpression that tzdb has two sets of data for the Republic of Ireland. It doesn't; it has only Europe/Dublin. As far as we know, Europe/Dublin is reasonably accurate for Dublin for the indefinite past, and is also accurate for the Republic of Ireland since it was founded. No other reliable "pre-1970 data" is available for the Republic of Ireland, as far as I know.

That IS part of the problem here ... I've got Ireland grouped with the rest of the truncated UK identifiers which may be a mistake on my part ... but I need to finish working through other more pressing ago before checking ... such as sorting out the .co.uk -> .uk fiasco :( -- Lester Caine - G8HFL ----------------------------- Contact - https://lsces.uk/wiki/Contact L.S.Caine Electronic Services - https://lsces.uk Model Engineers Digital Workshop - https://medw.co.uk Rainbow Digital Media - https://rainbowdigitalmedia.uk

Paul Eggert wrote:

Steve Summit wrote:

...

I'm not sure that's an entirely fair challenge, though. Given that (as I understand it) Java and ICU/CLDR use tt_isdst to decide whether to display their equivalents of "GMT" or "IST", I don't think they *can* get the right answer near 1970

Yes, Ireland in 1970 is an "unfair" challenge. That was its point. It was intended to illustrate the inadequacy of the current CLDR/Java model to represent real-world aspects of civil timekeeping.

...

tzdb changed its mind about the mapping at that point.

I'm not sure what you mean by "mapping",

Java/CLDR's mapping wants to be dst=0 -> GMT, dst=1 -> IST, which is of course incompatible with tzdb's current mapping. But if Java/CLDR were to somehow change to dst=0 -> IST, dst=1 -> GMT, to match tzdb's current mapping, it would then start getting the wrong answer before 1970, because for dates before 1970 (in 2019a, at least), tzdb uses dst=0 -> GMT, dst=1 -> IST.

CLDR+Java cannot handle Irish time correctly for past timestamps

We have not seen any demand for names before 1970, thus we haven't designed for more than two (regular) offsets per year for a given zone. It would not be hard, however, to add additional offsets, either for historic times, or if for some reason that becomes fashionable in the future. (Luckily, the tendency seems to be in the other direction, collapsing from 2 offsets into 1.) Mark On Sat, Jun 8, 2019 at 9:20 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

Steve Summit wrote:

...

I'm not sure that's an entirely fair challenge, though. Given that (as I understand it) Java and ICU/CLDR use tt_isdst to decide whether to display their equivalents of "GMT" or "IST", I don't think they *can* get the right answer near 1970

Yes, Ireland in 1970 is an "unfair" challenge. That was its point. It was intended to illustrate the inadequacy of the current CLDR/Java model to represent real-world aspects of civil timekeeping.

...

tzdb changed its mind about the mapping at that point.

I'm not sure what you mean by "mapping", but the 2018a change to Irish data was in response to a bug report about Irish time, a bug report that was investigated and found to be valid. Since tzdb can represent the Irish data as per Irish law and common use, the change was warranted from the tzdb point of view. And since Java's TZUpdater program currently rejects the changed data, I developed a 'rearguard' option to tzdb that lossfully converts the main-format tzdata into a rearguard format that pacifies TZUpdater.

However, even with the rearguard option (and even if we go back to circa 2017 code and data before this latest kerfuffle started), CLDR+Java cannot handle Irish time correctly for past timestamps due to what appear to be shortcomings in its model. This problem is not limited to Irish time; it also occurs for time in Los Angeles during World War II (see example below) and in several other areas, including Morocco right this minute and quite possibly in North America and Europe in the near future.

$ jshell | Welcome to JShell -- Version 12.0.1 | For an introduction type: /help intro

jshell> var jan1943 = java.time.Instant.ofEpochSecond(-852051600) jan1943 ==> 1943-01-01T07:00:00Z

jshell> var zone = java.time.ZoneId.of("America/Los_Angeles") zone ==> America/Los_Angeles

jshell> var dtf = java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss Z z (zzzz)") dtf ==> Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOf ... RT)' ''('ZoneText(FULL)')'

jshell> jan1943.atZone(zone).format(dtf) $4 ==> "1943-01-01 00:00:00 -0700 PDT (Pacific Daylight Time)"

jshell> $ TZ=America/Los_Angeles date -d@-852051600 +"%s %Y-%m-%d %H:%M:%S %z %Z" -852051600 1943-01-01 00:00:00 -0700 PWT

Near the end of the example above, Java says "PDT" where tzdb says "PWT", because Java can't handle PWT.

...

Now, it's true, isdst might not be the best key to use for this sort of thing any more. Do we have recommendations for what projects like Java and ICU/CLDR ought to be keying off of, if not isdst? (I suppose tt_abbrind, or more likely the actual string it indexes, might be better.)

I'm afraid they will need to solve this problem largely on their own, as one cannot look at tzdata and automatically derive strings like "Pacific War Time" or "Central Africa Ramadan Time": those strings are not in the data (not even in English), and there are no numeric equivalents either. The only partially-relevant info in tzdata consists of abbreviations like "IST" and "PDT" and unfortunately these abbreviations are well-documented to be ambiguous and historically inaccurate in some cases.

It should be possible for CLDR+Java to develop reasonably-reliable heuristics for guessing what string to use in some cases. For example, they could have a heuristic that "IST" means "India Standard Time" in Asia/Kolkata, "Israel Standard Time" in Asia/Gaza, Asia/Hebron and Asia/Jeruslaem, "Irish Summer Time" in Ireland before 1968-10-27, and "Irish Standard Time" in Ireland starting 1968-10-27. Similar heuristics could be used for other abbreviations, and if CLDR+Java tune the heuristics enough they'd be accurate. However, they'd have to do most of this work on their own. For example tzdb does not have an alphabetic abbreviation for the current time in Morocco (+00, a 1-hour negative DST where standard time is +01), so CLDR would have to invent an abbreviation there (presumably something like "Central Africa Ramadan Time" in English) and base its use on a heuristic like "when Africa/Casablanca is at +00 in the year 2019 or later, its time zone abbreviation is 'Central Africa Ramadan Time'".

This is because until October last year, +00 was standard time and +01 was daylight saving time in Morocco, but now it's the other way around.

I think we were talking about different things. I thought you were talking about the zones that had multiple offsets, so that's what I responded to. Sorry for the confusion. As for the "inverted" daylight savings that was introduced a couple of years ago (for Ireland, now Morocco), I don't consider that an architectural improvement, and there is no need to change CLDR to it *or* for it. For each <zone, locale> pair that has 2 offsets* within a year, all CLDR has and needs, is: (a) a name for the "less advanced" offset, and (b) a name for the "most advanced offset", (c) a name for "wall time" (generic). For historic reasons, (a) has the *internal* id of 'standard', and (b) has the *internal* id of 'daylight', but there is no requirement whatsoever that the translated names reflect the internal ids (many locales use the equivalent of "Winter Time" and "Summer Time"). {phone} On Sun, Jun 9, 2019, 18:57 Paul Eggert <eggert@cs.ucla.edu> wrote:

Mark Davis ☕️ wrote:

...

We have not seen any demand for names before 1970

The overall problem is not limited to names before 1970. It has occurred as recently as yesterday in Morocco. This is because until October last year, +00 was standard time and +01 was daylight saving time in Morocco, but now it's the other way around.

I presume that Morocco is low priority for Java+CLDR, as this change has not found its way into CLDR+Java yet. OpenJDK 12.0.1's data for Morocco has not been updated since Morocco last changed its rules, and OpenJDK mishandles UTC offsets for Moroccan timestamps as recent as yesterday, as shown in the example at the end of this email.

Java+CLDR can get away with a low priority for Morocco, as Morocco is a small part of the market. But it's likely that the issue can't continue to be ignored if North America and/or Europe change their zones as they are threatening to do soon. To some extent Morocco is a canary in the coal mine here, as it jumped the gun on the upcoming EU DST changes.

$ jshell | Welcome to JShell -- Version 12.0.1 | For an introduction type: /help intro

jshell> var zone = java.time.ZoneId.of("Africa/Casablanca") zone ==> Africa/Casablanca

jshell> var time = java.time.Instant.ofEpochSecond(1560011295) time ==> 2019-06-08T16:28:15Z

jshell> var dtf = java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss Z z (zzzz)") dtf ==> Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOf ... RT)' ''('ZoneText(FULL)')'

jshell> time.atZone(zone).format(dtf) $4 ==> "2019-06-08 17:28:15 +0100 WET (Western European Standard Time)"

jshell>

$ TZ=Africa/Casablanca LC_ALL=C date -d@1560011295 Sat Jun 8 16:28:15 +00 2019

The last line (from tzdb) is correct. In contrast, the Java-generated line "17:28:15 +0100 WET (Western European Standard Time)" is wrong by an hour and its UTC offset is miscalculated as +01, despite the fact that Java's name for the time zone is OK because WET is conventionally an alias for +00.

On Jun 07 2019, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 6/6/19 2:49 AM, Andreas Schwab wrote:

...

Just because the Ireland law calls it standard time doesn't mean it can't be summer time as well.

POSIX doesn't have the notion of summer time; it has only standard time and daylight saving time (also called "alternative time" in POSIX).

That's what the rest of the world calls summer time.

It's not unreasonable to use standard time in POSIX to represent standard time in Ireland.

No, it's unreasonable to break the world just because you can. GMT is standard time in Ireland during the winter, it doesn't make any sense to call that summer time. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."

On Jun 11, 2019, at 1:51 AM, Andreas Schwab <schwab@suse.de> wrote:

On Jun 07 2019, Paul Eggert <eggert@cs.ucla.edu> wrote:

...

On 6/6/19 2:49 AM, Andreas Schwab wrote:

...

Just because the Ireland law calls it standard time doesn't mean it can't be summer time as well.

POSIX doesn't have the notion of summer time; it has only standard time and daylight saving time (also called "alternative time" in POSIX).

That's what the rest of the world calls summer time.

That's what the part of the world that turns their clocks forward in spring and summer calls summer time. Some parts of the world don't do seasonal adjustment, although they may do adjustments for Ramadan. (Are there any countries that do both seasonal adjustment *and* Ramadan adjustment *in the same year*?) Other parts of the world, such as the country that occupies the bulk of an island slightly to the west of the island of Britain, appear to deem the time observed in spring and summer "standard time", and turn the clock backward in autumn and winter, calling that "winter time".

...

It's not unreasonable to use standard time in POSIX to represent standard time in Ireland.

No, it's unreasonable to break the world just because you can. GMT is standard time in Ireland during the winter,

Are you certain that the Irish government agrees with you here? Section 1 of the Standard Time Act, 1968: http://www.irishstatutebook.ie/eli/1968/act/23/section/1/enacted/en/html#sec... says (1) The time for general purposes in the State (to be known as standard time) shall be one hour in advance of Greenwich mean time throughout the year, and any reference in any enactment or any legal document (whether passed or made before or after the passing of this Act) to a specified point of time shall be construed accordingly unless it is otherwise expressly provided. (2) Any enactment expressed to operate or apply during a period of summer time shall be construed as operating or applying during the period prescribed as the period of summer time by the Summer Time Act, 1925 , and the Summer Time Order, 1926, that is to say, the period beginning at three o'clock in the morning of the day next following the third Saturday in the month of April or, if that day is Easter Day, beginning at that hour of the day next following the second Saturday in the month of April and, in either case, ending at that hour in the morning of the day next following the first Saturday in the month of October. and section 1 of the Standard Time (Amendment) Act, 1971 http://www.irishstatutebook.ie/eli/1971/act/17/enacted/en/print#sec1 says (1) (a) Notwithstanding section 1 (1) of the Standard Time Act, 1968, the time for general purposes in the State shall during a period of winter time be Greenwich mean time, and during such a period any reference in any enactment or any legal document (whether passed or made before or after the passing of this Act) to a specified point of time shall be construed accordingly unless it is otherwise expressly provided. (b) The period beginning at two o'clock Greenwich mean time in the morning of the 31st day of October, 1971, and ending at two o'clock Greenwich mean time in the morning of the 19th day of March, 1972, shall for the purposes of this Act be a period of winter time. (c) Subject to subsection (2) of this section, the period beginning at two o'clock Greenwich mean time in the morning of the Sunday following the fourth Saturday in October in any year after 1971 and ending either at two o'clock Greenwich mean time in the morning of the Sunday following the third Saturday in the month of March in the following year or, if the last-mentioned Sunday is Easter Day, at two o'clock Greenwich mean time in the morning of the Sunday following the second Saturday in the month of March in that year shall for the purposes of this Act be a period of winter time. The latter Act doesn't speak of winter time as being "standard time", just as "the time for general purposes in the State".

it doesn't make any sense to call that summer time.

No, it makes sense to call that winter time, as per the Standard Time (Amendment) Act, 1971. One API that the tzdb must support is the C localtime()/mktime() API. As specified in ISO/IEC 9899:2011(E): 7.27.1 Components of time The header <time.h> defines two macros, and declares several types and functions for manipulating time. Many functions deal with a calendar time that represents the current date (according to the Gregorian calendar) and time. Some functions deal with local time, which is the calendar time expressed for some specific time zone, and with Daylight Saving Time, which is a temporary change in the algorithm for determining local time. The local time zone and Daylight Saving Time are implementation-defined. ... The value of tm_isdst is positive if Daylight Saving Time is in effect, zero if Daylight Saving Time is not in effect, and negative if the information is not available. This is sufficiently vague (probably deliberately so) that: turning clocks forward from standard time in spring and summer could be called "Daylight Saving Time"; turning clocks backward from standard time in autumn and winter could be called "Daylight Saving Time"; adjusting clocks for Ramadan could be called "Daylight Saving Time"; etc., as they all are "temporary change[s] in the algorithm for determining local time". That does raise the question of whether "permanent Daylight Saving Time" would count. As has been noted earlier in this thread, the Single UNIX Specification 1) doesn't uniformly speak of "Daylight Saving Time" and 2) when it does, it's no clearer than the C standard about what it means. "Environment Variables": https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html says, in the section on TZ: The expanded format (for all TZ s whose value does not have a <colon> as the first character) is as follows: stdoffset[dst[offset][,start[/time],end[/time]]] Where: std and dst Indicate no less than three, nor more than {TZNAME_MAX}, bytes that are the designation for the standard (std) or the alternative (dst -such as Daylight Savings Time) timezone. Only std is required; if dst is missing, then the alternative time does not apply in this locale. speaks only of an "alternative timezone", giving "Daylight Savings[sic] Time" as an example of an alternative timezone, while the section on <time.h>: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/time.h.html says The value of tm_isdst shall be positive if Daylight Savings Time is in effect, 0 if Daylight Savings Time is not in effect, and negative if the information is not available. Paul Eggert has filed a bug against the Single UNIX Specification about this: http://austingroupbugs.net/view.php?id=1253 If that's resolved, that will, I hope, let us know whether "tm_isdst == 1" means, in locations where clocks are turned forward in spring/summer and backward in autumn/winter, "clocks are in the spring/summer setting" or "clocks are adjusted from standard time as defined by law". Now, it may be that, even if the resolution is that "tm_isdst == 1" means "clocks are in the spring/summer setting", so that the tzdb plus all C library implementations that use it must implement that behavior, there are *other* clients of the tzdb that need to provide an indication of whether the clock has been adjusted from standard time, in which case the tzdb would still need to treat Ireland differently, such that clocks are turned backward from standard time in the winter, even if the code that uses the tzdb to implement localtime()/mktime() needs to set tm_isdst to 1 during what is legally standard time.

On 6/11/19 1:14 PM, Guy Harris wrote:

Are there any countries that do both seasonal adjustment *and* Ramadan adjustment *in the same year*?

Morocco did that from 2013 through October 2018. That is, they sprang forward on March's last Sunday and fell back on October's last Sunday; and they also fell back before Ramadan and sprang forward afterward, so that there were four transitions per year. Egypt did something similar in 2010 and in 2014, as did Palestine's West Bank in 2011. tzdb can specify this succinctly, as any number of rules can apply during a year. CLDR+Java can handle this situation too (I don't know how, but I just now checked Morocco in 2017 and it worked OK). POSIX TZ strings cannot handle it, though, as POSIX provides for only two transitions per year. It's a bit of a mess to do it, partly because the Islamic calendar is astronomical and not algorithmic, and quite possibly we won't see anything similar in the near future with DST transitions falling out of favor in Europe anyway.

On 2019-06-11 19:50, Guy Harris wrote:

On Jun 11, 2019, at 1:14 PM, Guy Harris wrote:

...

On Jun 11, 2019, at 1:51 AM, Andreas Schwab wrote:

...

That's what the rest of the world calls summer time. That's what the part of the world that turns their clocks forward in spring and summer calls summer time. Or that's what the part of the world that turns their clocks forward in spring and summer, with the exception of the US and (English-speaking?) Canada, calls "summer time".

Indeed; in French speaking North America, it's l'heure normale/avancée du Pacifique/des Rocheuses/du Centre/de l'Est/de l'Atlantique/de Terre-Neuve (HN/AP/R/C/E/A/T). I imagine there are similar terms in Spanish for Mexico etc. and Portuguese for Brazil etc.

In any case, the point is that not all locales that do seasonal adjustment of time have "standard time" during the autumn and winter and "summer time" during the spring and summer; they might have "standard time" during the spring and summer, as the Republic of Ireland does.

...

One API that the tzdb must support is the C localtime()/mktime() API. As specified in ISO/IEC 9899:2011(E):

Neither ISO/IEC 9899:2018(E) nor the most recent draft I've seen of ISO/IEC 9899:202x(E): http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2346.pdf say anything significantly different - they continue to speak of "Daylight Saving Time", but leave it unspecified what it means, so as not to explicitly *disallow* it meaning something other than "temporarily turning clocks forward for spring and summer".

Paul, would somebody have to be a member of the C standards committee, or join their mailing list, in order to say "maybe the name 'Daylight Saving Time'" should be removed, in favor of something that doesn't imply a seasonal change where the algorithm is change to turn clocks forward, given that the algorithm might be changed for reasons that aren't what's conventionally thought of as seasonal (e.g., Ramadan) or might be changed to turn clocks *backward* in the winter"?

There are certainly former, and possibly current, members of the standards committees subscribed to this list, and may be participating in this thread (hands up list subscribers who have not chimed in on this thread ;^> ). Would also be an opportunity to extend TZ: TZ=std offset dst [offset],start[/time],end[/time][,start[/time],end[/time]]... where additional transition pairs could be added, better accommodating reality. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

On 06/06/2019 03:33, Steve Summit wrote:

Stephen Colebourne wrote: (As an aside: Is it my imagination, or was there a time when C had an array of 1, 2, or maybe more than 2 time zone strings, and you used tm_isdst as an index into them, such that tm_isdst functioned as a sort of an opaque token even though it had "dst" in its name?)

It's not your imagination. It still exists in the POSIX specs as the tzname array, but it is either 2 elements long or unspecified length, depending where you look: (1) http://pubs.opengroup.org/onlinepubs/9699919799/functions/tzname.html extern char *tzname[2]; The tzset() function shall set the external variable tzname as follows: tzname[0] = "std"; tzname[1] = "dst"; where std and dst are as described in XBD Environment Variables. (2) http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/time.h.html extern char *tzname[]; The value of tm_isdst shall be positive if Daylight Savings Time is in effect, 0 if Daylight Savings Time is not in effect, and negative if the information is not available. (From the above, it seems to me that tzname[0] applies when tm_isdst==0, and tzname[1] applies when tm_isdst>0.) -- -=( Ian Abbott <abbotti@mev.co.uk> || Web: www.mev.co.uk )=- -=( MEV Ltd. is a company registered in England & Wales. )=- -=( Registered number: 02862268. Registered address: )=- -=( 15 West Park Road, Bramhall, STOCKPORT, SK7 3JZ, UK. )=-

On Jun 5, 2019, at 7:33 PM, Steve Summit <scs@eskimo.com> wrote:

(As an aside: Is it my imagination, or was there a time when C had an array of 1, 2, or maybe more than 2 time zone strings, and you used tm_isdst as an index into them, such that tm_isdst functioned as a sort of an opaque token even though it had "dst" in its name?)

OK, time for another source dive! V7: https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/libc/gen/timezone.c had a timezone() function, which took an offset-in-minutes-from-UTC-with-west-of-UTC-being-positive argument and a "is this DST" argument, and looked up the zone in a table of North American time zone abbreviation names and returned either the result or, if it wasn't found, a string of the form "GMT{+,-}H:MM, with + being east of UTC and - being west of UTC. 4.2BSD: https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/lib/libc/gen/ti... offered the same API, added some European and Australian time zones to the table, and allowed a TZNAME environment variable, of the form "std,dst", to be used to set the zone name. Dating back at least as far as System III: https://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/lib/libc/pdp11/... the System Roman Numeral list of UNIXes had the tzname[] array, with 2 elements, the first of which was for "standard" time and the second of which was for "daylight saving" time. The System III man page for ctime(): https://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man3/ctime.... doesn't explicitly say that you use tm_isdst as an index into tzname[], or that the first element is for "standard" time and the second is for "daylight saving" time, but char *tzname[2] = {"EST", "EDT"}; somewhat suggests the latter. The System Roman Numeral mechanism for getting the time zone name is what POSIX adopted.

scs@eskimo.com (Steve Summit) writes:

(As an aside: Is it my imagination, or was there a time when C had an array of 1, 2, or maybe more than 2 time zone strings, and you used tm_isdst as an index into them, such that tm_isdst functioned as a sort of an opaque token even though it had "dst" in its name?)

tzname, and it's still a supported API (documented in tzset(3) on Linux). INN uses it when generating Date headers if it's available and tm.tm_zone is not (in a bit of portability that is now probably only applicable to museum exhibitions): /* Now, get a pointer to the time zone abbreviation, and if there is enough room in the buffer, add it to the end of the date string as a comment. */ if (!local) { tz_name = "UTC"; } else { #if HAVE_STRUCT_TM_TM_ZONE tz_name = tm.tm_zone; #elif HAVE_TZNAME tz_name = tzname[(tm.tm_isdst > 0) ? 1 : 0]; #else tz_name = NULL; #endif } tzset(3): The tzset() function initializes the tzname variable from the TZ environment variable. This function is automatically called by the other time conversion functions that depend on the timezone. In a System-V-like environment, it will also set the variables timezone (seconds West of UTC) and daylight (to 0 if this timezone does not have any daylight saving time rules, or to nonzero if there is a time, past, present or future when daylight saving time applies). -- Russ Allbery (eagle@eyrie.org) <http://www.eyrie.org/~eagle/>

* Paul Eggert:

On 5/30/19 2:14 AM, Stephen Colebourne wrote:

...

It is a compatibility requirement of the main Java libraries (including icu4j IIUC) that DST is always positive.

Where is this documented? And what do the main Java libraries do in an environment where the DST offset is negative? For example, if I run the following shell command on a POSIX-compatible host:

TZ='IST-1GMT0,M10.5.0,M3.5.0/1' java [arguments]

can you give an example of what goes wrong and why? This command gives Java the rules that Ireland has used since 1996, with Irish Standard Time (1 hour ahead of GMT) used in the summer and GMT (a "negative daylight saving time") in the winter.

Is this really the right example? I would have expected something negative after the slash. Thanks, Florian

On 5/29/19 7:34 AM, Stephen Colebourne wrote:

A hack was added to Joda-Time that reverses negative SAVE values which works so long as a rule set does not mix positive and negative save values. This is pretty awful as an approach because it is unreliable.

Does this hack suffice for Europe/Dublin? Europe/Dublin uses several rule sets, of which only one (the "Eire" rule set) uses negative save values, and the "Eire" rule set does not have positive save values. This makes it sound like Europe/Dublin should be OK for Joda-Time. I assume from your description that the hack does not suffice for Africa/Casablanca and Africa/El_Aaiun, as they use the Morocco rule set, and the Morocco rule set uses both positive save values (before 2019) and negative save values (for 2019 onwards). Would it help to split the Morocco rule set into two rule sets, say "OldMorocco" and "Morocco"? The former rule set would be for timestamps before 2019 and the latter would be for timestamps starting 2019. Although this change would complicate the 'africa' file a bit (as we'd have to add two zone continuation lines), the resulting TZif binary files should be identical and it sounds like it might be worth making such a change if it's the only thing preventing Joda-Time from working.

Proposal: Add an additional file with legacy versions of the rules that cause problems to downstream parsers.

The legacy file would be another file to maintain by hand. I prefer something more along the lines of the current approach, where the legacy files are generated automatically via "make rearguard_tarballs". Just today I received a report of success for someone using that approach with TZUpdater.

On Wed, 29 May 2019 at 18:05, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 5/29/19 7:34 AM, Stephen Colebourne wrote:

...

A hack was added to Joda-Time that reverses negative SAVE values which works so long as a rule set does not mix positive and negative save values. This is pretty awful as an approach because it is unreliable.

Does this hack suffice for Europe/Dublin? Europe/Dublin uses several rule sets, of which only one (the "Eire" rule set) uses negative save values, and the "Eire" rule set does not have positive save values. This makes it sound like Europe/Dublin should be OK for Joda-Time.

The hack added for Joda-Time is such that the current data format can be parsed correctly for all zones. It also handles the "Apr Sun>=25" type rules recently added. It is perfectly possible to add the same hack to ThreeTen-Backport and TZUpdater, I'm simply indicating that it *is* a hack, and the problem is not going to go away.

Would it help to split the Morocco rule set into two rule sets, say "OldMorocco" and "Morocco"?

No, that wouldn't really help. (I thought about such solutions, but both the Rule and Zone lines are affected by negative SAVE values.) And the parser can handle the Morocco rules. IIRC, the problem would be if there is a Zone line where rules apply with both positive and negative SAVE values. This doesn't happen with Morocco because there is a zone line starting at the end of 2018.

...

Proposal: Add an additional file with legacy versions of the rules that cause problems to downstream parsers. The legacy file would be another file to maintain by hand. I prefer something more along the lines of the current approach, where the legacy files are generated automatically via "make rearguard_tarballs".

In essence the choice for downstream parsers is a) a hack to try and reverse engineer the desired data b) rely on rearguard forever (which is risky as tzdb isn't promising forever) I'm simply suggesting an alternative proposal that would IMO be better than these two options. Another alternative would be to express both pieces of information in the data somehow - whether the period is summer or winter and the legal status of whether that is referred to as standard or daylight. eg. Eire would have values of DST-Winter and Std-Summer, whereas most places would have Std-Winter and DST-Summer. That way a parser could choose to read either the DST/Std value or the Summer/Winter value and proceed accordingly. Finally I'll note that *both* views of the data are sensible and reasonable: - offset-focus: base/standard time in winter, advanced/daylight time in summer (Java's choice and tzdb's old choice) - legal-focus: follow government law as to the meaning of standard/daylight (tzdb's new choice) Most Java libraries aren't going to change because doing so would impact compatibility in APIs.The real problem here is how tricky it is to reverse engineer the old data from the new data. Stephen

On Thu, 30 May 2019 at 13:04, Robert Elz <kre@munnari.oz.au> wrote:

What are you planning to do when some part of the US (say Illinois, or something else on Central time) decides to set their zone backwards by an hour for a month in the middle of winter

I imagine the code would just move the base offset back for that period. The key point is that both forms can express the same outcome in terms of when the clocks actually change. The practical differences are what the API says is the base offset and what the API says is daylight time. Where this debate goes wrong is emails that assert tzdb's choice to follow the legal definition is the only possible choice. It isn't. The other choice of exposing the smaller/larger offset is perfectly valid too. Just different. Stephen

the most common (standard) time

I predict that significant libraries and implementations will be on rearguard forever; if that is not maintained by the TZDB, the it would be cloned and maintained externally with rearguard modifications. The cost of doing anything else would be prohibitive. And there was really no reason whatsoever for changing policies. If we have a timezone with two separate offsets, one an hour ahead of the other, then the TZDB can chose to represent them as <0, 1> or as <-1, 0>. It is needless to have both in the database. For consistency one can simply decide to have them all be positive, which was the longstanding policy of this group until recently. The *name* of the time variants might use "standard" or "winter" or "lětni čas" or "夏時間", whatever is appropriate to the user's locale. There is no requirement that any two locales be aligned in what they call the "hour ahead" variant or the "hour behind" variant. The names attached to the variant by a given locale is really outside of the scope of the TZDB. Changing the policy to allow <-1, 0> has no particular practical benefit, and has just caused compatibility difficulties. You can't just wave a magic wand and expect longstanding APIs to return different values without causing problems for users. Mark And BTW, in most timezones around the world, a majority of the days in the year are on "daylight" time, so it is the ‘most common one’. On Thu, May 30, 2019 at 2:04 PM Robert Elz <kre@munnari.oz.au> wrote:

Date: Thu, 30 May 2019 11:54:35 +0100 From: Stephen Colebourne <scolebourne@joda.org> Message-ID: < CACzrW9CXTRW_G07E2Y-b2Q4+Ew5jt8LcFSrzR-YSg0gT-0SCTw@mail.gmail.com>

| Finally I'll note that *both* views of the data are sensible and reasonable: | - offset-focus: base/standard time in winter, advanced/daylight time | in summer (Java's choice and tzdb's old choice)

tzdb never made such a choice - it simply didn't have any data that happened to have the most common (standard) time advanced ahead of the less common offset.

What are you planning to do when some part of the US (say Illinois, or something else on Central time) decides to set their zone backwards by an hour for a month in the middle of winter - perhaps the winter olympics come to Denver or something, and they decide that being in the same timezone for that period has more economic and social wins than being an hour off the event times.

Certainly that might be unlikely, but it certainly is not impossible, and you're not likely to get much more than a year's warning if it were to happen (at best). Wouldn't planning for this kind of thing now be much more rational than just sticking your heads in the sand with a "we didn't consider that and it is too late now" attitude.

If the old APIs need to be deprecated, and a whole new set invented, then so be it - do that - the old ones can be supported, as best they can be given they are based upon false assumptions, for a long time, but averything should be encouraged to convert to something rational, with no in-built assumptions about what is possible or rational wrt local time.

| - legal-focus: follow government law as to the meaning of | standard/daylight (tzdb's new choice)

First, standard time is the time that applies *now* - whenever now is. If it has a name, distinct from the name that applies to the time at some other time of the year, that's fine (but almost irrelevant to anything).

| Most Java libraries aren't going to change because doing so would | impact compatibility in APIs.The real problem here is how tricky it is | to reverse engineer the old data from the new data.

I suspect that the real problem is that the current APIs are simply inadequate to describe real world time behaviour. Any assumptions made, anywhere, about almost anything, will almost guarantee that.

kre

On 2019-05-30 08:26, Brandon Smith wrote:

On Thu, May 30, 2019 at 8:51 AM Mark Davis ☕️ wrote:

...

On Thu, May 30, 2019 at 2:04 PM Robert Elz wrote:

...

On Thu, 30 May 2019 11:54:35 +0100, Stephen Colebourne wrote: the most common (standard) time I predict that significant libraries and implementations will be on rearguard forever; if that is not maintained by the TZDB, the it would be cloned and maintained externally with rearguard modifications. The cost of doing anything else would be prohibitive.

And there was really no reason whatsoever for changing policies. If we have a timezone with two separate offsets, one an hour ahead of the other, then the TZDB can chose to represent them as <0, 1> or as <-1, 0>. It is needless to have both in the database. For consistency one can simply decide to have them all be positive, which was the longstanding policy of this group until recently.

The *name* of the time variants might use "standard" or "winter" or "lětni čas" or "夏時間", whatever is appropriate to the user's locale. There is no requirement that any two locales be aligned in what they call the "hour ahead" variant or the "hour behind" variant. The names attached to the variant by a given locale is really outside of the scope of the TZDB.

Changing the policy to allow <-1, 0> has no particular practical benefit, and has just caused compatibility difficulties. You can't just wave a magic wand and expect longstanding APIs to return different values without causing problems for users.

And BTW, in most timezones around the world, a majority of the days in the year are on "daylight" time, so it is the ‘most common one’.

...

...

| Finally I'll note that *both* views of the data are sensible and reasonable: | - offset-focus: base/standard time in winter, advanced/daylight time | in summer (Java's choice and tzdb's old choice)

tzdb never made such a choice - it simply didn't have any data that happened to have the most common (standard) time advanced ahead of the less common offset.

What are you planning to do when some part of the US (say Illinois, or something else on Central time) decides to set their zone backwards by an hour for a month in the middle of winter - perhaps the winter olympics come to Denver or something, and they decide that being in the same timezone for that period has more economic and social wins than being an hour off the event times.

Certainly that might be unlikely, but it certainly is not impossible, and you're not likely to get much more than a year's warning if it were to happen (at best). Wouldn't planning for this kind of thing now be much more rational than just sticking your heads in the sand with a "we didn't consider that and it is too late now" attitude.

If the old APIs need to be deprecated, and a whole new set invented, then so be it - do that - the old ones can be supported, as best they can be given they are based upon false assumptions, for a long time, but averything should be encouraged to convert to something rational, with no in-built assumptions about what is possible or rational wrt local time.

| - legal-focus: follow government law as to the meaning of | standard/daylight (tzdb's new choice)

First, standard time is the time that applies *now* - whenever now is. If it has a name, distinct from the name that applies to the time at some other time of the year, that's fine (but almost irrelevant to anything).

| Most Java libraries aren't going to change because doing so would | impact compatibility in APIs.The real problem here is how tricky it is | to reverse engineer the old data from the new data.

I suspect that the real problem is that the current APIs are simply inadequate to describe real world time behaviour. Any assumptions made, anywhere, about almost anything, will almost guarantee that.

...

...

I predict that significant libraries and implementations will be on rearguard forever; if that is not maintained by the TZDB, the it would be cloned and maintained externally with rearguard modifications. The cost of doing anything else would be prohibitive.

My company has not been able to adapt to the new negative offsets as well and has been relying upon rearguard up to this point, given it breaks our application behavior (note C++ in our case).

The problem as I see it is around the definition of 'what is daylight saving time', and the implementation of negative DST offsets changes that definition as far as I can tell.  Wikipedia describes it as such "Daylight saving time (DST), also daylight savings time or daylight time (United States) and summer time (United Kingdom, European Union, and others), is the practice of advancing clocks during summer months so that evening daylight lasts longer, while sacrificing normal sunrise times."  [1] timeanddate.com <http://timeanddate.com> describes it as "Daylight Saving Time (DST) is the practice of setting the clocks forward one hour from standard time during the summer months, and back again in the fall, in order to make better use of natural daylight." [2].  The commonality there being "advance clocks".

Perhaps not everyone agrees that those are reliable sources.  But I think the reality is that much if not most of the software industry views them as such.  So as others seem to be suggesting to me, libraries and applications have forever been written based on the definition and understand that clocks move forward with DST transitions.  I know it has been said that this is a 'bad assumption', but the reality as I see it is that generally speaking all software has been written based upon what developers knew to be a 'definition' and not an 'assumption' at all.

So not only are we talking about incompatibilities with all the software/parsers involved with the libraries themselves (e.g. joda time), we are also talking about all software utilizing those libraries who have to account for various use cases that rely upon these "definitions".

As a side note, my company is highly impacted by the Brazil changes and I know it was alluded to that TZDB could perhaps use the next release to force the hand of the negative dst offset discussion (i.e. remove rearguard).  I would strongly request that we not do that given the number of requests by others for an updated Brazil release as well as the amount of compatibility concerns mentioned here with negative offsets.

Interesting points of view about your own problems but kind of missing the point about the *volunteer* maintainer's scope of maintaining the data, the tools, and the API to use the data, as intended, in their own *personal free time*. None of these complaints are from users of the data and code base, but third party users, including some commercial projects and companies, who appear to have caused problems by allowing access to low level details of the implementation. A lesson that should have been learned and remembered from twenty years ago, is that date/time/duration data must be opaque to apps and programmers, and they must use a high level API on it, otherwise one is no better off than using the low level C API. AFAICT tz db has always set up its rules to match whatever each government decrees: if that happens to be in a non Gregorian calendar - Hebrew or Muslim - the maintainers have used some calendar conversions to generate those dates for future years, as best they can predict, as with anything politically based. Now for their usual political reasons some governments are being different and instead of using standard and advanced summer times, they are using winter and advanced standard/summer times, negative standard offsets, 24-25 hour times, etc. Some people, possibly some managers in commercial orgs, are now complaining their apps won't work with the canonical data, because their organizations designed or implemented their applications with limitations, because they *chose not to use the canonical data, tools, or API*, or take the time to understand enough about time, politics, design, and programming, despite the myriads of articles and posts about such mistaken beliefs: Falsehoods "Programmers" Believe About <TOPIC>; see: https://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-abo... and all the related links to more falsehoods about time and other real life topics. One of those mistaken beliefs is that fixing these problems takes a lot of effort and costs a lot of money: I've heard people I work with and for make these kinds of statements without any basis except a "chicken little" attitude, and corrected them. Handling these kinds of changes is just part of the job, and if designs or code have to change, they were deficient or defective in the first place, so maybe have your best programmers look at your code base, count how many lines of code are affected by these fixes, how many minutes it will take to make the changes, and how much money that costs. If the answer is too much, you have a *much* bigger problem! The maintainer spent a few hours of his own personal free time coming up with and implementing rearguard format and making all the data changes required to accommodate legacy apps. There was no requirement to do so but he saw a real issue, mistakenly expecting those others to do their jobs and upgrade their apps to remove their limitations. It should only take others a few hours to make similar accomodations to upgrade their apps and remove their limitations. Those paid to support and maintain those legacy apps by their orgs now have an opportunity to do their jobs, a year later, by *choosing* getting those apps updated so they can continue to take advantage of the resources provided. Alternately, they can *choose* to pay their staff to maintain a version of the data compatible with their app, until another political change comes along to break it and force a change. The primary goal of the project should be to maintain the data, tools, and API for using the data. Third party, especially commercial, software should not be a major consideration, especially if they choose not to use the supported API, as they should have the resources to deal with any issues arising from *their choices*. Providing time for a transition or an interim data format during transition is being considerate over and above the required scope. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

Bradley White said:

...

The commonality there being "advance clocks". No one is questioning that the civil time advances in the summer. The question is only which side of the transition is labelled "standard".

Indeed. And in Ireland it's the summer one that's labelled "standard". -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

On May 30, 2019, at 12:13 PM, Clive D.W. Feather <clive@davros.org> wrote:

Brandon Smith said:

...

The problem as I see it is around the definition of 'what is daylight saving time', and the implementation of negative DST offsets changes that definition as far as I can tell. Wikipedia describes it as such [...] The commonality there being "advance clocks".

Perhaps not everyone agrees that those are reliable sources.

What makes you think they are?

Even *Wikipedia* doesn't consider Wikipedia a reliable source: https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources#User-generated_cont...

On May 30, 2019, at 3:56 PM, Alan Mintz <alan.mintz@gmail.com> wrote:

Well, sort of. Wikipedia pages are not, prima facie, reliable sources to be used for other wikipedia articles as a matter of policy – one is welcome to use the sources cited in such an article and evaluate them for use in citing an article, instead. They can be reliable, though, if one verifies the sources given.

I.e., use the sources in the Wikipedia article, not the Wikipedia article itself, as sources.

John Wilcock said:

FWIW, I've just updated https://en.wikipedia.org/wiki/Daylight_saving_time#Procedure to add a short note about negative DST in Ireland. If anyone here has any comments on my wording (or, ideally, any secondary sources that could be used use rather than citing the Irish Standard Time Act directly),

What's wrong with citing the primary source? -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

On Jun 4, 2019, at 12:24 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:

Clive D.W. Feather wrote:

...

What's wrong with citing the primary source?

Wikipedia policy prefers secondary sources. See:

https://en.wikipedia.org/wiki/WP:PSTS#Primary,_secondary_and_tertiary_source...

And sometimes that policy is just silly. Primary sources are original materials that are close to an event, and are often accounts written by people who are directly involved. They offer an insider's view of an event, a period of history, a work of art, a political decision, and so on. Primary sources may or may not be independent sources. An account of a traffic incident written by a witness is a primary source of information about the event; similarly, a scientific paper documenting a new experiment conducted by the author is a primary source on the outcome of that experiment. Historical documents such as diaries are primary sources. makes sense in cases where the notion of "an insider's view of an event" as a possibly-biased view makes sense, e.g. just because Apple claims that iOS 13 will launch applications twice as fast, that doesn't mean the claim should be taken as fact without independent non-Apple verification. On the other hand, a link to a news story reporting that law XXX does YYY isn't necessarily any better than a link to the section of law XXX where it explicitly says it does YYY. If "law XXX does YYY" is an interpretation, maybe that would make sense, but a link to Joe Bob reporting that the Standard Time (Amendment) Act, 1971, says that Subject to subsection (2) of this section, the period beginning at two o'clock Greenwich mean time in the morning of the Sunday following the fourth Saturday in October in any year after 1971 and ending either at two o'clock Greenwich mean time in the morning of the Sunday following the third Saturday in the month of March in the following year or, if the last-mentioned Sunday is Easter Day, at two o'clock Greenwich mean time in the morning of the Sunday following the second Saturday in the month of March in that year shall for the purposes of this Act be a period of winter time is not an improvement over a link to section 1 of the Standard Time (Amendment) Act, 1971.

On Jun 4, 2019, at 1:25 AM, John Wilcock <john@tradoc.fr> wrote:

Le 04/06/2019 à 09:24, Paul Eggert a écrit :

...

Clive D.W. Feather wrote:

...

What's wrong with citing the primary source? Wikipedia policy prefers secondary sources. See: https://en.wikipedia.org/wiki/WP:PSTS#Primary,_secondary_and_tertiary_source...

Also, the primary source only backs the claim that Ireland uses negative DST, but doesn't back the claim that most countries use positive DST. Are there actually any examples other than Ireland?

I.e., examples of negative DST other than Ireland? There are plenty of examples of positive DST.

John Wilcock said:

Firstly, any suggestions for a source to cite for the claim that most countries use positive DST?

Unlikely to be any. https://en.wikipedia.org/wiki/Daylight_saving_time_by_country lists 69 countries that observe clock changes and 175 that don't. You may argue with their definition of "country" (I wouldn't consider Guernsey a separate country) but it's clear that less than a third do. If the EU's plans go ahead then at least 39 of those 69 will stop observing it. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

On 6/4/19 4:39 AM, John Wilcock wrote:

Firstly, any suggestions for a source to cite for the claim that most countries use positive DST?

That isn't quite correct, as most countries don't use DST at all. A more-accurate statement is that most uses of DST employ positive DST. You can cite this email as a source, if you like. The mailing-list archive is at: https://mm.icann.org/pipermail/tz/

are there any examples of negative DST other than Ireland?

In addition to Ireland (starting 1971), tzdb also reports Morocco (starting 2019), Namibia (1994-2017), and Czechoslovakia (1946-1947).

*Clive Wrote:* *>> What makes you think they are? If I had tried to cite either of thoseduring my latest degree course, I'd have been penalized significantly. * Regarding timeanddate.com of those specifically, I thought that was founded by Steffen Thorsen who I thought was a contributor and resource to this database via past announcements. *Clive Wrote:>> But it *is* following the definition of "daylight time". It's just thatthat doesn't match what you thought it was.* So what is the correct definition of DST or is there one? If I review the sources linked from the IANA website itself [1], I again see the idea that DST is the process of advancing clocks forward [2] (page #2 and following). Again, referencing timeanddate.com for Ireland [3] it shows a +1 hour (DST Start) in the Summer (i.e. GMT->IST) and -1 hour (DST End) in the winter (i.e. IST->GMT). This seems to align with the idea that DST on is the period in which clocks were advanced forward. I thought part of the broader issue and discussion here was that the tm_isdst flag is now true in winter for Ireland indicating they observe DST on in the winter and not the summer per this prior tz thread [4]. That was a lengthy discussion and I admit I only recently started following these announcements to try and digest all this information, so if there was a conclusion to that I missed please do correct me. Perhaps I've deviated from this thread's primary point here. Our specific issue is not with the parsing itself as we can certainly parse the data. Our initial known issue is with the behavioral implications to our applications that came when inverting the tm_isdst flag for Ireland. This change seems to contradict the definition of DST that we have thought to be true and as far as I can tell is the definition supported everywhere outside of this discussion. An example of such an issue is the fact that we now have a situation in which the transition from DST off to DST on results in an ambiguous local time period for Ireland. Based on the above definitions and understanding of what is DST, this has led to an understanding that the ambiguous period is therefore always the transition from DST on to DST off. Before the implementation of negative DST offsets, this was true for Ireland. After the introduction of negative DST offsets, this is no longer true. I imagine there are other implications to our applications, I haven't yet got a grasp on what those are. My sense from this and other threads was that we aren't alone in this problem and it seems to me to go hand and hand with the 'Standard' discussion. To be clear, I'm not claiming to be an expert on any of this by any means here. I fully recognize there are things we as a company are doing wrong. I'm merely trying to grasp and understand if this is one of them and admittedly having a hard time given the conflicting viewpoints in here and information found elsewhere online. My original comments were primary to: 1) I get the sense my company is not unique in our problems here given these discussions and that lack of clarity I'm finding online. So as others stated here, I think the vast majority of software is likely in the same place here and simply offering support to that feedback. 2) Expressing our need/desire to further delay the complete removal of rearguard as it relates to: *"The TZ Coordinator is empowered to decide, as the designated expert, appropriate changes, but SHOULD take into account views expressed on the mailing list."* 3) I've been recently tasked with trying to figure this all out and the impact to my company, so indeed this is precisely why I came to this group/discussion to *"... **take the time to understand enough about time, politics, design, and programming, despite the myriads of articles and posts about such mistaken beliefs".* [1] https://data.iana.org/time-zones/tz-link.html [2] http://www.webexhibits.org/daylightsaving/b2.html [3] https://www.timeanddate.com/time/zone/ireland/dublin [4] https://mm.icann.org/pipermail/tz/2018-January/025836.html

On May 31, 2019, at 9:49 AM, Guy Harris <guy@alum.mit.edu> wrote:

On May 31, 2019, at 6:52 AM, Brandon Smith <smith.b78987@gmail.com> wrote:

...

So what is the correct definition of DST or is there one? If I review the sources linked from the IANA website itself [1], I again see the idea that DST is the process of advancing clocks forward [2] (page #2 and following). Again, referencing timeanddate.com for Ireland [3] it shows a +1 hour (DST Start) in the Summer (i.e. GMT->IST) and -1 hour (DST End) in the winter (i.e. IST->GMT). This seems to align with the idea that DST on is the period in which clocks were advanced forward. I thought part of the broader issue and discussion here was that the tm_isdst flag is now true in winter for Ireland indicating they observe DST on in the winter and not the summer per this prior tz thread [4].

The definition of "daylight savings time" is up to dictionaries, convention, etc.. It generally appears to mean "the time used during some part of spring and summer, in order to give more hours of daylight in the evening and fewer hours of daylight in the morning".

"tm_isdst" was originally a UNIXism, so if there's anything that defines what *it* means, it's currently the Single UNIX Specification. What the Single UNIX Specification page for <time.h>:

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/time.h.html

says is

The value of tm_isdst shall be positive if Daylight Savings Time is in effect, 0 if Daylight Savings Time is not in effect, and negative if the information is not available.

On the other hand, the Environment Variables section of the Single UNIX Specification: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html says, of the TZ environment variable: The expanded format (for all TZ s whose value does not have a <colon> as the first character) is as follows: stdoffset[dst[offset][,start[/time],end[/time]]] Where: std and dst Indicate no less than three, nor more than {TZNAME_MAX}, bytes that are the designation for the standard (std) or the alternative (dst -such as Daylight Savings Time) timezone. Only std is required; if dst is missing, then the alternative time does not apply in this locale. suggesting that there's "standard time" and "alternative time"; if "standard time" is "standard time as specified by law", then, in the Republic of Ireland, "alternative time" is what's observed in the autumn and winter, when the clocks are turned backwards from standard time. Perhaps the Open Group should be asked to clarify this.

Brandon Smith wrote:

we now have a situation in which the transition from DST off to DST on results in an ambiguous local time period for Ireland.

For what it's worth, this particular situation is not new in tzdata. For example, Europe/Kiev has a transition from +03 (without DST) to +02 (with DST) on 1941-09-19 at 24:00, which means that timestamps that day between 23:00 and 24:00 are ambiguous in Kiev. This transition has been present since the release of tzdata 1999j two decades ago. I'm sure there are other examples.

On 2019-06-01 01:43, Paul Eggert wrote:

Brandon Smith wrote:

...

we now have a situation in which the transition from DST off to DST on results in an ambiguous local time period for Ireland. For what it's worth, this particular situation is not new in tzdata. For example, Europe/Kiev has a transition from +03 (without DST) to +02 (with DST) on 1941-09-19 at 24:00, which means that timestamps that day between 23:00 and 24:00 are ambiguous in Kiev. This transition has been present since the release of tzdata 1999j two decades ago. I'm sure there are other examples.

Every time zone that has seasonal offset changes has periods per year where local time does not exist (spring forward) and ambiguous duplicated local time periods (fall back). Or am I missing something different? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

On 2019-06-01 10:53, Paul Eggert wrote:

Brian Inglis wrote:

...

Every time zone that has seasonal offset changes has periods per year where local time does not exist (spring forward) and ambiguous duplicated local time periods (fall back). Or am I missing something different? Brandon Smith said that his software had trouble with transitions where (1) tm_isdst==0 before and tm_isdst>0 after the transition, and (2) the clock moves backward so there are ambiguous timestamps before the transition. The implication was that this was a new situation, and so there's some justification in modifying the data to avoid it. My point was that it's not a new situation, in that other such transitions have been in the database for decades. This is related to Robert Elz's point that mktime is not up to handling arbitrary ambiguous timestamps. Although mktime can handle ambiguous timestamps where tm_isdst changes, it can't handle ambiguous timestamps where tm_isdst is the same before and after the transition. By "can't handle" I mean that mktime's caller has no control over which of the ambiguous timestamps is returned by mktime, and the caller isn't even informed whether the timestamp is ambiguous.

Thanks for the clarification, and I could see where, for ambiguous summer time periods, the state of tm_isdst could be used as input to hint to mktime which output was desired. Where tm_isdst is also ambiguous, that is no help for input or output, and from other discussions, an earlier/later input selection (and output indicator) would be desirable. [North American electrical power generation, load, transmission, and distribution systems operate with local times to accommodate local time peak/non-peak/off-peak, holiday, and weekend power schedules and tariffs (which are remarkably consistent across local time zones), and the systems have to be able to handle each local time zone's ambiguous hours (where used), with the second instance usually labelled something like e.g. 2*, and the consequent 25 and 23 hour days. APIs that don't allow for and handle such local time variations have to be considered defective!] -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

Brandon Smith wrote:

I would say we are at a minimum of a year out still to get this fully adapted.

You should be OK then, as there are no plans to remove "make rearguard_tarballs" and I doubt whether it'll go away that quickly. Still, this shouldn't be an excuse for delaying for a year....

...

...

...

For what it's worth, this particular situation is not new in tzdata. For example, Europe/Kiev has a transition from +03 (without DST) to +02 (with DST) on 1941-09-19 at 24:00, which means that timestamps that day between 23:00 and 24:00 are ambiguous in Kiev. This transition has been present since the release of tzdata 1999j two decades ago. I'm sure there are other examples.

Maybe I'm just missing it but as far as I can tell Europe/Kiev is implemented as a positive offset still such that the Summer Time (i.e. +03) would still have DST on and the DST off period would be +02.

Not for the 1941 transition I'm talking about. In that transition, Kiev simultaneously (1) changed its STD-UTC offset from +03 to +01, and (2) changed from standard time to daylight-saving time with the usual 1-hour saving. The combined effect of the two simultaneous changes was to (1) switch Kiev's UTC offset from +03 to +02, and (2) change its tm_isdst flag from zero to positive. This transition has the properties you mentioned as possibly being problematic for your software, as it has ambiguous timestamps just before a transition from non-DST to DST.

This is the use case we are used to. This would align to a statement made when Ireland rules were first introduced suggesting this was the first implementation of negative offsets in tzdata (https://mm.icann.org/pipermail/tz/2018-January/025876.html).

That email was talking about something different: a negative SAVE value in a Rule line. That's a different way to cause the kind of transition that you mentioned might cause a problem for your software.

On 5/30/19 7:26 AM, Brandon Smith wrote:

libraries and applications have forever been written based on the definition and understand that clocks move forward with DST transitions.  I know it has been said that this is a 'bad assumption', but the reality as I see it is that generally speaking all software has been written based upon what developers knew to be a 'definition' and not an 'assumption' at all.

Although some libraries have been based on that assumption, it is certainly not universal. On the contrary, IEEE Std 1003.1-1998 requires support for so-called "negative DST", this requirement has remain unchanged from 1988 through the current (2018) edition of POSIX, and a lot of software supports the POSIX model for time zones. The problem we're facing is that (a) some non-POSIX-compatible libraries have trouble with negative DST, and (b) details about what these libraries actually do support remain unclear.

I know it was alluded to that TZDB could perhaps use the next release to force the hand of the negative dst offset discussion (i.e. remove rearguard).

There's no intent to remove rearguard format in the next release. You'll still be able to get rearguard format by running "make rearguard_tarballs", a command that should run automatically if you have the right tools: basically a POSIX environment and some freely-available portable programs like GNU tar. The change I'm considering is merely to have downstream providers or users run "make rearguard_tarballs" if they need rearguard tarballs. Part of the idea is to simplify distribution and to make distribution errors less likely. Part of the idea is to nudge downstream users of non-POSIX-compatible software to rely on their upstream providers instead of having users individually deal with any compatibility issues. And part of the idea is to encourage those who get data directly from iana.org to use software that is compatible with the main data format. The overall goal is to decouple downstream users from upstream changes, so that we don't have to wait for all downstream users to upgrade before we can install changes upstream. These are all worthy goals. Of course one must consider both costs and benefits, but overall the next release seems like a good time to try out a change like this.

My company has not been able to adapt to the new negative offsets What have been the obstacles, and what progress has been made so far? That is, the problem has been publicized widely for well over a year; what's the rough timeframe for your company's process for adapting to negative offsets?

On 2019-05-31 18:19, Guy Harris wrote:
> On May 31, 2019, at 3:49 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>> On the contrary, IEEE Std 1003.1-1998 requires support for so-called "negative DST",
> Can we promote the term "Daylight Shifting Time"?
> 1) Nothing's being "saved", it's just moved from the morning to the evening.
> 2) It can refer either to shifting it from the morning to the evening, as 
> many countries do in the spring, or to shifting it from the evening to the
> morning, as Ireland does in the autumn.
> 3) You don't have to worry about "Saving" vs. "Savings".
> 4) It's still "DST", so you don't have to change, for example, "tm_isdst".

I like the POSIZ TZ env var use of alternative time zone.

Most government decrees say something like we observe standard or true time zone
offset X from effective time until end time then summer or daylight or
alternative time zone offset Y from later effective time until next end time.

In these statements the times and offsets are all independent, depending on
latitude as the times of the seasons change between hemispheres.

A better model for seasonal time changes would just be time zone offsets between
times, ignoring how many transitions per year, or definitions of such as
standard or summer, which should be reduced to just labels for the time zones
during those periods: who really cares or needs to know which is which?

For the current period of Ramadan where some time zones vary, what should the
label be now: the equivalent of Location Ramadan Time, and should the preceding
or following period label be Location Summaer Time?

Each seasonal industry deals with its own seasonal time periods, uncorrelated to
government decrees and astronomical equinoxes, without any difficulties or
regulations.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

On 2019-05-31 21:00, Paul Eggert wrote:

Brian Inglis wrote:

...

For the current period of Ramadan where some time zones vary, what should the label be now: the equivalent of Location Ramadan Time, and should the preceding or following period label be Location Summer Time?

That naming convention would not hold up, I'm afraid. Although Ramadan occurs during northern-hemisphere summer this year, the Islamic calendar is lunar and does not stay in sync with the Gregorian calendar, and by 2026 Ramadan will be occurring during northern-hemisphere winter.

We don't have to worry about inventing a naming convention for this in tzdb, as it is using the labels +01 and +00 for time in Morocco.

The query is what does the (CLDR/ICU) label apply to: the offset or the time period? The label would seem to apply to the period of Location Summer Time, Location Winter Time, Location Daylight Saving Time, Location Standard Time, rather than denoting the time zone offset. So what then should you label the extra periods: use the same labels as the previous uses of the same offsets, or give the extra periods their own unique labels, and what should those unique labels indicate? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.

Paul, CLDR keys off either tzdb ids ( example: Indiana/Marengo https://unicode.org/cldr/charts/latest/by_type/timezones.north_america.html#... <https://unicode.org/cldr/charts/latest/by_type/timezones.north_america.html#...> ) or “metazones”, see https://www.unicode.org/reports/tr35/tr35-dates.html#Metazone_Names <https://www.unicode.org/reports/tr35/tr35-dates.html#Metazone_Names> The metazones are groupings used for translation (formatting) of dates only, not for time calculation. In this specific example, the metazone “Africa_Western” encompasses Africa/Brazzaville, Africa/Bangui, and others. <timezone type="Africa/Windhoek"> <usesMetazone to="1990-03-20 22:00" mzone="Africa_Southern"/> <usesMetazone to="1994-03-20 22:00" from="1990-03-20 22:00" mzone="Africa_Central"/> <usesMetazone to="2017-10-23 22:00" from="1994-03-20 22:00" mzone="Africa_Western"/> <usesMetazone from="2017-10-23 22:00" mzone="Africa_Central"/> </timezone> -- Steven R. Loomis | @srl295 | git.io/srl295

El jun. 1, 2019, a las 12:05 a. m., Paul Eggert <eggert@cs.ucla.edu> escribió:

Brian Inglis wrote:

...

The query is what does the (CLDR/ICU) label apply to: the offset or the time period? The label would seem to apply to the period of Location Summer Time, Location Winter Time, Location Daylight Saving Time, Location Standard Time, rather than denoting the time zone offset. So what then should you label the extra periods: use the same labels as the previous uses of the same offsets, or give the extra periods their own unique labels, and what should those unique labels indicate?

I'm afraid I'm having trouble connecting your question to the CLDR tables, because I don't understand how CLDR works in detail. Looking at <https://unicode.org/cldr/charts/latest/by_type/> it appears that CLDR doesn't key on the string "Africa/Casablanca" when trying to generate labels for time zones. Instead, it keys on strings like "Africa Western" (I don't know where these strings come from; they were never in tzdb) and translates these keys to strings like "heure d’Afrique de l’Ouest" (French). The list of such keys for African time zones is:

Africa Central Africa Eastern Africa Southern Africa Western Cape Verde French Southern Indian Ocean Mauritius Reunion Seychelles

These mostly appear to reflect abbreviations that were formerly in tzdb (e.g., MUT for "Mauritius Time") but were removed from tzdb in 2017 on the grounds that tzdb should reflect existing practice instead of imposing its own invented abbreviations. Evidently CLDR has more of a packrat mentality, and does not remove such entries even if the upstream source (tzdb in this case) no longer uses them even indirectly. And it's not clear to me how a program would automatically map Africa/Casablanca's "+01" and "+02" to "Africa Western" and "Morocco Ramadan" (or whatever).

The whole thing is a bit of a mess, I'm afraid. I'm not sure we'd be doing CLDR favors by inventing names like "Morocco Ramadan Time", as this would merely give translators more make-work that in practice would be useless or confusing or both.

I agree. And note that in many languages, two offsets are not "standard" and "whatever", they are is "summer" and "winter". ("Standard" is a misnomer anyway, when more than half the time it isn't that offset.) Mark On Mon, Jun 3, 2019 at 11:37 AM Robert Elz <kre@munnari.oz.au> wrote:

Date: Fri, 31 May 2019 17:19:22 -0700 From: Guy Harris <guy@alum.mit.edu> Message-ID: <C6840C54-C178-497A-A254-CB736D983542@alum.mit.edu>

| On May 31, 2019, at 3:49 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: | | > On the contrary, IEEE Std 1003.1-1998 requires support for so-called "negative DST", | | Can we promote the term "Daylight Shifting Time"?

It would be more consistent with other such acronyms, and easier to read, if it were "Shifted Daylight Time" (cf: ISO, UTC, ...)

But even better, just call it "the time" and be done with it. Having two names (standard and **whatever**) implies that only two are possible (and leads to the dumb assumptions that people tend to make).

When we need to refer to a specific time in a specific place, on a specific date, we need all of that information to do it properly. Allowing anyone to believe otherwise, ever, is to do them a great disservice, even if in particular cases some of it ends up being implied by other parts.

kre

On Mon 2019-06-03T11:55:33+0200 Mark Davis ☕️ hath writ:

I agree. And note that in many languages, two offsets are not "standard" and "whatever", they are is "summer" and "winter".

("Standard" is a misnomer anyway, when more than half the time it isn't that offset.)

United States law says that both winter and summer are designated as "standard time". The law does not have such a thing as daylight time. https://www.law.cornell.edu/uscode/text/15/260a -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m

Robert Elz wrote:

But even better, just call it "the time" and be done with it. Having two names (standard and **whatever**) implies that only two are possible (and leads to the dumb assumptions that people tend to make).

Indeed. And isn't Antarctica/Troll the example of a place with more than two? (Commented out currently in the antarctica file, to be sure, but expect more fireworks when, as the file says, "2014b is more prevalent"...)

Thanks for the reminder Paul. Ramanand plans to address this issue in the coming weeks. We'll certainly be testing with the latest tzdb development version to ensure everything is working as expected. Regards, Sean. On 27/05/19 19:02, Paul Eggert wrote:

DONATELLA SABELLICO wrote:

...

Do you know when the Timezone package will be released'

We plan to get a new release out reasonably soon, and well before November. However, there is no fixed schedule or specific plan. In the meantime I suggest that you test your software by using the tzdb development version and reporting your results back. You can currently retrieve it by running this command:

git clone https://github.com/eggert/tz

I would like more testing by people other than myself. It's quite possible that the next release will have problems and I want to hear bug reports and "it's all OK!" reports back. I'm not hearing anything, which leads me to suspect that a new release must not be that urgent.

Here is one longstanding issue that I would like to see progress on:

https://www.oracle.com/technetwork/java/javase/tzupdater-readme-136440.html#...

That is, Oracle TZUpdater fails to handle the forms of DST currently used in Ireland and Morocco, and historically used in Czechoslovakia, Japan and Namibia. Although this was reported as a P3 (major loss of function) bug in January 2018 <https://bugs.openjdk.java.net/browse/JDK-8195595>, I am not plugged into the JDK development effort and don't know what sort of progress is being made here.

Ramanand, you're assigned that bug report. What's the status of the fix for this issue, and of testing the fix? Have you tested with data taken from the current tzdb development version? Although I doubt whether the Brazil changes themselves will cause problems with TZUpdater I've been surprised in the past, and also the next release will contain changes not related to Brazil, so I'd appreciate some feedback here.

On 6/21/19 2:02 PM, Roelco Rens wrote:

Any planning already on this new release for the Brazil changes?

The idea is to do it sooner rather than later. If you'll recall from that earlier email, there was a longstanding issue that I wanted to see progress on, related to Oracle TZUpdater. When I asked about that I got email back saying that the relevant person at Oracle was on vacation. They should be back so I'll ping Oracle now. If I don't get a response soon, we can go ahead and issue a new release and deal with any problems in a later release.

Deborah Goldsmith wrote:

Any news?

I exchanged email yesterday with Oracle about this. They are currently looking at modifying TZUpdater to support vanguard format. Although I don't want TZUpdater issues to delay things unnecessarily, I thought I'd wait a bit longer as any TZUpdater fix (or at least clarification about if/when a fix might be expected) could help smooth the next tzdb release.

On 6/26/19 1:51 PM, Deborah Goldsmith wrote:

They’re not the only client of tzdb that’s still using rearguard, so I’m not sure what the issue is?

They're the client that we've gotten the most feedback from; squeaky wheel and all that. The main issue is that I am planning to stop distributing rearguard-format tzdata tarballs from my personal web pages at UCLA. Instead, users or distributors desiring rearguard format should use the output of "make rearguard_tarballs" or of "make rearguard.zi". This procedural change won't alter the tarballs released via IANA. There's also a less-important issue involving CLDR vs current and future Morocco rules. I doubt whether either issue will affect code or data in the next tzdb release. However, it may affect the NEWS file and/or the release announcement.

On 6/26/19 5:47 PM, Matthew Donadio wrote:

would you entertain any help for build automation for releases and/or development nightlies that would be hosted there to offload some of the manual work you do?  Or if one of us maintained a fork on GitHub or BitBucket that did this (only mentioning BB because I am more comfortable with their automation)?

Something like that might be helpful. However, I've been reluctant to do this myself, partly because I think it would make for more work for me (not less), and partly due to licensing concerns: the GitHub platform (as opposed to Git itself) is closed source, and tzdb development should require only portable tools that are freely available so I would rather not rely on GitHub-specific automation. One possibility is to do build automation on some platform that is not closed-source; I don't know whether BitBucket has similar issues. At any rate I wouldn't let any of this stop you; if you can easily build something downstream on BitBucket, that would be a useful resource I'd think, even if it's not part of tzdb proper.