EST5EDT should not resolve to America/New_York because they have different offsets
Hello - I recently filed a GitHub issue against the perl DateTime::TimeZone library, which uses your database: https://github.com/houseabsolute/DateTime-TimeZone/issues/58#issue-286444592... I noticed that EST5EDT now resolves to America/New_York. This is problematic because the two do not have the same offset - EST5EDT is -5:00 and America/New_York is -04:56:02. My ticket was closed because the behavior appears to be caused by changes in your database in commit a0b09c0 ( https://github.com/eggert/tz/commit/a0b09c0230089252acf2eb0f1ba922e99f7f4a03 ). Is this something that you would consider changing? Thanks! - Austin Hill
On Thu, 20 Feb 2025 at 18:46, Austin Hill via tz <tz@iana.org> wrote:
Hello -
I recently filed a GitHub issue against the perl DateTime::TimeZone library, which uses your database: https://github.com/houseabsolute/DateTime-TimeZone/issues/58#issue-286444592...
I noticed that EST5EDT now resolves to America/New_York. This is problematic because the two do not have the same offset - EST5EDT is -5:00 and America/New_York is -04:56:02.
That was the offset prior to 1920, but it's -5:00 now. Is it meaningful to use EST5EDT for dates before the UNIX epoch? If not, then any date that might use EST5EDT will have the correct offset.
My ticket was closed because the behavior appears to be caused by changes in your database in commit a0b09c0 (https://github.com/eggert/tz/commit/a0b09c0230089252acf2eb0f1ba922e99f7f4a03).
Is this something that you would consider changing?
Thanks! - Austin Hill
On Thu, 20 Feb 2025 at 19:01, Jonathan Wakely <jwakely@redhat.com> wrote:
On Thu, 20 Feb 2025 at 18:46, Austin Hill via tz <tz@iana.org> wrote:
Hello -
I recently filed a GitHub issue against the perl DateTime::TimeZone library, which uses your database: https://github.com/houseabsolute/DateTime-TimeZone/issues/58#issue-286444592...
I noticed that EST5EDT now resolves to America/New_York. This is problematic because the two do not have the same offset - EST5EDT is -5:00 and America/New_York is -04:56:02.
That was the offset prior to 1920, but it's -5:00 now. Is it meaningful to use EST5EDT for dates before the UNIX epoch? If not, then any date that might use EST5EDT will have the correct offset.
Your github issue uses the date 0000-01-01 as the example, but it seems pretty meaningless to refer to either America/New_York or EST4EDT for that date.
My ticket was closed because the behavior appears to be caused by changes in your database in commit a0b09c0 (https://github.com/eggert/tz/commit/a0b09c0230089252acf2eb0f1ba922e99f7f4a03).
Is this something that you would consider changing?
Thanks! - Austin Hill
On 2025-02-20 05:40, Austin Hill via tz wrote:
Is this something that you would consider changing?
The change to the obsolescent Zone EST5EDT, which made it a link to America/New_York, was intentional and was in response to a bug report by Justin Grant related to JavaScript. See: https://github.com/eggert/tz/commit/a0b09c0230089252acf2eb0f1ba922e99f7f4a03 Formerly, EST5EDT corresponded to an imaginary time zone history; now it corresponds to the history of a documented location. For what it's worth, the name EST5EDT is present for backward compatibility for long-obsolete Unix System V platforms, and shouldn't be used in new software.
Paul Eggert via tz <tz@iana.org> writes:
Formerly, EST5EDT corresponded to an imaginary time zone history; now it corresponds to the history of a documented location. For what it's worth, the name EST5EDT is present for backward compatibility for long-obsolete Unix System V platforms, and shouldn't be used in new software.
I take the OP's point to be that if we're keeping it around for System V compatibility, it ought to continue to mean what it meant on System V, whether that corresponds to any physical location or not. regards, tom lane
On Feb 20, 2025, at 11:26 AM, Tom Lane via tz <tz@iana.org> wrote:
I take the OP's point to be that if we're keeping it around for System V compatibility, it ought to continue to mean what it meant on System V, whether that corresponds to any physical location or not.
I don't remember whether the tz code originally supported, for TZ settings that don't correspond to files, parsing the setting the same way non-file-based TZ parsers do. If not, perhaps files such as "EST5EDT" were created to allow (some of) those settings to work. Support for full SV/POSIX parsing of TZ settings was added at some point; if the intent is to have those settings do *exactly* what they would do on systems without file-based TZ implementations, perhaps we should no longer supply those files. (On the other hand, that may break sites that depend on them working the way they do with the existing files.)
On 2025-02-20 11:26, Tom Lane wrote:
I take the OP's point to be that if we're keeping it around for System V compatibility, it ought to continue to mean what it meant on System V
Unfortunately doing that would cause more trouble than it'd cure, as the System V EST5EDT rules don't match what clocks do now in the eastern US. I think they also don't correspond to what clocks did there before 1970. In other words, users don't really want true System V compatibility. They just want to keep using the same name for US eastern time.
it ought to continue to mean what it meant on System V, whether that corresponds to any physical location or not.
I have no opinion about what Zone the Link EST5EDT should map to. (Although I'd be inclined to avoid changing it unless there's a really strong reason to cause the churn.) But I have a strong opinion that EST5EDT and other deprecated Zone names should remain Link names, because: * The current TZDB policy for merging zones is that they have the same rules since 1970. It would be weird if widely-used Zone names like Europe/Oslo are merged while obscure, deprecated System V names are not. * Every Zone name means more overhead for the ecosystem: more cached strings on devices, more localized strings needed, more user-facing noise in UI time zone pickers, etc. Spending that overhead for long-deprecated names seems like an unwise tradeoff.
I recently filed a GitHub issue against the perl DateTime::TimeZone library, which uses your database:
https://github.com/houseabsolute/DateTime-TimeZone/issues/58#issue-286444592...
The core problem you're running into is that this library is eagerly "canonicalizing" (resolving the Zone name for a Link name) its input names. This means that code using this library may change its behavior whenever the host system updates its time zone rules. Current JavaScript implementations have the same problem, which has resulted in previously-working code (e.g. automated tests) suddenly breaking after an OS update. Fear of those breaks has led some JS implementations to refuse to update to modern Zone names. For example, Chrome and Safari will transform Asia/Kolkata inputs to Asia/Calcutta outputs. Same for Asia/Ho_Chi_Minh => Asia/Saigon and Europe/Kyiv => Europe/Kiev. This has caused many years of developer complaints <https://docs.google.com/presentation/d/1MVBKAB8U16ynSHmO6Mkt26hT5U-28OjyG9-L...> . To avoid these problems, JS's new Temporal.ZonedDateTime <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...> API will not resolve Link names to Zone names when constructing these objects. If you provide Asia/Calcutta or EST5EDT as input, we'll store* that name and give it back* to the caller if they ask for it. (*Well, we actually return a case-normalized version of the string so that implementations can store a 10-bit index into the ~600 Zone and Link names, so if you provide asIA/CalCUTTA then you'll get Asia/Calcutta back. But capitalization won't change across TZDB releases so this optimization doesn't interfere with the goal of not breaking programs when the OS is updated.) Other libraries like Joda-Time used to canonicalize their inputs, but they stopped doing it for similar reasons as noted above: to avoid churn after TZDB updates. We encourage all time zone library maintainers to do the same thing that JS Temporal.ZonedDateTime will be doing: don't change users' Zone or Link name inputs other than normalizing letter case. This may help the overall TZDB ecosystem be more stable in the face of changes, and will make it less risky for users to update their TZDB data without risking breaking their software. Thanks, Justin
On Thu, Feb 20, 2025 at 2:30 PM Justin Grant via tz <tz@iana.org> wrote:
The core problem you're running into is that this library is eagerly "canonicalizing" (resolving the Zone name for a Link name) its input names. This means that code using this library may change its behavior whenever the host system updates its time zone rules.
That's not really the issue for this particular library. I should know, since I wrote it. The Perl DateTime-TimeZone distribution is a bit unusual among TZ libraries because it actually compiled the IANA TZ DB down to individual Perl modules. It _never_ looks at the installed tz files. Why? I dunno, it made sense to me when I wrote this stuff many years ago. Nowadays, it seems suboptimal, but I'm not going to change it now, and there is a drop-in (for use with DateTime.pm) alternative that just uses the tz files, DateTime-TimeZone-Tzfile <https://metacpan.org/pod/DateTime::TimeZone::Tzfile>. So the problem in this case is a bit different. if you update your DateTime-TimeZone install, you get a new TZ DB, even if your system hasn't changed.
To avoid these problems, JS's new Temporal.ZonedDateTime <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...> API will not resolve Link names to Zone names when constructing these objects. If you provide Asia/Calcutta or EST5EDT as input, we'll store* that name and give it back* to the caller if they ask for it. (*Well, we actually return a case-normalized version of the string so that implementations can store a 10-bit index into the ~600 Zone and Link names, so if you provide asIA/CalCUTTA then you'll get Asia/Calcutta back. But capitalization won't change across TZDB releases so this optimization doesn't interfere with the goal of not breaking programs when the OS is updated.)
It would be possible for DateTime-TimeZone to preserve the name, but AFAICT the issue in the original email was that the actual time zone rules changed.
We encourage all time zone library maintainers to do the same thing that JS Temporal.ZonedDateTime will be doing: don't change users' Zone or Link name inputs other than normalizing letter case. This may help the overall TZDB ecosystem be more stable in the face of changes, and will make it less risky for users to update their TZDB data without risking breaking their software.
At this point I think it's probably too late to make this the default behavior in Perl's DateTime-TimeZone. There is a big backwards compatibility issue in changes here, because it's been the way it is for so long (first release in 2003!). Cheers, Dave Rolsky http://blog.urth.org https://github.com/autarch
Hi Dave! Yep, agreed on all counts. Popular libraries are often stuck with their original behavior forever. So my advice above (to not canonicalize the user's input time zone identifier to follow links) is much more practical for new libraries, or major versions of existing libraries that are OK to break backwards compatibility.
That's not really the issue for this particular library. I should know, since I wrote it.
Oh! Sorry, I misunderstood. So any disruption of TZDB changes is when a user upgrades your library, not when they upgrade the OS. This is a good strategy to make things more predictable for developers. In JS, there's a mix of approaches. JavaScriptCore (Apple's JS engine) relies on the OS's time zone data, while V8 (Google) and SpiderMonkey (Firefox) use a bundled-in approach like you do, as do most 3rd-party timezone libraries in the JS ecosystem so that they can run in browsers where the file system is unavailable. Best, Justin
On 2025-02-20 02:45 PM, Paul Eggert via tz wrote:
On 2025-02-20 11:26, Tom Lane wrote:
I take the OP's point to be that if we're keeping it around for System V compatibility, it ought to continue to mean what it meant on System V
Unfortunately doing that would cause more trouble than it'd cure, as the System V EST5EDT rules don't match what clocks do now in the eastern US. I think they also don't correspond to what clocks did there before 1970.
In other words, users don't really want true System V compatibility. They just want to keep using the same name for US eastern time. That's true for most users. But not necessarily for special cases.
For example, some professional video facilities might find it better to ignore DST shifts to avoid discontinuities in the SMPTE video timecode, which cannot tolerate a DST shift in some cases, essentially choosing to run "permanent stand time". Yes, the display time would not match the local wall clock in summer time, but that is less important than continuous uninterrupted time. EST5EDT and the others (PST8PDT and such) give administrators that option. I'm maintaining it in my work for that reason.
On Feb 20, 2025, at 12:37 PM, Brooks Harris via tz <tz@iana.org> wrote:
On 2025-02-20 02:45 PM, Paul Eggert via tz wrote:
In other words, users don't really want true System V compatibility. They just want to keep using the same name for US eastern time.
That's true for most users. But not necessarily for special cases.
For example, some professional video facilities might find it better to ignore DST shifts to avoid discontinuities in the SMPTE video timecode, which cannot tolerate a DST shift in some cases, essentially choosing to run "permanent stand time". Yes, the display time would not match the local wall clock in summer time, but that is less important than continuous uninterrupted time. EST5EDT and the others (PST8PDT and such) give administrators that option.
What option? EST5EDT isn't permanent stand(and) time, it supports DST, following some unspecified-by-the-setting rules. If you want *permanent standard time* for the UTC-5:00 time zone, that'd be "EST5", which does *not* correspond to a tzdb file; it works as a TZ setting because the tz code parses TZ POSIX/SV-style if it doesn't correspond to a file.
On 2025-02-20 04:17 PM, Guy Harris wrote:
On Feb 20, 2025, at 12:37 PM, Brooks Harris via tz<tz@iana.org> wrote:
On 2025-02-20 02:45 PM, Paul Eggert via tz wrote:
In other words, users don't really want true System V compatibility. They just want to keep using the same name for US eastern time. That's true for most users. But not necessarily for special cases.
For example, some professional video facilities might find it better to ignore DST shifts to avoid discontinuities in the SMPTE video timecode, which cannot tolerate a DST shift in some cases, essentially choosing to run "permanent stand time". Yes, the display time would not match the local wall clock in summer time, but that is less important than continuous uninterrupted time. EST5EDT and the others (PST8PDT and such) give administrators that option. What option? EST5EDT isn't permanent stand(and) time, it supports DST, following some unspecified-by-the-setting rules.
If you want *permanent standard time* for the UTC-5:00 time zone, that'd be "EST5", which does *not* correspond to a tzdb file; it works as a TZ setting because the tz code parses TZ POSIX/SV-style if it doesn't correspond to a file. Right. My mistake, I meant EST5 and the like.
Brooks Harris via tz <tz@iana.org> writes:
That's true for most users. But not necessarily for special cases.
For example, some professional video facilities might find it better to ignore DST shifts to avoid discontinuities in the SMPTE video timecode, which cannot tolerate a DST shift in some cases, essentially choosing to run "permanent stand time". Yes, the display time would not match the local wall clock in summer time, but that is less important than continuous uninterrupted time. EST5EDT and the others (PST8PDT and such) give administrators that option. I'm maintaining it in my work for that reason.
I don't understand. The historical System V EST5EDT and PST8PDT zones certainly had daylight saving time changes, just on a different schedule than current US time zones. -- Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>
If you want *permanent standard time* for the UTC-5:00 time zone, that'd be "EST5", which does *not* correspond to a tzdb file; it works as a TZ setting because the tz code parses TZ POSIX/SV-style if it doesn't correspond to a file.
Agree. For this case, Etc/GMT+5 seems like the right zone to use. We don't need to bring in obsolete SystemV zones for this case.
participants (9)
-
Austin Hill -
Brooks Harris -
Dave Rolsky -
Guy Harris -
Jonathan Wakely -
Justin Grant -
Paul Eggert -
Russ Allbery -
Tom Lane