New IANA-timezone support date/time library
I’ve just finished an extensive update of a JavaScript/TypeScript date/time library that I’d previously used only for my own astronomy website, skyviewcafe.com <http://skyviewcafe.com/>. I’ve done enough updates to this library now that it may be of interest to many other software developers, especially if they want to handle the odd edge cases that exist in timezone handling, like 47-hour days and missing days and the like. One unique features is that it's possible for users to get live updates of the tz database, which usually don’t become available to applications without, at the very least, restarting an application, if not rebooting one’s computer. Libraries in this category are hardly unique, but one of the most popular libraries, Moment.js, is being phased out, so this seemed to be to be a good time to introduce a new one. Among the (possible) unique features, you can do things like find the specific moments of DST transitions and other transitions, like UTC offset changes, and get descriptions of these changes, like this: new DateTime('2021-11-07', 'America/New_York').getDiscontinuityDuringDay() → { start: '02:00:00', end: '01:00:00', delta: -3600000 } // fall back! // As soon as it’s midnight on the 30th, it’s instantly midnight on the 31st, erasing 24 hours:<br> new DateTime('2011-12-30', 'Pacific/Apia').getDiscontinuityDuringDay()) → { start: '00:00:00', end: '24:00:00', delta: 86400000 } // As soon as it’s midnight on the 31th, turn back to 1AM on the 30th, adding 23 hours to the day:<br> new DateTime('1969-09-30', 'Pacific/Kwajalein').getDiscontinuityDuringDay() → { start: '24:00:00', end: '01:00:00', delta: -82800000 } My updated library can be found here: https://www.npmjs.com/package/@tubular/time <https://www.npmjs.com/package/@tubular/time> Besides the IANA timezone support, this library also features the ability to handle variable switch-over from the Julian to the Gregorian calendar, which can be of helpful to those handling historical dates.
Thanks for the info. I scanned through the Readme page, but I couldn't find the answer to my question: Does your library have the ability to set the underlying database to a specific TZDB version (e.g. 2019f, 2021a)? I would like this feature to get deterministic functional testing, instead of having the OS change the underlying TZDB version underneath me. The HowardHinnant C++ date library (https://github.com/HowardHinnant/date) is the only library that I know that has this feature. I am looking for a second library. On Mon, Feb 22, 2021 at 9:32 AM Kerry Shetline via tz <tz@iana.org> wrote:
I’ve just finished an extensive update of a JavaScript/TypeScript date/time library that I’d previously used only for my own astronomy website, skyviewcafe.com. I’ve done enough updates to this library now that it may be of interest to many other software developers, especially if they want to handle the odd edge cases that exist in timezone handling, like 47-hour days and missing days and the like. One unique features is that it's possible for users to get live updates of the tz database, which usually don’t become available to applications without, at the very least, restarting an application, if not rebooting one’s computer.
Libraries in this category are hardly unique, but one of the most popular libraries, Moment.js, is being phased out, so this seemed to be to be a good time to introduce a new one.
Among the (possible) unique features, you can do things like find the specific moments of DST transitions and other transitions, like UTC offset changes, and get descriptions of these changes, like this:
new DateTime('2021-11-07', 'America/New_York').getDiscontinuityDuringDay() → { start: '02:00:00', end: '01:00:00', delta: -3600000 } // fall back!
// As soon as it’s midnight on the 30th, it’s instantly midnight on the 31st, erasing 24 hours:<br> new DateTime('2011-12-30', 'Pacific/Apia').getDiscontinuityDuringDay()) → { start: '00:00:00', end: '24:00:00', delta: 86400000 }
// As soon as it’s midnight on the 31th, turn back to 1AM on the 30th, adding 23 hours to the day:<br> new DateTime('1969-09-30', 'Pacific/Kwajalein').getDiscontinuityDuringDay() → { start: '24:00:00', end: '01:00:00', delta: -82800000 }
My updated library can be found here: https://www.npmjs.com/package/@tubular/time
Besides the IANA timezone support, this library also features the ability to handle variable switch-over from the Julian to the Gregorian calendar, which can be of helpful to those handling historical dates.
On 2021-02-22, at 11:04:48, Brian Park via tz wrote:
Thanks for the info. I scanned through the Readme page, but I couldn't find the answer to my question: Does your library have the ability to set the underlying database to a specific TZDB version (e.g. 2019f, 2021a)?
How many TZDB versions do you want to support?
I would like this feature to get deterministic functional testing, instead of having the OS change the underlying TZDB version underneath me. The HowardHinnant C++ date library (https://github.com/HowardHinnant/date) is the only library that I know that has this feature. I am looking for a second library.
I notice that on MacOS or Linx I can use parent directory (TZ=../.. etc.) references and/or symbolic links to access a customized version of TZDB Does that help you? -- gil
On Mon, Feb 22, 2021 at 1:24 PM Paul Gilmartin via tz <tz@iana.org> wrote: On 2021-02-22, at 11:04:48, Brian Park via tz wrote:
Thanks for the info. I scanned through the Readme page, but I couldn't find the answer to my question: Does your library have the ability to set the underlying database to a specific TZDB version (e.g. 2019f, 2021a)?
How many TZDB versions do you want to support?
Personally, I have run into instances where I have need arbitrary versions for testing purposes. For example, simulating a version an OS / distro had installed when doing something like a `git bisect`. --Matthew Donadio (matt@mxd120.com)
At this point, the @tubular/time library only works with whatever the latest tzdb version was at the time a particular release gets built, or whatever the latest version of tzdb is that has been made available for download live via the release of a new version of @tubular/time. The library doesn’t go straight to the tzdb to grab the updates, but rather to its own repo where the data has been pre-processed into a more easily digested form. I am, however, considering translating the Java code that does the preprocessing into JavaScript, and I’ll definitely consider making that code available as an add-on module to my main library. The Java code can access all tzdb versions, as will any JavaScript port, so that should provide what you’d be looking for. Just a vague plan for now, however. No release schedule ;) -Kerry
On 2021-02-22, at 13:04, Brian Park <brian@xparks.net> wrote:
Thanks for the info. I scanned through the Readme page, but I couldn't find the answer to my question: Does your library have the ability to set the underlying database to a specific TZDB version (e.g. 2019f, 2021a)?
I would like this feature to get deterministic functional testing, instead of having the OS change the underlying TZDB version underneath me. The HowardHinnant C++ date library (https://github.com/HowardHinnant/date <https://github.com/HowardHinnant/date>) is the only library that I know that has this feature. I am looking for a second library.
My Noda Time <https://nodatime.org> library for .NET supports this as well. See https://nodatime.org/3.0.x/userguide/tzdb for more details, including a source code example. Finally, java.time supports this too to some extent via ZoneRulesProvider <https://docs.oracle.com/javase/10/docs/api/java/time/zone/ZoneRulesProvider....>, although there's a significant gap in terms of a public implementation that allows you to load data from a file or directory. (I suspect that sort of flexible implementation is present, but not public.) There may be third party implementations. Jon On Mon, 22 Feb 2021 at 18:12, Brian Park via tz <tz@iana.org> wrote:
Thanks for the info. I scanned through the Readme page, but I couldn't find the answer to my question: Does your library have the ability to set the underlying database to a specific TZDB version (e.g. 2019f, 2021a)?
I would like this feature to get deterministic functional testing, instead of having the OS change the underlying TZDB version underneath me. The HowardHinnant C++ date library (https://github.com/HowardHinnant/date) is the only library that I know that has this feature. I am looking for a second library.
On Mon, Feb 22, 2021 at 9:32 AM Kerry Shetline via tz <tz@iana.org> wrote:
I’ve just finished an extensive update of a JavaScript/TypeScript date/time library that I’d previously used only for my own astronomy website, skyviewcafe.com. I’ve done enough updates to this library now that it may be of interest to many other software developers, especially if they want to handle the odd edge cases that exist in timezone handling, like 47-hour days and missing days and the like. One unique features is that it's possible for users to get live updates of the tz database, which usually don’t become available to applications without, at the very least, restarting an application, if not rebooting one’s computer.
Libraries in this category are hardly unique, but one of the most popular libraries, Moment.js, is being phased out, so this seemed to be to be a good time to introduce a new one.
Among the (possible) unique features, you can do things like find the specific moments of DST transitions and other transitions, like UTC offset changes, and get descriptions of these changes, like this:
new DateTime('2021-11-07', 'America/New_York').getDiscontinuityDuringDay() → { start: '02:00:00', end: '01:00:00', delta: -3600000 } // fall back!
// As soon as it’s midnight on the 30th, it’s instantly midnight on the 31st, erasing 24 hours:<br> new DateTime('2011-12-30', 'Pacific/Apia').getDiscontinuityDuringDay()) → { start: '00:00:00', end: '24:00:00', delta: 86400000 }
// As soon as it’s midnight on the 31th, turn back to 1AM on the 30th, adding 23 hours to the day:<br> new DateTime('1969-09-30', 'Pacific/Kwajalein').getDiscontinuityDuringDay() → { start: '24:00:00', end: '01:00:00', delta: -82800000 }
My updated library can be found here: https://www.npmjs.com/package/@tubular/time
Besides the IANA timezone support, this library also features the ability to handle variable switch-over from the Julian to the Gregorian calendar, which can be of helpful to those handling historical dates.
Jon and I attacked the problem from opposite angles. The InnerDrive.TimeZones<https://www.nuget.org/packages/InnerDrive.TimeZones/> package interprets tzdb rather than compiling it, so by definition it takes arbitrary versions of the data. Of course, Jon’s runs about 8 times faster… From: tz <tz-bounces@iana.org> On Behalf Of Jon Skeet via tz Sent: Tuesday 23 February 2021 02:11 To: Brian Park <brian@xparks.net>; IANA TZ Database <tz@iana.org> Subject: Re: [tz] New IANA-timezone support date/time library My Noda Time<https://nodatime.org> library for .NET supports this as well. See https://nodatime.org/3.0.x/userguide/tzdb for more details, including a source code example. Finally, java.time supports this too to some extent via ZoneRulesProvider<https://docs.oracle.com/javase/10/docs/api/java/time/zone/ZoneRulesProvider....>, although there's a significant gap in terms of a public implementation that allows you to load data from a file or directory. (I suspect that sort of flexible implementation is present, but not public.) There may be third party implementations. Jon
@Kerry Shetline: I think you are saying that that tubular/time parses the TZDB files into its own format, and packages the data into the library. It looks like you provide 3 versions of the database. As far as I can tell, the version number is stored only as comments in `src/timezone-small.ts`, `src/timezone-large-alt.ts`, and `src/timezone-large.ts`. Your Readme does not mention the TZDB version number. I found the version number in the GitHub release notes, but I wasn't able to find the release notes from the npmjs.com site. Would consider adding an API to provide programmatic access to the TZDB version number? @Paul Gilmartin: With regards to how many TZDB versions I want to support, ideally I want to be able to specify the last several versions. Within the same OS version or Docker image, I would like at least 2 versions: One version for the most recent TZDB release (e.g. 2021a), so that I can test it as soon as it is released, and one version for the previous release (e.g. 2020f). Often I want a 3rd version, for whichever version was packaged with the JDK that comes on that particular Docker image. But sometimes the version that comes with the `python-pytz` library is different, so I need a 4th version. Your hint about the TZ environment variable is interesting, and it might be useful for the `python-dateutil` library. (I found the instructions for regenerating the `/usr/share/zoneinfo` files here: https://unix.stackexchange.com/questions/457862/how-can-i-install-the-tz-dat...). I had to give up on python-dateutil, because although it comes bundled with a specific TZDB version, it overrides that with the OS tz files if they exist, so I could not get reproducible test results on different Docker images. @Jon Skeet: Thanks for confirming that Noda Time has support for selecting the TZDB version. I was aware of your library, and was impressed with its documentation, but I don't know .NET or C# so I didn't want to invest the time without knowing whether it had support for this feature. But now that I do, maybe I'll learn enough C# to validate against your library. With regards to java.time, I knew about ZoneRulesProvider, but I could never find sufficient documentation about how to actually generate my own tz database for java.time. @David Braverman: Thanks for the pointer to InnerDrive.TimeZones. I'm not familiar with the .NET ecosystem, so I can't quite figure out what the InnerDrive framework does. But I will keep it in mind.
Thanks for mentioning this. I installed the attached to link to it, as well to some other libraries I found (this seems to be an area that's rapidly evolving). I am by no means up-to-speed on JavaScript support for tzdb, so please feel free to suggest further changes.
Kerry, that library looks like a good addition to the JS ecosystem. The APIs around discontinuities and live updates are not easily found in most other JS libraries. Though I have some queries, based on some of the larger community grumblings about the old Moment API: 1. I'm intrigued by the dual mutable/immutable modes. Moment.js being mutable caused a lot of bugs and confusion, and most other libraries since then have been immutable-only. 2. Is it possible for consumers to compile their own version of the data? I couldn't see any reference to how the custom data files are compiled. With Moment Timezone being in maintenance mode, its releases aren't always keeping up with tzdb updates, but at least there's a documented procedure for people to be able to compile and load the data themselves. 3. The compiled data files appear to contradict the tzdb source when it comes to primary zones vs aliases. For example, in tzdb Australia/Sydney is a primary Zone entry, with aliases such as Australia/ACT and Australia/NSW. But I noticed your library has Australia/ACT as the primary entry instead. (To be fair, Moment Timezone also has this issue, due to the way its data files are compiled [1].) 4. As I think someone else mentioned, we're trying to discourage the raw display of time zone identifiers to end users, because they're very often misinterpreted. The "theory" file in tzdb has more detail [2]. I think having an API to return zone identifiers grouped by region might encourage the display of identifiers in zone picker interfaces, which we'd like to avoid. On another note, there is a proposal currently being developed for a new core JS API around date/time handling: https://github.com/tc39/proposal-temporal The API is close to being finalised, but I encourage you to take a look at the proposal and give any feedback you might have. Your experience in building this library would be helpful in identifying potential problems, especially regarding different calendars. [1]: https://github.com/moment/moment-timezone/issues/835 [2]: https://data.iana.org/time-zones/theory.html#naming On 24 Feb 2021, at 06:07, Paul Eggert via tz <tz@iana.org> wrote: Thanks for mentioning this. I installed the attached to link to it, as well to some other libraries I found (this seems to be an area that's rapidly evolving). I am by no means up-to-speed on JavaScript support for tzdb, so please feel free to suggest further changes. Paul, that list looks good to me. You've captured the main libraries favoured by the JS community. Cheers, Gil
participants (8)
-
Brian Park -
David Braverman -
Gilmore Davidson -
Jon Skeet -
Kerry Shetline -
Matthew Donadio -
Paul Eggert -
Paul Gilmartin