Thanks for the comments, I'll update the proposal in response. My responses are here:

* Having global state can lead to problems (even in single-threaded apps), and this proposal has global state in the implementation of the 'set_tzpath' function. Instead of that function, how about an optional argument to the ZoneInfo constructors? The optional argument would specify the path for apps that don't want the default path. Then apps wouldn't need set_tzpath.

Yes, `set_tzpath` is the part of this proposal that I am most ambivalent about and also, unsurprisingly, the one I've gotten the most pushback about. My response to that aspect of it is found in my last reply in this post: https://discuss.python.org/t/pep-615-support-for-the-iana-time-zone-database... Basically, both the mutable global state and the constructor argument have undesirable interactions with the caching behavior of the main constructor (and dropping the caching behavior would cause worse problems). My position is that you probably /shouldn't/ call set_tzpath except in specific situations like tests or "call it once in main() before anything else has run", and that a free function that mutates the global state accomplishes this better and encourages people not to make a bunch of calls to the constructor with inconsistent search paths. I'm amenable to dropping the whole thing, though, TBH.

* The proposal is silent on the issue of what happens during updates to tzdata files while a Python program is running. Should this update be reflected immediately in all Python-level calls? Should it wait until the next call to ZoneInfo on the affected zone, which means that zones are not immutable? Should the update be ignored and the program proceed as if the update hasn't happened? Or are all these valid implementations of the spec? Whichever you decide, the decision should be written down so that developers are put on notice.

I am going to update this to clarify, but I think this is /mostly/ covered by the caching behavior described in the section on constructors <https://www.python.org/dev/peps/pep-0615/#id6>, once I make explicit the assumption that the full time zone data must be eagerly read into memory at construction (rather than being implemented in terms of system calls or something of that nature). With that assumption in place, the answer is that the data is updated whenever a cache miss occurs - the first time any given zone is constructed or, depending on the implementation, the first time it is constructed after a previous version has been ejected from the cache (in the reference implementation, we use a "strong" LRU cache of 8 zones and an unbound "weak" cache, so if you construct 9 zones and hold no references to any of them, constructing the first one again will be a cache miss, and the other 8 will be a cache hit). This does mean that if you call ZoneInfo("America/New_York") when your installed zoneinfo data is 2019c and then you upgrade to 2020a and call ZoneInfo("US/Eastern"), the two objects may have different behaviors, but I think this is mainly unavoidable without a pretty significant performance impact. In any case, I'll add a whole section on "what happens if the tzdata is updated during an interpreter run", thanks for the pointer.

* How do I get tzdata-based leap second support, if I want it? Or are leap seconds specifically out of scope?

Beyond the fact that I plan to ship non-"zone" files in the tzdata fallback package (and thus include the leap seconds), leap seconds are out of scope for this proposal. Python's datetime type has no support for leap seconds currently, and other than being tracked in the same database, I think they're at least somewhat orthogonal to the primary problem we're solving here (a tzinfo implementation). Leap second support is on my long list of improvements for the datetime module, so I'll probably get around to it at some point in the future. I have had another query about leap seconds, so I'll probably want to add to the proposal that it's out of scope.

* You might want to mention TZDIST (RFC 7808) as an alternative source for tzdata that a Python implementation could use instead of TZif files in a local filesystem.

Yes, I will have to look into this. My main concern is that my hope is to try to use a time zone data source that can be managed at the system level, independent of language stack. I will admit to never having looked into the details, but I was under the impression that tzdist was something that the system would consume, rather than individual programs, is that wrong? I also am not clear - are there public tzdist servers, or is the suggestion that we would have a Python-specific tzdist service and end users would subscribe to it for updates? I'm mainly asking because I decided early on (on some very good advice) that effectively distributing the data is a big enough task on its own that it would bog down the initial implementation to try and handle both at once, so my goal with this is to get something that will work /if you have the data/, and provide a reasonable way to get the data and handle the data distribution in a separate proposal. If tzdist is consistent with a backwards compatible upgrade from a version using TZif files at some point in the future, I'm happy to put it off as, "We should look into this when we try to solve the distribution problem." It sorta seems like it /should/ be possible to seamlessly transition from system files to tzdist (at least depending on how strong our promises are about the tz search path, anyway). In any case, thanks so much for the comments, you've given me a lot of food for thought already! Best, Paul

Thanks for this perspective. To clarify, we already have some (albeit somewhat hidden) functionality related to handling time zone /locales/ and local system time. This particular proposal (PEP 615) is only to provide an interface with the time zone database (i.e. it is for mapping the keys America/New_York to a representation of the data), it is not for supporting querying the system time zone locales. On 2/25/20 11:39 PM, Brian Inglis wrote:

On 2020-02-25 19:40, Paul Ganssle wrote:

...

Thanks for the comments, I'll update the proposal in response. In any case, thanks so much for the comments, you've given me a lot of food for thought already! You may also want to bear in mind and take account of the existing system conventions around default user and system time zones, locales, and formats (I get annoyed if your tool ignores my configured preferences, or I have to set yet another preference, which probably lacks features, sucks, or doesn't do what I want e.g. MS Windows settings do not support time zone formats like %z %Z):

$ echo $TZ # env var TZ contains the user default time zone name America/Edmonton $ head /etc/timezone # /etc/timezone contains the system default timezone name America/Edmonton $ llgo /etc/localtime # /etc/localtime links to the system default timezone file lrwxrwxrwx 1 36 Feb 25 16:23 /etc/localtime -> /usr/share/zoneinfo/America/Edmonton $ l /usr/share/i18n/locales/en* # template locale definitions /usr/share/i18n/locales/en_AG /usr/share/i18n/locales/en_IN /usr/share/i18n/locales/en_AU /usr/share/i18n/locales/en_NG /usr/share/i18n/locales/en_BW /usr/share/i18n/locales/en_NZ /usr/share/i18n/locales/en_CA /usr/share/i18n/locales/en_PH /usr/share/i18n/locales/en_DK /usr/share/i18n/locales/en_SC /usr/share/i18n/locales/en_GB /usr/share/i18n/locales/en_SG /usr/share/i18n/locales/en_HK /usr/share/i18n/locales/en_US /usr/share/i18n/locales/en_IE /usr/share/i18n/locales/en_ZA '/usr/share/i18n/locales/en_IE@euro' /usr/share/i18n/locales/en_ZM /usr/share/i18n/locales/en_IL /usr/share/i18n/locales/en_ZW $ apropos locale-gen # generates configured definitions from templates locale-gen (8) - generates localisation files from templates locale.gen (5) - Configuration file for locale-gen $ llgo /etc/locale.* -rw-r--r-- 1 3.0K May 1 2019 /etc/locale.alias -rw-r--r-- 1 9.2K Feb 10 12:05 /etc/locale.gen $ localedef --list-archive # shows the system locales supported in # LOCPATH default $LOCALEDIR/locale-archive # LOCALEDIR default /usr/lib/locale/ en_CA en_CA.iso88591 en_CA.utf8 en_GB en_GB.iso88591 en_GB.utf8 en_US en_US.iso88591 en_US.utf8 $ locale # shows the current user locale settings LANG=en_CA.UTF-8 LANGUAGE=en_CA:en_GB:en_UK:en LC_CTYPE="en_CA.UTF-8" LC_NUMERIC="en_CA.UTF-8" LC_TIME="en_CA.UTF-8" LC_COLLATE="en_CA.UTF-8" LC_MONETARY="en_CA.UTF-8" LC_MESSAGES="en_CA.UTF-8" LC_PAPER="en_CA.UTF-8" LC_NAME="en_CA.UTF-8" LC_ADDRESS="en_CA.UTF-8" LC_TELEPHONE="en_CA.UTF-8" LC_MEASUREMENT="en_CA.UTF-8" LC_IDENTIFICATION="en_CA.UTF-8" LC_ALL= $ locale -ck LC_TIME # shows the current user locale time category settings LC_TIME abday="Sun;Mon;Tue;Wed;Thu;Fri;Sat" day="Sunday;Monday;Tuesday;Wednesday;Thursday;Friday;Saturday" abmon="Jan;Feb;Mar;Apr;May;Jun;Jul;Aug;Sep;Oct;Nov;Dec" mon="January;February;March;April;May;June;July;August;September;October;November;December" am_pm="am;pm" d_t_fmt="%Y %b %d %a %T%z" d_fmt="%F" t_fmt="%T" t_fmt_ampm="%I:%M:%S %p %Z" era= era_year="" era_d_fmt="" alt_digits= era_d_t_fmt="" era_t_fmt="" time-era-num-entries=0 time-era-entries="S" week-ndays=7 week-1stday=19971130 week-1stweek=1 first_weekday=2 first_workday=2 cal_direction=1 timezone="" date_fmt="%Y %b %e %a %T %Z" time-codeset="UTF-8" alt_mon="January;February;March;April;May;June;July;August;September;October;November;December" ab_alt_mon="Jan;Feb;Mar;Apr;May;Jun;Jul;Aug;Sep;Oct;Nov;Dec"

On 2/26/20 7:14 AM, Paul Ganssle wrote:

I have heard a potential proposal for doing things like "find out what ZoneInfo, if any, applies to my current locale and get a time zone object for it", but I consider this mostly out of scope

What may well be in scope, though, is the notion of "wall clock time", which is universally supported and which Python applications will almost surely want to access. On GNU/Linux and similar platforms, wall clock time is what you get when the TZ environment variable is not set. Typically it's determined by /etc/localtime (and defaults to UTC if there is no /etc/localtime). In the tzcode reference implementation, you can create a wall clock timezone object by calling tzalloc(NULL). Python could have ZoneInfo(None) do the same thing. The reference implementation also supports the notion that TZ='' means UTC without leap seconds, though this is less well supported elsewhere and since leap seconds are out of scope for you this is not something you need to worry about now.

On 2020-02-26 14:31, Paul Eggert wrote:

On 2/25/20 6:40 PM, Paul Ganssle wrote:

...

My main concern is that my hope is to try to use a time zone data source that can be managed at the system level, independent of language stack. I will admit to never having looked into the details, but I was under the impression that tzdist was something that the system would consume, rather than individual programs, is that wrong?

As a protocol, tzdist can be used both by individual programs and by the system. It's still pretty new and I don't know of any census, but at least Cyrus supports it so that clients can have the latest tzdata even if their OS tzdata is out-of date; see <https://www.cyrusimap.org/imap/download/installation/http/caldav.html>.

Reading that, Cyrus supports converting tz data sources to iCalendar data with vzic which it then makes available via CalDAV for calendar clients; it does not appear to support zoneinfo binary data or TzDist for tzdata clients: "This module is designed to use the IANA Time Zone Database data (a.k.a. Olson Database) converted to the iCalendar format. Cyrus uses a modified vzic to convert IANA formatted data into iCalendar format." and predates TzDist: "recently downloaded time zone data (e.g. “IANA Time Zone Database v.2013h”)" The CMU server no longer appears to be available.

Absolutely. All I'm saying is that the proposal should not preclude having the implementation use tzdist rather than TZif files in the OS. One can use tzdist to distribute TZif data, after all.

Allow your proposal backend to work with local or remote files or tzdist server. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.