leap second theory
I was trying to point someone to a reference describing how the "right" zones can be used to enable a system to handle leap seconds more correctly, but I discovered that this isn't really documented anywhere. (There's Steve Allen's page at https://www.ucolick.org/~sla/leapsecs/right+gps.html , of course, but that describes a distinct variant of the scheme.) Would it make sense to add something like the attached patch to theory.html?
On Aug 17, 2019, at 5:29 AM, Steve Summit <scs@eskimo.com> wrote:
I was trying to point someone to a reference describing how the "right" zones can be used to enable a system to handle leap seconds more correctly, but I discovered that this isn't really documented anywhere. (There's Steve Allen's page at https://www.ucolick.org/~sla/leapsecs/right+gps.html , of course, but that describes a distinct variant of the scheme.)
Would it make sense to add something like the attached patch to theory.html?
I might have it say something more like The mechanism assumes that the time stamps being converted count all seconds since January 1, 1970 00:00:00 UTC, *including* leap seconds. (This is by contrast to Posix time stamps, which explicitly ignore leap seconds.) and Note that time stamps that include leap seconds will not match Posix UTC time_t values. making it a property of the time stamp value rather than of the OS, and emphasizing that the epoch being used is the same as for POSIX time stamps (rather than being whatever the TAI version of January 1, 1970 00:00:00 would be called, PTP-style). A UN*X system might, for example, have gettimeofday() etc. continue to offer POSIX timestamps and have a separate API that returns time stamps that advance by 1 second every second.
Steve Summit wrote:
Would it make sense to add something like the attached patch to theory.html?
Yes, and thanks for suggesting it. I installed the attached patch, which builds on your suggestion and also covers NTP and the 'leapseconds' file, which nowadays is used more often than the direct approach, or at least that's my impression. After I wrote this patch, I saw Guy's comments which I think this patch covers as well.
On Aug 17, 2019, at 4:45 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
… the 'leapseconds' file, which nowadays is used more often than the direct approach, or at least that's my impression.
Here’s another data point for you: http://eel.is/c++draft/time.clock.utc.overview#1 This is the draft C++20 spec which specifies two clocks: 1. system_clock: Models Unix Time (does not count leap seconds). 2. utc_clock: Counts leap seconds. And provides bidirectional mappings between these two sets of time points. And the easiest way to implement that mapping is to read in the IANA tz leapseconds file. Howard
On 2019-08-17 17:38, Howard Hinnant wrote:
On Aug 17, 2019, at 4:45 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
… the 'leapseconds' file, which nowadays is used more often than the direct approach, or at least that's my impression.
Here’s another data point for you: http://eel.is/c++draft/time.clock.utc.overview#1 This is the draft C++20 spec which specifies two clocks: 1. system_clock: Models Unix Time (does not count leap seconds). 2. utc_clock: Counts leap seconds. And provides bidirectional mappings between these two sets of time points. And the easiest way to implement that mapping is to read in the IANA tz leapseconds file.
WG14/C is looking at: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2402.pdf which seems somewhat based on the POSIX standard. The POSIX Austin CSRG is discussing clarifying tm_isdst: https://www.mail-archive.com/search?q=time&l=austin-group-l%40opengroup.org and other issues. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.
Brian Inglis wrote:
WG14/C is looking at:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2402.pdf
which seems somewhat based on the POSIX standard.
Thanks for the heads-up. I see that this WG14/C draft is calling localtime_r "reentrant", but it's not: in POSIX, localtime_r relies on global state to obtain the time zone, and behavior is undefined if one thread calls localtime_r while another thread is changing the time zone. Similarly for some of the other "reentrant" functions mentioned in that draft. If WG14 wants reentrant, it should standardize localtime_rz etc. a la tzcode and NetBSD. These functions avoid the races inherent to localtime_r etc. I'll cc. this to Jens Gustedt to give the draft's author a heads-up. Even if WG14 decides to stick with POSIXish rather than reentrant functions, the rationale document should explain why.
Hello Paul, On Sat, 17 Aug 2019 17:38:07 -0700 Paul Eggert <eggert@cs.ucla.edu> wrote:
Brian Inglis wrote:
WG14/C is looking at:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2402.pdf
which seems somewhat based on the POSIX standard.
Thanks for the heads-up. I see that this WG14/C draft is calling localtime_r "reentrant", but it's not: in POSIX, localtime_r relies on global state to obtain the time zone, and behavior is undefined if one thread calls localtime_r while another thread is changing the time zone. Similarly for some of the other "reentrant" functions mentioned in that draft.
Right, I probably was much too optimistic :(
If WG14 wants reentrant, it should standardize localtime_rz etc. a la tzcode and NetBSD. These functions avoid the races inherent to localtime_r etc.
No, I guess that MT-safe with the specific conditions for locale and perhaps env should be enough. If I see this correctly, C itself has currently no means to change the TZ nor even an environment variable from inside the program run, so all we can do here is just to add some warnings: if the implementation permits changes to global state concerning calendar time or somehow changes the LC_TIME category, there will be trouble in a multi-threaded program execution. Seeing that, we should add some phrase to the C standard concerning strftime, too.
I'll cc. this to Jens Gustedt to give the draft's author a heads-up.
I appreciate, thanks.
Even if WG14 decides to stick with POSIXish rather than reentrant functions, the rationale document should explain why.
Locale and the interaction with threads is a can of worms, so I'd probably not open that. Thanks Jens -- :: INRIA Nancy Grand Est ::: Camus ::::::: ICube/ICPS ::: :: ::::::::::::::: office Strasbourg : +33 368854536 :: :: :::::::::::::::::::::: gsm France : +33 651400183 :: :: ::::::::::::::: gsm international : +49 15737185122 :: :: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::
On 8/18/19 12:29 PM, Jens Gustedt wrote:
Locale and the interaction with threads is a can of worms, so I'd probably not open that.
Yes, thanks, that all sounds reasonable. I hope that WG14/C will produce a rationale document explaining this sort of thing.
On 2019-08-17 18:38, Paul Eggert wrote:
Brian Inglis wrote:
WG14/C is looking at: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2402.pdf which seems somewhat based on the POSIX standard.
Thanks for the heads-up. I see that this WG14/C draft is calling localtime_r "reentrant", but it's not: in POSIX, localtime_r relies on global state to obtain the time zone, and behavior is undefined if one thread calls localtime_r while another thread is changing the time zone. Similarly for some of the other "reentrant" functions mentioned in that draft.
If WG14 wants reentrant, it should standardize localtime_rz etc. a la tzcode and NetBSD. These functions avoid the races inherent to localtime_r etc. I'll cc. this to Jens Gustedt to give the draft's author a heads-up. Even if WG14 decides to stick with POSIXish rather than reentrant functions, the rationale document should explain why.
To me and probably most, *thread safe* also implies ability to safely call any other function, including changing the time zone per thread: either constraint, for support, or its lack, of different time zones (or anything else relevant) per thread, should be explicitly specified and referenced in any "thread safe" interface definition, or possibly punted to have the implementation so specify. [Despite the mooted "unpopularity" of C with developers, it's still not looking like anything could replace it for core libraries, utilities, and systems, in the next few decades, so adding robustness to the specifications should be a focus, in the ways suggested below. Adding bounds checking interfaces a la Annex K.3.8 with rsize_t AKA size_t, and errno_t ...time_s() functions, which I have not seen mentioned in any time.h I have come across since 2011 (it appears MS had something like this around 2005), may be more useful and successful if it were limited to only adding constraints to widely existing implementations like ...time_rz() or strftime_l(). If innovating interfaces, such as e.g. timespec_get(), using POSIX struct timespec, rather than POSIX clock_gettime(), add value with generality and longevity (as with GNU libraries and utilities), not just a different interface to existing function e.g. extend and fix struct time {time_t t_s; time_as t_as;} where time_as is in attoseconds ~1e18/s ~2^60/s, similar to previous extensions (unless there are reasons to prefer the NTP datestamp-like seconds fraction with the MSB .5s and the LSB ~0.05as), with prototypes like time_get( struct time t [static 1], enum time_base base), and add things like tm_as to struct tm (time_t tm_sec would also be welcome), with some new %fw.pX strftime format, and/or e.g. %fw.pS, %fw.ps. For features, remove months and maybe also years from yyyymmL dates that nobody remembers, specify simple numbers for versions like __STDC_VERSION_TIME_H__ 2 or provide simple equivalents enumerating the versions for tests e.g. __STDC_VERSION_TIME_H_2__. Add symbols for all the magic numbers still littering function specifications, including parameter and return values like 0/-1, as their meanings are very context dependent: e.g. _TIME_SUCCESS and _TIME_FAILURE, or if necessary _LOCALTIME_SUCCESS and _LOCALTIME_FAILURE, ..., etc. These should all be pre-specified in the library feature interface headers, rather than being hard coded, enumerated or defined by ourselves in company, project, site, group, or developer local headers wrapping uses of the standard headers.] -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.
Brian Inglis wrote:
The POSIX Austin CSRG is discussing clarifying tm_isdst:
https://www.mail-archive.com/search?q=time&l=austin-group-l%40opengroup.org
That issue was resolved last month as "Accepted as Marked". See: http://austingroupbugs.net/view.php?id=1253 This was fallout from our June discussion, e.g.: https://mm.icann.org/pipermail/tz/2019-June/028043.html and the Austin Group interpretation was the expected one, e.g., tm_isdt > 0 corresponds to the "GMT" part of the TZ setting "IST-1GMT0,M10.5.0,M3.5.0/1" even though its UTC offset is less than that of the "IST" part.
On Sat, Aug 17, 2019 at 4:45 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
+However, the leap second support of this package is rarely used directly +because POSIX requires leap seconds to be ignored
For what it's worth, I never like the term "ignored" in this context. For example, I might claim that POSIX requires leap seconds to be taken into account (in order to remove them from the "seconds-since-the-epoch" count), and that it is the "right" clock that ignores them. Conversely, others might say leap seconds are ignored in the POSIX absolute<->civil conversions, and taken into account in the "right" conversions. So, "ignored" doesn't really apply to either system ... they both deal with leap seconds in some fashion.
Steve Summit wrote:
I was trying to point someone to a reference describing how the "right" zones can be used to enable a system to handle leap seconds more correctly, ...
Humpf! I notice the current MacOS Mojave doesn't install the "right" subtree. I paxed it in from my desktop Linux guest and it seems to work, so the code is there but the data are lacking. -- gil
On Aug 18, 2019, at 1:02 PM, Paul Gilmartin via tz <tz@iana.org> wrote:
Steve Summit wrote:
I was trying to point someone to a reference describing how the "right" zones can be used to enable a system to handle leap seconds more correctly, ...
Humpf! I notice the current MacOS Mojave doesn't install the "right" subtree. I paxed it in from my desktop Linux guest and it seems to work, so the code is there but the data are lacking.
The macOS Mojave clocks don’t report UTC, they report Unix Time (don’t count leap seconds). So it would be incorrect for them to use the “right” subtree. That being said, I am working with them in the hopes of getting them to publis the IANA leapseconds information in _some_ form (the “right” subtree being one of those possible forms, the leapseconds source file being another). Howard
Howard Hinnant wrote:
I am working with them in the hopes of getting them to publis the IANA leapseconds information in_some_ form (the “right” subtree being one of those possible forms, the leapseconds source file being another).
I suggest the latter, as it's a lot smaller and less likely to go wrong. Red Hat distributes /usr/share/zoneinfo/leapseconds, whereas Ubuntu distributes /usr/share/zoneinfo/leap-seconds.list. These have the same leap-second info but use different formats; leapseconds is input suitable for zic, whereas leap-seconds.list is the format that the NIST uses. I don't know why GNU/Linux distributions differ in this area. I hope Apple chooses one format or the other rather than going their own way.
On Aug 18, 2019, at 1:36 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
Howard Hinnant wrote:
I am working with them in the hopes of getting them to publis the IANA leapseconds information in_some_ form (the “right” subtree being one of those possible forms, the leapseconds source file being another).
I suggest the latter, as it's a lot smaller and less likely to go wrong.
That’s my hope as well.
Red Hat distributes /usr/share/zoneinfo/leapseconds, whereas Ubuntu distributes /usr/share/zoneinfo/leap-seconds.list.
Thanks, I didn’t know this.
These have the same leap-second info but use different formats; leapseconds is input suitable for zic, whereas leap-seconds.list is the format that the NIST uses. I don't know why GNU/Linux distributions differ in this area. I hope Apple chooses one format or the other rather than going their own way.
<fingers crossed!> Howard
BTW, Windows has recently added support for handling, rather than hiding, leap seconds: https://techcommunity.microsoft.com/t5/Networking-Blog/Leap-Seconds-for-the-... An application must opt into time-with-leap-seconds with a SetProcessInformation() call. That article claims that 1) routines returning FILETIME values (somewhat equivalent to time_t/struct timeval/etc.) are not affected by that setting; 2) routines returning SYSTEMTIME (somewhat equivalent to struct tm) are affected, and can return 60 as the number of seconds. I don't know what Windows did with FILETIME values before the recent change; did they count 100 nanosecond intervals, with the FILETIME value incrementing by 1 every 100 nanoseconds *regardless* of whether a leap second was occurring or not, or did they do POSIX-style "pause the clock" or "slow down the clock" or... stuff to arrange that, from YYYY-DD-MM 23:59:59 to YYYY-DD-MM 23:59:60, the FILETIME value doesn't increment (or resets after 100,000,000 nanoseconds so that the leap second isn't counted)? I'm *guessing* that, after the change, the FILETIME value increases by 1 every 100 nanoseconds, regardless of whether a leap second is occurring or not.
participants (8)
-
Bradley White -
Brian Inglis -
Guy Harris -
Howard Hinnant -
Jens Gustedt -
Paul Eggert -
Paul Gilmartin -
scs＠eskimo.com