If leap seconds go away, should TZif leap-second tables expire?

I have a question about leap-second table expiration in TZif files. Please bear with a longish explanation first; the actual question is at the end of this message. Since release 2020a, TZDB's zic command has supported "Expires" lines in the leapseconds file, and has propagated that info to TZif files using an extension to Internet RFC 8536. This extension is a bit of a hack: if a TZif file's last leap-second record has the same correction as the penultimate record, then the last record is the leap second expiry. For example, if the file 'leap-seconds.list' contains the line: #@ 3928521600 which stands for 28 June 2024; and if you run the command 'make EXPIRES_LINE=1 leapseconds', the file 'leapseconds' will contain: Expires 2024 Jun 28 00:00:00 and if you then run 'make install', the resulting TZif files with leap seconds will contain two leap transitions with correction 27, one transition for 1483228826 (representing 2016-12-31 23:59:60 UTC, the most recent leap second), and one transition for 1719532827 (representing 2017-06-28 00:00:00 UTC, the leap second table expiration). This Expires-line feature was introduced to TZDB in release 2020a, replacing an older hack using an "#expires" comment. However, in the interest of stability the feature has been disabled by default and you must use a non-default setting like "make EXPIRES_LINE=1" to enable it. Our current draft -09 revision to Internet RFC 8536 documents this new feature for the planned version 4 of the TZif file. See <https://datatracker.ietf.org/doc/draft-murchison-rfc8536bis/09/> section 3.2 item "corr(ection)", which says: "... in version 4 files with two or more leap-second records, the correction value of the last two records MAY be the same, with the occurrence of last record indicating the expiration time of the leap-second table." "The leap-second table expiration time is the time at which the table no longer records the presence or absence of future leap- second corrections, and post-expiration timestamps can not be accurately calculated. For example, a leap-second table published in January, which predicts the presence or absence of a leap second at June's end, might expire in mid-December because it is not known whether a leap second will occur at December's end. To complicate matters, a year ago the General Conference on Weights and Measures adopted a resolution calling for leap seconds to be phased out by 2035 <https://www.bipm.org/en/cgpm-2022/resolution-4>. If this is indeed done as late as possible there's a chance we could have our first-ever negative leap second before 2035 <https://insidegnss.com/will-we-have-a-negative-leap-second/> which could cause software havoc, and it's not clear whether the authorities would go through with this. One more thing: the World Radio Conference is currently meeting in Dubai and this meeting may take further action regarding leap seconds. Draft -09 currently has the following text about the possible impending demise of leap seconds: "If leap seconds become permanently discontinued, as requested by the General Conference on Weights and Measures [CGPM-2022-R4], leap-second tables published after the discontinuation time SHOULD NOT expire, since they will not be updated in the foreseeable future." With all the above in mind, here is my question. Should TZif leap-second tables have no expiration date if leap seconds are discontinued, as draft -09 suggests? Or should they have an expiration date of (say) January 2100, which is likely a lower bound on the date that the leap-second regime would be replaced by some other regime, such as leap minutes? Or should we do something else about TZif leap-second expiry after leap seconds are discontinued?

I believe that the proposed text regarding it not expiring is what would make most sense. It is forward compatible with any option, after all. Jacob Pratt On Wed, Nov 29, 2023 at 2:32 AM Paul Eggert via tz <tz@iana.org> wrote:
I have a question about leap-second table expiration in TZif files. Please bear with a longish explanation first; the actual question is at the end of this message.
Since release 2020a, TZDB's zic command has supported "Expires" lines in the leapseconds file, and has propagated that info to TZif files using an extension to Internet RFC 8536. This extension is a bit of a hack: if a TZif file's last leap-second record has the same correction as the penultimate record, then the last record is the leap second expiry. For example, if the file 'leap-seconds.list' contains the line:
#@ 3928521600
which stands for 28 June 2024; and if you run the command 'make EXPIRES_LINE=1 leapseconds', the file 'leapseconds' will contain:
Expires 2024 Jun 28 00:00:00
and if you then run 'make install', the resulting TZif files with leap seconds will contain two leap transitions with correction 27, one transition for 1483228826 (representing 2016-12-31 23:59:60 UTC, the most recent leap second), and one transition for 1719532827 (representing 2017-06-28 00:00:00 UTC, the leap second table expiration).
This Expires-line feature was introduced to TZDB in release 2020a, replacing an older hack using an "#expires" comment. However, in the interest of stability the feature has been disabled by default and you must use a non-default setting like "make EXPIRES_LINE=1" to enable it.
Our current draft -09 revision to Internet RFC 8536 documents this new feature for the planned version 4 of the TZif file. See <https://datatracker.ietf.org/doc/draft-murchison-rfc8536bis/09/> section 3.2 item "corr(ection)", which says:
"... in version 4 files with two or more leap-second records, the correction value of the last two records MAY be the same, with the occurrence of last record indicating the expiration time of the leap-second table."
"The leap-second table expiration time is the time at which the table no longer records the presence or absence of future leap- second corrections, and post-expiration timestamps can not be accurately calculated. For example, a leap-second table published in January, which predicts the presence or absence of a leap second at June's end, might expire in mid-December because it is not known whether a leap second will occur at December's end.
To complicate matters, a year ago the General Conference on Weights and Measures adopted a resolution calling for leap seconds to be phased out by 2035 <https://www.bipm.org/en/cgpm-2022/resolution-4>. If this is indeed done as late as possible there's a chance we could have our first-ever negative leap second before 2035 <https://insidegnss.com/will-we-have-a-negative-leap-second/> which could cause software havoc, and it's not clear whether the authorities would go through with this. One more thing: the World Radio Conference is currently meeting in Dubai and this meeting may take further action regarding leap seconds.
Draft -09 currently has the following text about the possible impending demise of leap seconds:
"If leap seconds become permanently discontinued, as requested by the General Conference on Weights and Measures [CGPM-2022-R4], leap-second tables published after the discontinuation time SHOULD NOT expire, since they will not be updated in the foreseeable future."
With all the above in mind, here is my question.
Should TZif leap-second tables have no expiration date if leap seconds are discontinued, as draft -09 suggests? Or should they have an expiration date of (say) January 2100, which is likely a lower bound on the date that the leap-second regime would be replaced by some other regime, such as leap minutes? Or should we do something else about TZif leap-second expiry after leap seconds are discontinued?

Date: Tue, 28 Nov 2023 23:31:18 -0800 From: Paul Eggert via tz <tz@iana.org> Message-ID: <b1216f8c-8fc6-433d-b4da-c8a3f47db402@cs.ucla.edu> | Should TZif leap-second tables have no expiration date if leap seconds | are discontinued, as draft -09 suggests? I don't think TZif files should have a leap second expiration date at all, nor should there be any kind of transition at that point. I have always interpreted the expiration date in the bulletins as applying to the bulletins themselves, not to the leap seconds they advise about. That's why it makes no sense to continus having an expiration date in the bulletin if leap seconds are discontinued. Otherwise a new bulletin would still need to be issued every 6 months, all saying that no leap seconds were being added or deleted during the validity of the bulletin. The expiration date that currently exists allows readers to determine if the bulletin they're reading contains valid data, or if it is obsolete, and they should go get a newer one - that's all. Future data in TZif files is always just a guess, it might make some sense to have data in their indicatinh the time up until when we believe the data is reliable (more or less the publication date - lder data might have errors, but those are errors, ie": bugs, rather than unavoidable). That might help people determine if they want to go look for newer data, but unless you want to commit to always issuing a new release periodically. even if nothing has changed, it cannot really be an expiration date, just a hint. What do you believe this field in the TZif files is useful for now? kre

On 2023-11-29 01:16, Robert Elz wrote:
What do you believe this field in the TZif files is useful for now?
I don't use leap second expiry myself, so I'm not the best person to ask - which is why I posted the question. However, here's a brief summary of what I know. As you may recall, a few years ago the tz list was periodically asked by downstream users to update the leap-seconds.list file, even when that file's list of leap seconds did not change. This was because leap-seconds.list contains a special comment line like "#@ 3928521600" containing an expiration time, and NTPD rejects a leap-seconds.list file with expiration time in the past. See, for example, Martin Burnicki's 2015 email here: https://mm.icann.org/pipermail/tz/2015-December/023032.html We haven't been getting reminders like this recently. I expect this is because NTPD has declined in popularity. Although the original NTPD is no longer maintained (see Nate Hopper's New Yorker piece a year ago <https://www.newyorker.com/tech/annals-of-technology/the-thorny-problem-of-ke...>), NTPsec <https://www.ntpsec.org/> is still alive and kicking and I think it still reads leap-seconds.list and looks at the expiration. However, I think many systems that formerly would have used NTPD are now using Chrony <https://chrony-project.org/>. Chrony does not currently read leap-seconds.list directly; instead, it gets leap second information by setting TZ="right/UTC", which causes most Linux implementations to consult /usr/share/zoneinfo/right/UTC, which is a TZif file with leap seconds. Chrony then uses localtime and mktime to infer leap seconds. Because localtime/mktime ignore leap second expiry, Chrony cannot deduce the leap second expiration time of the TZif file even if the TZif file contains this info. Coincidentally, yesterday Patrick Oppenlander proposed adding a leap-seconds.list parser to Chrony <https://www.mail-archive.com/chrony-dev@chrony.tuxfamily.org/msg02696.html>. Although Oppenlander's patch does not support leap second expiry, it might be just a matter of time before it adds support for that too. Internet RFC 8536 specifies a provision for truncating TZif files so that timestamps after time T are not supported; this is partly to save space when transmitting them, and partly I assume for the same reason that leap-seconds.list has an expiration date - some TZif distributors want to ensure their users get periodic updates. However, after RFC 8536 came out we discovered that this truncation provision in some sense collides with leap second expiry: should a truncated table mean that leap seconds expire too? or vice versa? That sort of thing. This is partly why I asked the question. Given the above, I plan to ask the chrony folks about this issue. If they don't care about leap second expiry in TZif files it's likely nobody else cares either.

Date: Wed, 29 Nov 2023 11:28:00 -0800 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <2cfab05c-3580-4172-a412-525c43f75298@cs.ucla.edu> | As you may recall, a few years ago the tz list was periodically asked by | downstream users to update the leap-seconds.list file, even when that | file's list of leap seconds did not change. Sure, while we're distributing the leap seconds raw data, that should be complete (even if in a different format from the bulletin - distilling the info from many of them into one place), that's fine. Keeping the expiration date there isn't an issue - as long as the bulletins retain it. When (if) they stop publishing an expiration date (which probably means no more regular bulletins) then it should be removed from the files that are distributed with tzdata. | We haven't been getting reminders like this recently. I expect this is | because NTPD has declined in popularity. That, or that recently those files have been being regularly updated. If there's no need for a new tzdata distribution soon though, I think you might see requests again, as the ones currently in 2023c are due to expire in a little over a month. | Chrony then uses localtime and mktime to infer leap seconds. That's kind of bizarre, but whatever they feel is best. | Although Oppenlander's patch does not support leap second expiry, it | might be just a matter of time before it adds support for that too. As long as it is getting it from the leap-seconds file (one of the variants) that's fine. | Internet RFC 8536 specifies a provision for truncating TZif files so | that timestamps after time T are not supported; this is partly to save | space when transmitting them, and partly I assume for the same reason | that leap-seconds.list has an expiration date - some TZif distributors | want to ensure their users get periodic updates. Perhaps - though anything which fails to convert (as accurately as is believed possible) any time_t which can be represented in a struct tm wouldn't be POSIX conformant. Just stopping at some arbitrary time isn't permitted. Obviously everything about the future is just best guess. Whether users feel the need for updates or not (ideally of course, that would simply happen - at least for net connected systems) is ultimately their choice. | However, after RFC 8536 | came out we discovered that this truncation provision in some sense | collides with leap second expiry: should a truncated table mean that | leap seconds expire too? or vice versa? If localtime() were really refusing to convert times, then who cares? If not, then there is no real expiration date in the file, just a lack of future data, for when it matters. kre

On 11/29/23 14:07, Robert Elz wrote:
anything which fails to convert (as accurately as is believed possible) any time_t which can be represented in a struct tm wouldn't be POSIX conformant.
Such an implementation would conform to current POSIX, because the only way to get the failing behavior is with a TZ setting like TZ='/name/of/truncated/TZ/file' that is not POSIX-conforming, and with such a TZ setting the implementation can do what it likes. POSIX 202x/D3 tightens this up, though, as it requires support for geographical and special timezones like TZ='Europe/London'. The details of this are a bit hazy, unfortunately, and I suppose an implementer determined to make localtime fail on timestamps outside a truncated TZif file's range might be able to argue that POSIX allows that. Anyway as a practical matter I agree with you. Regardless of what POSIX says, having localtime fail merely because a TZif file is truncated would break so many applications that it's not a plausible implementation.

Robert Elz via tz wrote in <3221.1701295665@jacaranda.noi.kre.to>: | Date: Wed, 29 Nov 2023 11:28:00 -0800 | From: Paul Eggert <eggert@cs.ucla.edu> | Message-ID: <2cfab05c-3580-4172-a412-525c43f75298@cs.ucla.edu> | || As you may recall, a few years ago the tz list was periodically asked by || downstream users to update the leap-seconds.list file, even when that || file's list of leap seconds did not change. ... || Chrony then uses localtime and mktime to infer leap seconds. | |That's kind of bizarre, but whatever they feel is best. They use mktime and gmtime: tm = gmtime(&when); if (!tm) return tz_leap; stm = *tm; /* Temporarily switch to the timezone containing leap seconds */ tz_env = getenv("TZ"); if (tz_env) { if (strlen(tz_env) >= sizeof (tz_orig)) return tz_leap; strcpy(tz_orig, tz_env); } setenv("TZ", leap_tzname, 1); tzset(); /* Get the TAI-UTC offset, which started at the epoch at 10 seconds */ t = mktime(&stm); if (t != -1) tz_tai_offset = t - when + 10; /* Set the time to 23:59:60 and see how it overflows in mktime() */ stm.tm_sec = 60; stm.tm_min = 59; stm.tm_hour = 23; t = mktime(&stm); if (tz_env) setenv("TZ", tz_orig, 1); else unsetenv("TZ"); tzset(); if (t == -1) return tz_leap; if (stm.tm_sec == 60) tz_leap = LEAP_InsertSecond; else if (stm.tm_sec == 1) tz_leap = LEAP_DeleteSecond; *tai_offset = tz_tai_offset; Wouldn't it have been nice if the world would have gone on distributing TAI and simply the offset to UTC, instead of anything else. (And an indicator, smart people outperform me with ideas there.) Btw chrony says /* Allow leap second only on the last day of June and December */ and i am not sure this is correct. I noted for this response that ISO C 23 has no match for TZ nor timezone. Nor putenv / setenv / unsetenv. ... || However, after RFC 8536 || came out we discovered that this truncation provision in some sense || collides with leap second expiry: should a truncated table mean that || leap seconds expire too? or vice versa? | |If localtime() were really refusing to convert times, then who cares? |If not, then there is no real expiration date in the file, just a lack |of future data, for when it matters. I think bets can be taken whether they do the switch before the negative leap, shall it occur, or not. On another list i said (after that decision) all the troubles will happen and then it is thrown apart saying "good it is gone", .. instead of trying to do something to make working with it smooth, for which several decades of opportunities exist(ed). I have negative feelings with it being thrown off, "i said", as mankind went for thousands of years to be able to get it right, and it was magic, it was a miracle, it was a shining laugther, and now we can, intellectually penetrated even, like our friend the Californian astronomer (also in this list) with all his bells and whistles (it is a science), and throw it away because of false business decisions. (I think nothing should be done. For dates 1972-X leap second information will hold forever, unless you blur it off.) --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)

On 2023-11-29 12:28, Paul Eggert via tz wrote:
On 2023-11-29 01:16, Robert Elz wrote:
What do you believe this field in the TZif files is useful for now?
I don't use leap second expiry myself, so I'm not the best person to ask - which is why I posted the question. However, here's a brief summary of what I know.
As you may recall, a few years ago the tz list was periodically asked by downstream users to update the leap-seconds.list file, even when that file's list of leap seconds did not change. This was because leap-seconds.list contains a special comment line like "#@ 3928521600" containing an expiration time, and NTPD rejects a leap-seconds.list file with expiration time in the past. See, for example, Martin Burnicki's 2015 email here:
https://mm.icann.org/pipermail/tz/2015-December/023032.html
We haven't been getting reminders like this recently. I expect this is because NTPD has declined in popularity. Although the original NTPD is no longer maintained (see Nate Hopper's New Yorker piece a year ago <https://www.newyorker.com/tech/annals-of-technology/the-thorny-problem-of-ke...>), NTPsec <https://www.ntpsec.org/> is still alive and kicking and I think it still reads leap-seconds.list and looks at the expiration. However, I think many systems that formerly would have used NTPD are now using Chrony <https://chrony-project.org/>.
Coincidentally, yesterday Patrick Oppenlander proposed adding a leap-seconds.list parser to Chrony <https://www.mail-archive.com/chrony-dev@chrony.tuxfamily.org/msg02696.html>. Although Oppenlander's patch does not support leap second expiry, it might be just a matter of time before it adds support for that too.
Given the above, I plan to ask the chrony folks about this issue. If they don't care about leap second expiry in TZif files it's likely nobody else cares either.
The leap-seconds.list expiration date specifies the limit of when the current delta TAI-UTC and stepless UTC period is *KNOWN* to be valid, for those systems and processes which rely on knowing some time scale with certainty and accuracy within a fraction of a second (NTP < 128ms, normally ~ 1us). After the expiration date, the uncertainty is >1s! IERS now reports and updates this consistently and correctly and specifies {Jun,Dec}-28 following the next leap second update effective date e.g. Bulletin C 66 2023-07-04 00:00Z NTP 3897417600 announces no leap second 2023-12-31 24:00Z NTP 3913081200 and that is valid until 2024-06-24 00:00Z NTP 3928521600 and those values are specified in leap-seconds.list. There have been times since 2017 when NIST has not published updates soon enough and time keepers have made their own, or switched to IERS which is definitive. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry

On Tue 2023-11-28T23:31:18-0800 Paul Eggert via tz hath writ:
Should TZif leap-second tables have no expiration date if leap seconds are discontinued, as draft -09 suggests?
That seems best. Asserting a specific date like 2100 is inconsistent with the inability to predict changes in every other aspect of time zone information. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m

On 2023-11-29 10:15, Steve Allen via tz wrote:
On Tue 2023-11-28T23:31:18-0800 Paul Eggert via tz hath writ:
Should TZif leap-second tables have no expiration date if leap seconds are discontinued, as draft -09 suggests?
That seems best.
Asserting a specific date like 2100 is inconsistent with the inability to predict changes in every other aspect of time zone information.
And the second is expected to be redefined in CODATA 2030 or earlier publication using better frequency standards than caesium - possibly strontium. [Reminds me - CODATA 2022 - should it not be out by now?] -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry
participants (6)
-
Brian Inglis
-
Jacob Pratt
-
Paul Eggert
-
Robert Elz
-
Steffen Nurpmeso
-
Steve Allen