New Yorker article on David Mills and NTP
Readers of this list may be interested in this article: "The Thorny Problem of Keeping the Internet's Time" https://www.newyorker.com/tech/annals-of-technology/the-thorny-problem-of-ke... There are a few bobbles: the author seems a bit confused over whether NTP is an Internet RFC or a piece of software, and whether NTP is the IETF's only concern. And the New Yorker's predilection for diereses in English is rather comically distracting when ritually applied to the phrase "Coordinated Universal Time". Nevertheless, a nice read, covering both the technical issues and the people involved, with a particularly touching portrait of Mills himself.
<<On Mon, 03 Oct 2022 09:43:19 -0400, Steve Summit via tz <tz@iana.org> said:
There are a few bobbles: the author seems a bit confused over whether NTP is an Internet RFC or a piece of software,
In many ways, that is not all that different from the confusion we experience on this list from time to time between the Reference Implementation (tzcode) and other libraries which parse either TZif or the tzdata source files, and the issues that arise when those other implementations don't behave quite the same way as the Reference Implementation does -- particularly when they expose (and users come to depend on) what the maintainers consider "implementation details". And of course both communities end up having to paper over the mistake that the POSIX committee made with regard to leap seconds, but do it in different, not entirely compatible ways. -GAWollman
On Mon 2022-10-03T15:45:59-0400 Garrett Wollman via tz hath writ:
And of course both communities end up having to paper over the mistake that the POSIX committee made with regard to leap seconds, but do it in different, not entirely compatible ways.
The POSIX committee did not make a mistake about leap seconds. They had no choice. Doing anything else would have required the existence of an authority who had already created a robust scheme for communicating the list of past and future leap seconds. No such authority was ever funded, and that was because the leap second itself resulted from earlier instances of failure to communicate. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m
On 3 Oct 2022, at 20:58, Steve Allen via tz <tz@iana.org> wrote:
On Mon 2022-10-03T15:45:59-0400 Garrett Wollman via tz hath writ:
And of course both communities end up having to paper over the mistake that the POSIX committee made with regard to leap seconds, but do it in different, not entirely compatible ways.
The POSIX committee did not make a mistake about leap seconds. They had no choice. Doing anything else would have required the existence of an authority who had already created a robust scheme for communicating the list of past and future leap seconds. No such authority was ever funded, and that was because the leap second itself resulted from earlier instances of failure to communicate.
I think the problem goes back further than that. Back in the early 70s there were 86400 seconds in a day, every day. It was only with the advent of leap seconds that some days has 86401 or 86499 seconds. The trouble is, programs get to this time tomorrow by adding 86400 to the current time: that constant is implicit (or explicit) in so many places fixing that would have been well-nigh impossible and add significant complexity to even the most mundane shell script. So Posix says "there are 86400 seconds in a day"[1] and the epoch, time zero, isn't exactly the dawn of the 1970s[2]. The enormously simplifies all time calculations at some minor cost of annoying astronomers :) and making elapsed time and time-of-day different: sleep(86400) isn't sleep until this time tomorrow and using gettimeofday() to measure elapsed time sometimes returns a negative number[3]. jch [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html, section 4.16 [2] not even in the UK because we stayed on BST that winter. [3] this did cause some programs to crash, spectacularly.
-- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m
On Tue 2022-10-04T09:27:32+0000 John Haxby hath writ:
Back in the early 70s there were 86400 seconds in a day, every day. It was only with the advent of leap seconds that some days has 86401 or 86499 seconds.
A clock that is never reset does not correspond to anything that humans can observe. At no point in history has there been a source of official time that does not experience resets of its clock. When the clock on the wall in my house is wrong and I reset it the foundation does not crack and rain does not pour through the roof. The implementation of time in computing systems is fragile because it is based on an unrealistic notion that clocks are always right and never need to be reset. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m
On Oct 4, 2022, at 12:07, Steve Allen via tz <tz@iana.org> wrote:
The implementation of time in computing systems is fragile because it is based on an unrealistic notion that clocks are always right and never need to be reset.
I suspect that one of the primary reasons that such time resets have been so fraught (and hence bitterly opposed by many ‘operations’ type folks) is their relative rarity. There have been only 27 leap seconds introduced since 1972, which means that deployed systems rarely have to cope with one and thus get the inevitable bugs identified and shaken out. It would almost be beneficial if such an event would occur at least once per annum, to allow the ‘reset’ logic to be regularly exercised. Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | There cannot be a crisis next week. My schedule is already full. | | | | -- Henry Kissinger | |---------------------------------------------------------------------|
Fred Gleason wrote:
I suspect that one of the primary reasons that such time resets have been so fraught [...] is their relative rarity [...] deployed systems rarely have to cope with one and thus get the inevitable bugs identified and shaken out.
It would almost be beneficial if such an event would occur at least once per annum, to allow the 'reset' logic to be regularly exercised.
A few years back I did a bunch of work (sadly as yet unpublished) trying to improve support for leap seconds, and one of the things I wrote was a mock NTP server serving time on hundreds of different test ports, with each port offering fake leap seconds on a different day. So, theoretically, if you wanted to test your system's leapsecond handling, all you'd have to do was pick the right port and you could have your leap second tomorrow. (Unfortunately it wouldn't have been nearly as convenient to use as I had anticipated, because it turns out that stock ntpd doesn't offer an easy way to configure an upstream server connection on other than the default port 123.)
On 10/4/22 11:05:49, Fred Gleason via tz wrote:
On Oct 4, 2022, at 12:07, Steve Allen wrote:
The implementation of time in computing systems is fragile because it is based on an unrealistic notion that clocks are always right and never need to be reset.
... There have been only 27 leap seconds introduced since 1972, which means that deployed systems rarely have to cope with one and thus get the inevitable bugs identified and shaken out.
It would almost be beneficial if such an event would occur at least once per annum, to allow the ‘reset’ logic to be regularly exercised.
So UTC might run 1e-7 slower than the TAI rate, impelling about 3 "negative" leap seconds (the innocuous kind) annually to test the software. There are 3 desiderata: o Atomic accuracy (UT1 violates this) o Agreement with solar time (TAI violates this) o Freedom from discontinuities (UTC violates this) Pick your two favorites. -- gil
Paul Gilmartin via tz said:
There are 3 desiderata: o Atomic accuracy (UT1 violates this) o Agreement with solar time (TAI violates this) o Freedom from discontinuities (UTC violates this)
"Quick, cheap, works - pick any two" -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On 2022-10-04 16:07, Steve Allen via tz wrote:
A clock that is never reset does not correspond to anything that humans can observe. At no point in history has there been a source of official time that does not experience resets of its clock.
Well, yes, if you take local civil time scales, or UTC, as "official time" -- these times scales are *defined* to have discontinuous steps (eg, at switches between winter and summer time and at leap seconds). So any clock strictly following one of these time scales must have discontinuities. Clocks following continuous time scales (all others: TAI, TCB, TCG, TDT, TT and the various solar times) can stay (and several have stayed in the past) close to these time scales without any discontinuous steps. And the clock on my laptop is even able to follow UTC (less strictly) without experiencing any "resets" (only slight changes in rate). The upcoming redefinition of UTC will stop its discontinuities for at least the next 100 years, so clocks may follow UTC more closely without any resets. Michael Deckers.
Date: Tue, 4 Oct 2022 09:07:26 -0700 From: Steve Allen via tz <tz@iana.org> Message-ID: <20221004160726.GA5446@ucolick.org> | When the clock on the wall in my house is wrong and I reset it the | foundation does not crack and rain does not pour through the roof. No, because neither the foundations, nor the roof, care what the time is in any way at all. You could turn your calendar to an entirely different month, with just the same lack of effect. However, you could have electronics which are affected - your heating, or cooling, might not turn on, or off, as expected (especially if you moved the time from just before the scheduelled event to just after it). Or, perhaps your kids knew that the clock on the wall was wrong (fast), and were relying upon that to get to the bus stop to catch the bus to school, you move the clock backwards by 10 minutes, and they miss the bus. They catch the next, but then they're late to school. This isn't the first time this has happened (your clock isn't all that reliable), and the school decides to suspend your kids for the day. They ring your wife to come and collect them, if no-one comes within the hour, social services will be called. She leaves work in a hurry to make sure she arrives in time. Her bosses don't like her unexplained absence, and terminate her employment, with no good reference (unreliable employee). Without a job, and no immediate chance of getting a new one, your wife is unable to help contribute to paying your house mortgage, and after a while, the bank forecloses. Now you're out of your home, with a bad credit reference, no real chance of even renting anything suitable, the whole family living in your car. All because you reset the clock. kre
On Wed 2022-10-05T04:14:42+0700 Robert Elz hath writ:
Or, perhaps your kids knew that the clock on the wall was wrong (fast), and were relying upon that to get to the bus stop to catch the bus to school, you move the clock backwards by 10 minutes, and they miss the bus. [rest of adventure redacted]
We have constructed a system wherein the components only work as expected with a simplistic notion of time that tells lies to them. -- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m
Steve Allen wrote in <20221004224331.GA26297@ucolick.org>: |On Wed 2022-10-05T04:14:42+0700 Robert Elz hath writ: |> Or, perhaps your kids knew that the clock on the wall was wrong (fast), |> and were relying upon that to get to the bus stop to catch the bus |> to school, you move the clock backwards by 10 minutes, and they miss the |> bus. [rest of adventure redacted] | |We have constructed a system wherein the components only work as |expected with a simplistic notion of time that tells lies to them. You can smear a bit, here and there. To adjust the outcome to the expectation. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
I would add that Posix relies on the c standard, and tillions of devices have only 86400 holes into which only 86400 seconds per day can go. It's infeasible to retrofit an extra hole into those systems to accommodate that 86401th positive leap-second. And some hole will be left empty if there is a negative leap-second. Computer time and UTC with leap-seconds are incommensurate and we're probably just going to have to live with it for quite a while. -Brooks On 2022-10-03 3:58 PM, Steve Allen via tz wrote:
On Mon 2022-10-03T15:45:59-0400 Garrett Wollman via tz hath writ:
And of course both communities end up having to paper over the mistake that the POSIX committee made with regard to leap seconds, but do it in different, not entirely compatible ways. The POSIX committee did not make a mistake about leap seconds. They had no choice. Doing anything else would have required the existence of an authority who had already created a robust scheme for communicating the list of past and future leap seconds. No such authority was ever funded, and that was because the leap second itself resulted from earlier instances of failure to communicate.
-- Steve Allen<sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064https://www.ucolick.org/~sla/ Hgt +250 m
<<On Thu, 6 Oct 2022 15:35:01 -0700, Guy Harris via tz <tz@iana.org> said:
On Oct 6, 2022, at 1:41 PM, Brooks Harris via tz <tz@iana.org> wrote:
I would add that Posix relies on the c standard,
Which appears neither to mandate nor forbid leap seconds.
C (intentionally) does not make any requirements on the encoding of a time_t, other than requiring that it be numeric. tzcode contains lots of complexity to deal with the numerous possible different realizations allowed by the C standard. C allows time_t to be a double. (I haven't looked at recent C standards so I don't know if C allows time_t to be complex or decimal.) The POSIX standard, on the other hand, requires that time_t be an integral type, and gives a closed-form formula for how a UTC time is to be converted to a time_t which explicitly requires every day to be exactly 86,400 seconds long. (POSIX does not include tzcode's timegm() interface; applications are expected to copy the formula from the standard.) The practical upshot of this is that POSIX systems that implement leap seconds must tick the same second twice for a positive leap second, and must skip a tick if there is ever a negative leap second -- either way, making interval calculations inaccurate if performed in "seconds since the Epoch" rather than struct tm. (Some people model the POSIX behavior as "the Epoch" changing every time a leap second is inserted or subtracted, which scarcely improves matters.) The POSIX requirement is annotated as an "extension" to the C standard. -GAWollman
Garrett Wollman wrote in <25407.23180.512283.959616@khavrinen.csail.mit.edu>: |<<On Thu, 6 Oct 2022 15:35:01 -0700, Guy Harris via tz <tz@iana.org> said: |> On Oct 6, 2022, at 1:41 PM, Brooks Harris via tz <tz@iana.org> wrote: |>> I would add that Posix relies on the c standard, | |> Which appears neither to mandate nor forbid leap seconds. ... |The POSIX standard, on the other hand, requires that time_t be an |integral type, and gives a closed-form formula for how a UTC time is |to be converted to a time_t which explicitly requires every day to be |exactly 86,400 seconds long. (POSIX does not include tzcode's |timegm() interface; applications are expected to copy the formula from |the standard.) The practical upshot of this is that POSIX systems |that implement leap seconds must tick the same second twice for a |positive leap second, and must skip a tick if there is ever a negative |leap second -- either way, making interval calculations inaccurate if But only to throw in that for most systems which are not always-on, and even for some which are, this is rather hypothetic since hardware clocks that i know drift (and Linux hwclock(8) documents "The Hardware Clock is usually not very accurate. However, much of its inaccuracy is completely predictable - it gains or loses the same amount of time every day. This is called systematic drift.") To misuse this drift (it is -2.577933 per ~35 hours here without /etc/adjtime) to adjust clocks seems quite natural; to me being able to keep a human clock (civil time) synchronized with the real day length of our planet remains a cultural achievement, even extending the according predictive capabilities of Stonehenge or the Pyramids (in fact many cultures had something similar it seems). I agree it is all wrong and when i really focus on the problem i am always frustrated how bad i code. CLOCK_TAI would be an alternative for programming purposes, and having a nice leap second table at hand is a good thing (on the according ML PH-Kamp came over with a DNS-based solution for leap announcement a few years back, later stating he was just reiterating an idea from over a decade earlier: i jumped on this idea, because i really liked it!) Unfortunately CLOCK_TAI is not available everywhere. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
On 7 Oct 2022, at 00:24, Steffen Nurpmeso via tz <tz@iana.org> wrote:
But only to throw in that for most systems which are not always-on, and even for some which are, this is rather hypothetic since hardware clocks that i know drift (and Linux hwclock(8) documents "The Hardware Clock is usually not very accurate. However, much of its inaccuracy is completely predictable - it gains or loses the same amount of time every day. This is called systematic drift.")
The clock described by hwclock(8) is not the system clock, it's the RTC for Linux/Unix it's only used to set the initial system time on boot. The system clock is the one that has a predictable drift (eg 7ppm on this machine here). The RTC is basically a 2¢ chip and every bit as accurate as you'd expect; my wind-up watch is more accurate. jch
On 10/6/22 15:45, Garrett Wollman via tz wrote:
C (intentionally) does not make any requirements on the encoding of a time_t, other than requiring that it be numeric. tzcode contains lots of complexity to deal with the numerous possible different realizations allowed by the C standard.
Despite its complexity, tzcode no longer supports every possibility allowed by the C standard. I removed support for floating-point time_t in 2013e, with the comment "It wasn't tested and probably never worked, no platform used it, and the latest POSIX no longer allows it."
(I haven't looked at recent C standards so I don't know if C allows time_t to be complex or decimal.)
In C23 time_t cannot be complex. It can be a decimal floating type, which would be problematic as mentioned above.
exactly 86,400 seconds long. (POSIX does not include tzcode's timegm() interface
This will change soon, as C23 will standardize timegm and POSIX will follow suit.
The practical upshot of this is that POSIX systems that implement leap seconds must tick the same second twice for a positive leap second, and must skip a tick if there is ever a negative leap second -- either way, making interval calculations inaccurate if performed in "seconds since the Epoch" rather than struct tm.
Leap smearing is reasonably popular. It also conforms to POSIX, with no duplicate or skipped ticks, and with intervals calculated the same way using time_t vs struct tm. However, it has a different set of problems.
<<On Thu, 6 Oct 2022 18:34:11 -0700, Paul Eggert <eggert@cs.ucla.edu> said:
exactly 86,400 seconds long. (POSIX does not include tzcode's timegm() interface
This will change soon, as C23 will standardize timegm and POSIX will follow suit.
POSIX will follow suit in about 2028, based on prior history. (The current POSIX specification is aligned to C11 and the one we're working on right now is aligned to C17.) -GAWollman
On 2022-10-06 19:34, Paul Eggert via tz wrote:
On 10/6/22 15:45, Garrett Wollman via tz wrote:
C (intentionally) does not make any requirements on the encoding of a time_t, other than requiring that it be numeric. tzcode contains lots of complexity to deal with the numerous possible different realizations allowed by the C standard.
Despite its complexity, tzcode no longer supports every possibility allowed by the C standard. I removed support for floating-point time_t in 2013e, with the comment "It wasn't tested and probably never worked, no platform used it, and the latest POSIX no longer allows it."
AFAIR only C libraries for IBM 360/370/390/z e.g. IBM OS/MVS and VM/CMS SAS/C, used double time_t to handle the IBM ToD clock (later SysPlex timer IIRC) specified in the arch to update bit 52 (BE) every µs, but provided total 64(/128) bits with as much precision as required for discrete timestamps for hardware error logs. Looks like SAS/C support stopped in 2001. [Gig was porting a Windows 16 bit product to work across IBM OSes!] -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]
On Oct 6, 2022, at 3:48 PM, Brooks Harris <brooks@edlmax.com> wrote:
On 2022-10-06 6:35 PM, Guy Harris wrote:
On Oct 6, 2022, at 1:41 PM, Brooks Harris via tz <tz@iana.org> wrote:
I would add that Posix relies on the c standard, Which appears neither to mandate nor forbid leap seconds.
Take your pick. :-)
Which is exactly what POSIX did; they chose "forbid". Some other platform could choose "mandate". Both platforms' implementation of C could conform to the standard.
On Oct 6, 2022, at 1:41 PM, Brooks Harris via tz <tz@iana.org> wrote:
I would add that Posix relies on the c standard, and tillions of devices have only 86400 holes into which only 86400 seconds per day can go. It's infeasible to retrofit an extra hole into those systems to accommodate that 86401th positive leap-second. And some hole will be left empty if there is a negative leap-second. Computer time
"Computer time" meaning what? Time represented as a number of ticks since a fixed epoch, for various definitions of "tick" (which may or may not involve a fixed number of SI seconds per tick, and may not involve every N SI seconds, where N is not necessarily an integer and not necessarily >= 1, being counted)? Time represented as year/month/day/hour/minute/second-possibly-including-fractions-of-a-second? Or some other representation?
and UTC with leap-seconds
Which, as far as I know, is the only type of UTC we have now. (No, POSIX time, handed to localtime(), does not, as far as I know, yield anything corresponding to UTC, so it's not UTC without leap seconds.)
participants (14)
-
Brian Inglis -
Brooks Harris -
Clive D.W. Feather -
Fred Gleason -
Garrett Wollman -
Guy Harris -
John Haxby -
Michael H Deckers -
Paul Eggert -
Paul Gilmartin -
Robert Elz -
scs＠eskimo.com -
Steffen Nurpmeso -
Steve Allen