Re: [tz] leapseconds.awk, LEAP directives and references
In the course of reviewing the AWK script leapseconds.awk, I was disturbed by the fact that it (1) operated on the comments (probably not a good idea) in the NIST produced leap-seconds.list file rather than the NTP timestamp itself and (2) that it assumed that leap seconds would only occur in December and June. The regulation that governs leap seconds, ITU-R TF.460-6 <https://www.itu.int/rec/R-REC-TF.460-6-200202-I/>, actually allows leap seconds to occur at the end of any UTC month. Annex 1, Section 2: Leap Seconds 2.1 A positive or negative leap-second should be the last second of a UTC month, but first preference should be given to the end of December and June, and second preference to the end of March and September. I note, in the event of a (highly unlikely) negative leap second, the LEAP directive generated is for 23:59:59 which, in the case of a negative leap second, doesn't exist. I reviewed the C code and, it seems to me, the transaction added should actually be for the second before, 23:59:58, as the transaction itself causes 23:59:59 to be skippedaltogether. (Kinda hard to skip an event if you're in it.) I bring this up as I've alrady written the changes necessary but I want to verify the correctness of this last beforesubmitting them. Also, I've an AWK script that uses the IERS file Leap_Seconds.dat rather than the NIST leap-seconds.listfile. Leap_Seconds.dat is a straight list of leap-seconds. USNO MAIA also has straight list of leap-seconds in a different format. See URLs: <http://hpiers.obspm.fr/iers/bul/bulc/Leap_Seconds.dat> <http://maia.usno.navy.mil/ser7/leapsec.dat> Either of these would, particularly the IERS file as it has few but excellent comments, be a better source of leap-second information than the once removed NIST file leap-seconds.list; which is purposed for another task. Lastly, thanks to Dr. Eggert for including the reference I gave him to the AAPT Resource Letter that included Terry Quinn's article in the NIST Time and Frequency Division's PDF archive (1168). Anyone wanting to know about Time and Frequency Measurement might want to read the other articles as well as Dr. Quinn's. Another good, comprehensive article is "The leap second: its history and possible future" by R.A. Nelson, et al, from Metrogia Vol 38, pp. 509-529 (2001). PDF available (with permission) at: <https://www.cl.cam.ac.uk/~mgk25/time/metrologia-leapsecond.pdf>
On Tue, Apr 4, 2017 at 12:02 PM, Chris Woodbury via tz <tz@iana.org> wrote:
I note, in the event of a (highly unlikely) negative leap second, the LEAP directive generated is for 23:59:59 which, in the case of a negative leap second, doesn't exist. I reviewed the C code and, it seems to me, the transaction added should actually be for the second before, 23:59:58, as the transaction itself causes 23:59:59 to be skippedaltogether. (Kinda hard to skip an event if you're in it.)
The "Leap" line references the second that should be removed, so 23:59:59 (stationary) is correct.
On Tuesday, April 4, 2017 6:30 PM, Bradley White <bww@acm.org> wrote:
...
The "Leap" line references the second that should be removed, so 23:59:59 (stationary) is correct.
Interesting. Thanks for the information. It will be helpful knowing the correct way to encode a negative leap second. It's not what I'd have intuited. I'm still plowing through all the C code (the ultimate "documentation").
On 04/04/2017 09:02 AM, Chris Woodbury via tz wrote:
I reviewed the C code and, it seems to me, the transaction added should actually be for the second before, 23:59:58, as the transaction itself causes 23:59:59 to be skippedaltogether.
Perhaps Arthur David Olson or Bradley White can take a look at this, as they're the leap second gurus for tz. I've never tested negative leap seconds.
Also, I've an AWK script that uses the IERS file Leap_Seconds.dat rather than the NIST leap-seconds.listfile.
Although that would be a better technical solution, in the past we've shied away from it because the IERS file is copyrighted and we and we and our downstream users don't clearly have permissions to copy and redistribute it.
USNO MAIA also has straight list of leap-seconds in a different format. <http://maia.usno.navy.mil/ser7/leapsec.dat>
Thanks, I was not aware of this file. It is public-domain, so perhaps we could use it. Is it updated in a timely fashion when new IERS bulletins are published? And is that web server highly available? These have been sticking points for the NIST file.
Another good, comprehensive article is "The leap second: its history and possible future" by R.A. Nelson, et al, from Metrogia Vol 38, pp. 509-529 (2001). PDF available (with permission) at:
<https://www.cl.cam.ac.uk/~mgk25/time/metrologia-leapsecond.pdf>
Yes, we link to that in tz-link.htm.
Re: copyrighting and leap source information
On Tuesday, April 4, 2017 6:33 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 04/04/2017 09:02 AM, Chris Woodbury via tz wrote: I reviewed the C code and, it seems to me, the transaction added should actually be for the second before, 23:59:58, as the transaction itself causes 23:59:59 to be skippedaltogether.
Perhaps Arthur David Olson or Bradley White can take a look at this, as they're the leap second gurus for tz. I've never tested negative leap seconds.
Thanks, Bradley White answered directly. (Previous post)
Also, I've an AWK script that uses the IERS file Leap_Seconds.dat
rather than the NIST leap-seconds.list file. Although that would be a better technical solution, in the past we've shied away from it because the IERS file is copyrighted and we and we and our downstream users don't clearly have permissions to copy and redistribute it.
Where it the copyright notice to which you refer above? Yes, IERS has copyright notices on their web pages at IERS.Org. It says: © 2013 - Federal Agency for Cartography and Geodesy. All rights Reserved The information we are discussing is distributed from the Paris Observatory's IERS FTP site and is there to fulfill BIPM's various treaty obligations to signatories (the U.S. being one). I have found no files, anywhere in the <ftp://hpiers.obspm.fr/iers> directory tree, having a copyright notice. I'll call them and ask explicitly if you think it's necessary. Do you need a disclaimer in writing? (I suspect they'll think the treaties are enough.) I no more want to burn my paws over copyrighting than you, Paul. While I've not inquired of my attorney in this exact case; I've gone over copyrighting enough for my own work to have an understanding of the basic requirements. BIPM/IERS/EOC is a world standards AND treaty organization. The *information* it provides HAS to be public domain. (Imagine licensing the meter. They're the same, exact, crew.) The question is really whether their comments are in the public domain. I would humbly suggest that that would require an explicit copyright statement. There simply is none, anywhere.
USNO MAIA also has straight list of leap-seconds in a different format. <http://maia.usno.navy.mil/ser7/leapsec.dat> Thanks, I was not aware of this file. It is public-domain, so perhaps we could use it. Is it updated in a timely fashion when new IERS bulletins are published? And is that web server highly available? These have been
sticking points for the NIST file. The timestamp on the file was the same day as that of the IERS Bulletin C 52.
$ date -u '+%d %b %Y %T %Z' -r bulletinc.52 # The last actual leap sec announcement 06 Jul 2016 10:07:08 UTC $ date -u '+%d %b %Y %T %Z' -r Leap_Second.dat # This is the IERS complete list 06 Jul 2016 11:47:48 UTC $ date -u '+%d %b %Y %T %Z' -r leapsec.dat # This is the USNO MAIA complete list via FTP 06 Jul 2016 12:44:51 UTC (I downloaded these files using "curl -R" specifically to preserve their timestamps.) As for USNO MAIA's infrastructure; I have educated guesses. It's on the same sub-net as Tycho. Both Tycho and MAIA have their Point-of-Presence in Portsmouth, VA (Norfolk area) from my point of view. This USNO sub-net, of which Tycho and MAIA are a part, seems to be made available outside of MILnet through a special portal (DMZ?). They are often available when MILnet is not. However, you've got to have their IP addresses in your "hosts" file to use them when the MILnet DNS servers are unavailable. (The burden of being part of the most attacked network on the planet.) The "curl" RC of 6 (can't resolve host) clued me in to the USNO's DNS servers NOT being available being the cause of my access failures. I will note that I've NEVER had FTP nor HTTP timeouts going to the MAIA site (as I have often had with Tycho) except when MILnet (and their DNS) was unavailable. Nor, for that matter, have I ever had problems getting to hpiers.opspm.fr either. (Other OBSPM sites (ftp.obspm.fr) *do* seem to have a problem with being too busy.) The NIST access problem you've experienced *may* have been somewhat ameliorated recently. The leap-seconds.list file is no longer available via time.nist.gov. It is now at ftp://ftp.nist.gov/pub/time. It doesn't seem to be as busy as the round-robin time.nist.gov hosts were (of which about a third didn't run FTP servers). The leap-seconds.list source, Dr. Lavine's ftp://utcnist{,2}.colorado.edu/pub, are still busy but accessible. The latest leap-seconds.list change was a bit confusing as Dr. Levine changed the expiration timestamp (#$) but not the update time (#@) (which would have changed the name of the file, too). If he had, it would have been leap-seconds.3695414400 (#h 2e31582c a77e52e1 791cba06 76087c21 92b019b2).
...
On 04/06/2017 09:28 PM, Chris Woodbury wrote:
Where it the copyright notice to which you refer above?
I didn't refer to any copyright notice, as there is no such notice for the IERS file <https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list>. So by U.S. law (which now follows the 1988 Berne Convention) the file is copyrighted and cannot be reproduced without permission. See, for example: https://www.copyright.gov/circs/circ03.pdf
On Apr 7, 2017, at 3:22 PM, Paul Eggert <eggert@CS.UCLA.EDU> wrote:
On 04/06/2017 09:28 PM, Chris Woodbury wrote:
Where it the copyright notice to which you refer above?
I didn't refer to any copyright notice, as there is no such notice for the IERS file <https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list>. So by U.S. law (which now follows the 1988 Berne Convention) the file is copyrighted and cannot be reproduced without permission. See, for example:
There's a common confusion around the Internet that stuff posted on the Internet is in the public domain, unless a copyright notice is present. This is entirely false and has been false for decades. paul
<Paul.Koning@dell.com> writes:
There's a common confusion around the Internet that stuff posted on the Internet is in the public domain, unless a copyright notice is present.
This is entirely false and has been false for decades.
We have this issue a lot with "open source" software. If you don't use one of the actual open source licenses, then it is NOT open source for me. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/> Perl/Unix consulting, Technical writing, Comedy, etc. etc. Still trying to think of something clever for the fourth line of this .sig
I hesitate to take this thread further astray, but I think it's worth it. Sorry. Paul Eggert <eggert@cs.ucla.edu> wrote on Fri, 7 Apr 2017 at 12:22:34 -0700 in <7fe9a9c9-248a-288d-8923-a61167b5eaf4@cs.ucla.edu>:
I didn't refer to any copyright notice, as there is no such notice for the IERS file <https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list>. So by U.S. law (which now follows the 1988 Berne Convention) the file is copyrighted and cannot be reproduced without permission.
That is not precisely correct, in at least 2 ways: (1) A work may be in the public domain without an explicit declaration if the author has made it sufficiently clear in other places. For instance, works of the United States government are in the public domain and cannot be copywritten (see p.3 of Circular 3 that you cited). I'm not saying this is the case with the French government or the IERS, but lack of notice is not sufficient to decide one way or the other. (2) Just because a file is copyrighted does not mean it cannot be reproduced without permission. In the US we have "fair use," which is a complicated set of tests and which is not exactly the same in all countries. But nonetheless, it can be allowed (through fair use, and maybe other doctrines) to reproduce all or part of a copyrighted work absent explicit permission in the appropriate circumstances. Additionally, *facts* cannot be copyrighted. Again, I don't mean to opine on whether the IERS file constitutes facts as opposed to being a creative work, but there's at least some reason to think that it does. It's probably worth some effort to try to clarify how this all plays out with respect to the IERS file. Didn't someone volunteer to talk to them last year? --jhawk@mit.edu John Hawkinson
John Hawkinson wrote:
(1) A work may be in the public domain without an explicit declaration if the author has made it sufficiently clear in other places....
(2) Just because a file is copyrighted does not mean it cannot be reproduced without permission. In the US we have "fair use,"...
Additionally, *facts* cannot be copyrighted.
All quite true. Alas, none of these escape hatches would save us if we copied the entire IERS file, as (a) copying the whole thing clearly would not be fair use, (b) the file has not been placed in the public domain (and I doubt whether it can be, as its authors are under French law), and (c) the file clearly contains expression as well as simple records of facts.
I thought I would ask. --------------------------------- Dear Marshall, I confirm that IERS allows for unlimited copying and use of this and other EOP file,...as far you mention the reference! Best regards, Christian Le Dimanche 9 Avril 2017 00:04 CEST, tme@asteroidinitiatives.com a écrit: Dear Christian; I wanted to ask about the copyright status of files such as https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list as some people here in the US are nervous about this. Can you confirm that the IERS allows for unlimited copying and use of this and other EOP files? Regards Marshall Eubanks -- ____________________________________________________________ Christian Bizouard Service de la Rotation de la Terre - IERS Earth Orientation Parameter Center Observatoire de Paris / SYRTE 61, avenue de l'Observatoire 75014 Paris FRANCE christian.bizouard@obspm.fr tel : 00 33 1 40 51 23 35 http://hpiers.obspm.fr/eop-pc On Sat, Apr 8, 2017 at 4:05 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
John Hawkinson wrote:
(1) A work may be in the public domain without an explicit declaration
if the author has made it sufficiently clear in other places....
(2) Just because a file is copyrighted does not mean it cannot be reproduced without permission. In the US we have "fair use,"...
Additionally, *facts* cannot be copyrighted.
All quite true. Alas, none of these escape hatches would save us if we copied the entire IERS file, as (a) copying the whole thing clearly would not be fair use, (b) the file has not been placed in the public domain (and I doubt whether it can be, as its authors are under French law), and (c) the file clearly contains expression as well as simple records of facts.
On 04/09/2017 05:59 AM, Marshall Eubanks wrote:
I confirm that IERS allows for unlimited copying and use of this and other EOP file,...as far you mention the reference!
Thanks, this is good news, as it should let us redistribute the file as part of the tz database. To make the legal situation clear to everybody, would you please state the copying terms in or near the files in question? For example, you could add the following text to the contents of <https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat>: # Copyright 2017 International Earth Rotation and Reference Systems Service. # This file is licensed under the Creative Commons Attribution 4.0 # International License. To view a copy of this license, visit # http://creativecommons.org/licenses/by/4.0/ or send a letter to # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. Another possibility is to put similar text into <https://hpiers.obspm.fr/iers/bul/bulc/README> and mention that your copyright notice applies to all the files in that directory. I am suggesting the CC BY 4.0 license because it matches the intent stated informally in your email; the wording above is taken from <https://creativecommons.org/choose/non-web-popup?license_code=by&version=4.0>. Your organization may prefer some other license and/or wording; the details don't matter as long as it's clear that downstream users can copy the file unaltered and can use the file's data without restriction. The above text assumes that the IERS holds the copyright; if not, please substitute the correct copyright holder's name. Thanks again.
On Sun, Apr 9, 2017 at 5:28 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 04/09/2017 05:59 AM, Marshall Eubanks wrote:
I confirm that IERS allows for unlimited copying and use of this and other EOP file,...as far you mention the reference!
Thanks, this is good news, as it should let us redistribute the file as part of the tz database. To make the legal situation clear to everybody, would you please state the copying terms in or near the files in question? For example, you could add the following text to the contents of < https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat>:
# Copyright 2017 International Earth Rotation and Reference Systems Service. # This file is licensed under the Creative Commons Attribution 4.0 # International License. To view a copy of this license, visit # http://creativecommons.org/licenses/by/4.0/ or send a letter to # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
Another possibility is to put similar text into < https://hpiers.obspm.fr/iers/bul/bulc/README> and mention that your copyright notice applies to all the files in that directory.
I have made the suggestion. Regards Marshall
I am suggesting the CC BY 4.0 license because it matches the intent stated informally in your email; the wording above is taken from < https://creativecommons.org/choose/non-web-popup?license_co de=by&version=4.0>. Your organization may prefer some other license and/or wording; the details don't matter as long as it's clear that downstream users can copy the file unaltered and can use the file's data without restriction.
The above text assumes that the IERS holds the copyright; if not, please substitute the correct copyright holder's name.
Thanks again.
One thing to add. The wording says, "This file". So, the statement applies to only that file, not the files within a folder or directory generally. The README file (ftp://hpiers.obspm.fr/bul/bulc/README) was changed April 11th, (I think) with the intent of clarifying the copy situation of all files within the folder except that's not what it says. I suspect the Copyright statement should read more along these lines: ... # Copyright 2017 International Earth Rotation and Reference Systems Service. # The files in these folder, and its sub-folders, are licensed under the # Creative Commons Attribution 4.0 International License. To view this ... Someone may also want to move the statement one folder down towards the root of the tree but that's only a guess. This may all seem a quibble and somewhat ridiculous given Christian's intent, but remember, that's how lawyers make their living. :)
On Sunday, April 9, 2017 7:28 PM, Marshall Eubanks <marshall.eubanks@gmail.com> wrote:
On Sun, Apr 9, 2017 at 5:28 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 04/09/2017 05:59 AM, Marshall Eubanks wrote:
I confirm that IERS allows for unlimited copying and use of
this and other EOP file,...as far you mention the reference!
Thanks, this is good news, as it should let us redistribute the file as part of the tz database. To make the legal situation clear to everybody, would you please state the copying terms in or near the files in question? For example, you could add the following text to the contents of <https://hpiers.obspm.fr/iers/ bul/bulc/Leap_Second.dat>:
# Copyright 2017 International Earth Rotation and Reference Systems Service. # This file is licensed under the Creative Commons Attribution 4.0 # International License. To view a copy of this license, visit # http://creativecommons.org/lic enses/by/4.0/ or send a letter to # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
Another possibility is to put similar text into <https://hpiers.obspm.fr/iers/ bul/bulc/README> and mention that your copyright notice applies to all the files in that directory.
I have made the suggestion.
Regards Marshall
I am suggesting the CC BY 4.0 license because it matches the intent stated informally in your email; the wording >above is taken from <https://creativecommons.org/c hoose/non-web-popup?license_co de=by&version=4.0>. Your >organization may prefer some other license and/or wording; the details don't matter as long as it's clear that >downstream users can copy the file unaltered and can use the file's data without restriction.
The above text assumes that the IERS holds the copyright; if not, please substitute the correct copyright holder's >name.
Thanks again.
From my point of view, it would be helpful if the "# This file" comment were also copied into the leap-second file directly. That way, when we copy the leap-second file, we don't need to worry about also copying the copyright notice separately (as we are legally obligated to do); we can simply copy the one file that we need, as-is.
Chris Woodbury via tz said:
BIPM/IERS/EOC is a world standards AND treaty organization. The *information* it provides HAS to be public domain.
Sorry, but that's a very US-centric point of view. Everything the UK government does carries crown copyright and I can't just copy it without permission from their agents (HMSO, IIRC).
(Imagine licensing the meter. They're the same, exact, crew.)
The copyright on the treaty expired a while ago.
The question is really whether their comments are in the public domain. I would humbly suggest that that would require an explicit copyright statement.
Not necessarily explicitly in the document, but there has to be something. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
Perhaps Arthur David Olson or Bradley White can take a look at this, as they're the leap second gurus for tz. I've never tested negative leap seconds.
Using 23:59:59 for a negative leap second seems to do the right thing, as witness the typescript at the bottom of this message. @dashdashado Script started on Fri, Apr 7, 2017 2:37:27 PM $ grep 2020 leapseconds Leap 2020 Jun 30 23:59:59 - S $ make install TOPDIR=$PWD/tmp REDO=right_only >&- $ ./zdump -v America/New_York | grep 2020 America/New_York Sun Mar 8 06:59:59 2020 UT = Sun Mar 8 01:59:59 2020 EST isdst=0 gmtoff=-18000 America/New_York Sun Mar 8 07:00:00 2020 UT = Sun Mar 8 03:00:00 2020 EDT isdst=1 gmtoff=-14400 America/New_York Tue Jun 30 23:59:58 2020 UT = Tue Jun 30 19:59:58 2020 EDT isdst=1 gmtoff=-14400 America/New_York Wed Jul 1 00:00:00 2020 UT = Tue Jun 30 20:00:00 2020 EDT isdst=1 gmtoff=-14400 America/New_York Sun Nov 1 05:59:59 2020 UT = Sun Nov 1 01:59:59 2020 EDT isdst=1 gmtoff=-14400 America/New_York Sun Nov 1 06:00:00 2020 UT = Sun Nov 1 01:00:00 2020 EST isdst=0 gmtoff=-18000 $ exit exit Script done on Fri, Apr 7, 2017 2:38:11 PM On Tue, Apr 4, 2017 at 6:33 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 04/04/2017 09:02 AM, Chris Woodbury via tz wrote:
I reviewed the C code and, it seems to me, the transaction added should actually be for the second before, 23:59:58, as the transaction itself causes 23:59:59 to be skippedaltogether.
Perhaps Arthur David Olson or Bradley White can take a look at this, as they're the leap second gurus for tz. I've never tested negative leap seconds.
Also, I've an AWK script that uses the IERS file Leap_Seconds.dat
rather than the NIST leap-seconds.listfile.
Although that would be a better technical solution, in the past we've shied away from it because the IERS file is copyrighted and we and we and our downstream users don't clearly have permissions to copy and redistribute it.
USNO MAIA also has straight list
of leap-seconds in a different format. <http://maia.usno.navy.mil/ser 7/leapsec.dat>
Thanks, I was not aware of this file. It is public-domain, so perhaps we could use it. Is it updated in a timely fashion when new IERS bulletins are published? And is that web server highly available? These have been sticking points for the NIST file.
Another good, comprehensive article is "The leap second: its history
and possible future" by R.A. Nelson, et al, from Metrogia Vol 38, pp. 509-529 (2001). PDF available (with permission) at:
<https://www.cl.cam.ac.uk/~mgk25/time/metrologia-leapsecond.pdf>
Yes, we link to that in tz-link.htm.
participants (9)
-
Arthur David Olson -
Bradley White -
Chris Woodbury -
Clive D.W. Feather -
John Hawkinson -
Marshall Eubanks -
merlyn＠stonehenge.com -
Paul Eggert -
Paul.Koning＠dell.com