On Wed, Sep 7, 2016 at 4:04 AM, Jon Skeet <skeet@pobox.com> wrote:
(Then there's tzdist of course, but that's a whole other matter...)
I have been looking at RFC 7808 (aka tzdist) and I wonder what is the relationship between the proposed distribution standard and this project. While RFC 7808 mentions "Olson" as a possible "primary-source", it does not look like the proposed JSON format is capable of carrying all information provided by Olson raw files. The distribution format looks like a text variant of the binary zoneinfo files, but it is not clear whether or not the distributed data includes timezone abbreviations. Are there any plans to support the tzdist format in tzcode? Will IANA run its own "Time Zone Data Server"?
Alexander Belopolsky wrote:
I have been looking at RFC 7808 (aka tzdist) and I wonder what is the relationship between the proposed distribution standard and this project.
tzdist is a protocol for distibuting data derived from the tz project, along with other data (e.g., localization of zone names like "America/Los_Angeles").
The distribution format looks like a text variant of the binary zoneinfo files, but it is not clear whether or not the distributed data includes timezone abbreviations.
It should. That data is not specified directly by RFC 7808; it's in RFC 5545. There's not a 1-1 relationship between tzdist data and tz data.
Are there any plans to support the tzdist format in tzcode? Will IANA run its own "Time Zone Data Server"?
I don't know of any plans. As I've said, IANA doesn't have the resources to serve billions of clients.
On 07/09/16 18:56, Paul Eggert wrote:
Are there any plans to support the tzdist format in tzcode? Will IANA run its own "Time Zone Data Server"?
I don't know of any plans. As I've said, IANA doesn't have the resources to serve billions of clients.
tz was always viewed as the primary source for tzdist, until there IS a primary source we can't start to create mirror sites. The mirror servers will provide alternate views of the master data or simply mirror a selected version of the tz raw data. We need a master that can provide the complete history of changes to tz data properly identified, or at least one starting from a current live version. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Sep 7, 2016, at 2:35 PM, Lester Caine <lester@lsces.co.uk> wrote:
On 07/09/16 18:56, Paul Eggert wrote:
Are there any plans to support the tzdist format in tzcode? Will IANA run its own "Time Zone Data Server"?
I don't know of any plans. As I've said, IANA doesn't have the resources to serve billions of clients.
tz was always viewed as the primary source for tzdist, until there IS a primary source we can't start to create mirror sites. The mirror servers will provide alternate views of the master data or simply mirror a selected version of the tz raw data. We need a master that can provide the complete history of changes to tz data properly identified, or at least one starting from a current live version.
I would assume you could create a history from the entire set of archived tzdata* files. That would be a one-time activity, and if any manual adjustment is needed to deal with oddities in old files, that wouldn't be a problem because it would not happen again. Then as each new release comes out, it would be one more delta. Presumably the tzdist master would go to the IANA tz repository for this, and all the mirrors would avoid bothering IANA and take from the master. paul
On 09/07/2016 12:07 PM, Paul.Koning@dell.com wrote:
you could create a history from the entire set of archived tzdata* files.
Although we could do that (modulo the missing tarballs), the story's more complicated because we have two histories: the archived tarballs dating back to 1993, and the Git-based history dating back to 1984 (pre-2012 history was originally in SCCS form). The Git-based history is more detailed, as it records each commit whereas the tarballs merely record the (much rarer) releases. I would rather rely on the Git-based history, as it contains more information and it already exists. A downside is that nobody has correlated the Git-based history to the tarball history for releases before 2012e. (I tried, but ran into glitches and gave up.) Another downside is that minor details in older tarballs (e.g., file timestamps) are missing from or disagree with the Git-based history. But we can leave these less-pressing matters for some software historian of the future.
On 08/09/16 18:27, Paul Eggert wrote:
On 09/07/2016 12:07 PM, Paul.Koning@dell.com wrote:
you could create a history from the entire set of archived tzdata* files.
Although we could do that (modulo the missing tarballs), the story's more complicated because we have two histories: the archived tarballs dating back to 1993, and the Git-based history dating back to 1984 (pre-2012 history was originally in SCCS form). The Git-based history is more detailed, as it records each commit whereas the tarballs merely record the (much rarer) releases.
I would rather rely on the Git-based history, as it contains more information and it already exists. A downside is that nobody has correlated the Git-based history to the tarball history for releases before 2012e. (I tried, but ran into glitches and gave up.) Another downside is that minor details in older tarballs (e.g., file timestamps) are missing from or disagree with the Git-based history. But we can leave these less-pressing matters for some software historian of the future.
Looks like my 'phone' reply only went to Paul ... http://hg.lsces.org.uk/hg/tzdata/tags was my attempt to produce a 'diffable' version of the data tarballs. need to add in the resent releases, which is the easy bit, but I would like to go back down the list as well and tag the earlier releases. The thought a couple of years back however was to simply build a short repository from the tagballs on top of the pre 1996 data from the repo build. But it is all probably just of academic interest now so is there actually any point. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Wed, Sep 7, 2016 at 2:35 PM, Lester Caine <lester@lsces.co.uk> wrote:
tz was always viewed as the primary source for tzdist
Does tzdist have means to specify the "SAVE" time in an observance specification? When zic compiles the raw files, it only preserves the information on whether the DST is in effect (isdst) but looses the information about the amount of the "saved" daylight. This information cannot be reliably inferred from the zoneinfo files in the zones that used double summer time historically. (Note that DST as duration rather than just an isdst-like boolean is required by Python datetime.tzinfo interface. Hence my interest.)
On 09/07/2016 11:35 AM, Lester Caine wrote:
tz was always viewed as the primary source for tzdist, until there IS a primary source we can't start to create mirror sites.
But there is a primary source for tz: the latest release as described at https://www.iana.org/time-zones. This consists of the two gzipped tarballs that we've been talking about, currently the 2016f release. These tarballs can easily be mirrored elsewhere. A complete tzdist public server would need to do more than just mirroring tz releases. That would be a bigger project, one outside the scope of tz proper.
We need a master that can provide the complete history of changes to tz data properly identified, or at least one starting from a current live version. The iana.org website publishes all the tz releases that we've been able to get our hands on. Some older releases have been lost, alas, but I think we have every release published since 1999. For details, look for the string "is missing!" in the NEWS file.
On 07/09/16 21:16, Paul Eggert wrote:
On 09/07/2016 11:35 AM, Lester Caine wrote:
tz was always viewed as the primary source for tzdist, until there IS a primary source we can't start to create mirror sites.
But there is a primary source for tz: the latest release as described at https://www.iana.org/time-zones. This consists of the two gzipped tarballs that we've been talking about, currently the 2016f release. These tarballs can easily be mirrored elsewhere.
There needs to be a seed tzdist service somewhere so that the distribution mechanism designed into the protocol can be used. It has no provision to be seeded from tarballs ... that would have to be done manually initially, and then updates are via diffs to the current version. Some servers may well ignore tz updates and apply each change as they happen, but personally I think that is only going to make maintaiing historic data more difficult.
A complete tzdist public server would need to do more than just mirroring tz releases. That would be a bigger project, one outside the scope of tz proper.
We need a master that can provide the complete history of changes to tz data properly identified, or at least one starting from a current live version.
The iana.org website publishes all the tz releases that we've been able to get our hands on. Some older releases have been lost, alas, but I think we have every release published since 1999. For details, look for the string "is missing!" in the NEWS file.
While the provision of an historic tz version via tzdist is not a particularly difficult task. As Paul says, it's a one time step. The more important thing is to have a current version of tz available against which the update process can be implemented so creating the various 'diff' options. Essentially a fully functional tzdist service would allow you to check a legacy calendar complied with version x-? and find out if information on that clendar may be affected b a later change ... any later change. And if your system is working internally with a leapsecond clock, that would also include indication that times recorded for next year will need a one scond update once the leapsecond change is also posted. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Wed, Sep 7, 2016 at 1:56 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
IANA doesn't have the resources to serve billions of clients.
That's clear, but adding an option to zic to generate timezone data in tzdist format may not be out of reach. Ideally, tzcode would include a reference implementation of an RFC 7808 "Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
On 09/07/2016 01:03 PM, Alexander Belopolsky wrote:
That's clear, but adding an option to zic to generate timezone data in tzdist format may not be out of reach.
vzic already does that, no? See its description in tz-link.htm. Do we need to reinvent that wheel?
Ideally, tzcode would include a reference implementation of an RFC 7808 "Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
The test server that I found most-recently discussed in the tzdist archive was operated at CMU. I don't know the details (perhaps Tim does?) but I expect it was based on Cyrus IMAP, which uses vzic to convert from tz format. You can get a copy of Cyrus IMAP here: https://cyrusimap.org/
On 7 September 2016 at 16:52, Paul Eggert <eggert@cs.ucla.edu> wrote:
Ideally, tzcode would include a reference implementation of an RFC 7808
"Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
The test server that I found most-recently discussed in the tzdist archive was operated at CMU. I don't know the details (perhaps Tim does?) but I expect it was based on Cyrus IMAP, which uses vzic to convert from tz format.
Although I am, in fact, now at Carnegie Mellon University, I wasn't involved with the tzdist demo server. That was Ken Murchison's doing. -- Tim Parenti
On 09/07/2016 05:00 PM, Tim Parenti wrote:
On 7 September 2016 at 16:52, Paul Eggert <eggert@cs.ucla.edu <mailto:eggert@cs.ucla.edu>> wrote:
Ideally, tzcode would include a reference implementation of an RFC 7808 "Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
The test server that I found most-recently discussed in the tzdist archive was operated at CMU. I don't know the details (perhaps Tim does?) but I expect it was based on Cyrus IMAP, which uses vzic to convert from tz format.
Although I am, in fact, now at Carnegie Mellon University, I wasn't involved with the tzdist demo server. That was Ken Murchison's doing.
Yes, I am to blame. Its bundled with our CalDAV/CardDAV server which is part of Cyrus IMAP. It would be non-trivial to make it a standalone service. The folks that work on Bedework (namely Mike Douglass) may have something that can operate as an applet within Apache. -- Kenneth Murchison Principal Systems Software Engineer Carnegie Mellon University
Do indeed - though I think it needs a little tweaking to fully comply. To be accurate it's a java servlet implementation which runs on a java compliant server - tomcat, jboss/wildfly etc On 9/7/16 17:25, Ken Murchison wrote:
On 09/07/2016 05:00 PM, Tim Parenti wrote:
On 7 September 2016 at 16:52, Paul Eggert <eggert@cs.ucla.edu <mailto:eggert@cs.ucla.edu>> wrote:
Ideally, tzcode would include a reference implementation of an RFC 7808 "Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
The test server that I found most-recently discussed in the tzdist archive was operated at CMU. I don't know the details (perhaps Tim does?) but I expect it was based on Cyrus IMAP, which uses vzic to convert from tz format.
Although I am, in fact, now at Carnegie Mellon University, I wasn't involved with the tzdist demo server. That was Ken Murchison's doing.
Yes, I am to blame. Its bundled with our CalDAV/CardDAV server which is part of Cyrus IMAP. It would be non-trivial to make it a standalone service. The folks that work on Bedework (namely Mike Douglass) may have something that can operate as an applet within Apache.
-- Kenneth Murchison Principal Systems Software Engineer Carnegie Mellon University
On Wed, Sep 7, 2016 at 4:52 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 09/07/2016 01:03 PM, Alexander Belopolsky wrote:
That's clear, but adding an option to zic to generate timezone data in
tzdist format may not be out of reach.
vzic already does that, no?
Not really. As far as I can tell, vzic produces a single RFC 2445 vztimezone specification for each IANA zone. All generated .ics files have the same DTSTART set to 19700101T000000 and RFC 7808 TZUNTIL property is not supported at all.
Do we need to reinvent that wheel?
For some reason vzic author had to reinvent the wheel and write his own zone file parser. This part does not need to be reinvented. Since zic already knows how to translate rules to POSIX TZ string format, translating to iCalendar RRULE does not seem to require a lot of code. However, what I had in mind was that zic could implement the RFC 7808 "expand" action < https://tools.ietf.org/html/rfc7808#section-5.4> which is not implemented in vzic.
On 09/07/2016 05:40 PM, Alexander Belopolsky wrote:
As far as I can tell, vzic produces a single RFC 2445 vztimezone specification for each IANA zone. All generated .ics files have the same DTSTART set to 19700101T000000 and RFC 7808 TZUNTIL property is not supported at all.
Thanks, I didn't know about these limitations. So it's a subset of what RFC 7808 envisions.
what I had in mind was that zic could implement the RFC 7808 "expand" action <https://tools.ietf.org/html/rfc7808#section-5.4> which is not implemented in vzic.
Is the idea that a web server would wait for a request about some particular range of time zone data, and then fire up an instance of zic to generate just that data? That sounds slow. Or were you thinking of making zic a long-running process, or caching its results via something like Squid? Something like that might be doable. It sounds reasonably ambitious....
On Wed, Sep 7, 2016 at 9:20 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
what I had in mind was that zic could implement the RFC 7808 "expand"
action <https://tools.ietf.org/html/rfc7808#section-5.4> which is not implemented in vzic.
Is the idea that a web server would wait for a request about some particular range of time zone data, and then fire up an instance of zic to generate just that data? That sounds slow.
I would see IANA reference implementation as an étalon against which developers can test their implementations, not as a component to be reused. In this role performance is not really a consideration.
Or were you thinking of making zic a long-running process, ...
I would rather see a libzic that would expose a parser, some kind of in-memory representation of zones and rules and a set of functions that will mirror RFC 7808 APIs.
On 09/07/2016 08:40 PM, Alexander Belopolsky wrote:
On Wed, Sep 7, 2016 at 4:52 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 09/07/2016 01:03 PM, Alexander Belopolsky wrote:
That's clear, but adding an option to zic to generate timezone data
in tzdist format may not be out of reach.
vzic already does that, no?
Not really. As far as I can tell, vzic produces a single RFC 2445 vztimezone specification for each IANA zone. All generated .ics files have the same DTSTART set to 19700101T000000 and RFC 7808 TZUNTIL property is not supported at all.
I have my own "forked" copy of vzic that I use for the tzdist service that I implemented in Cyrus which fixes a few bugs and plugs a couple of holes in the vzic that has been out there for a while. Its handles all rules currently in tzdata in a more compact version (RRULEs vs RDATEs) than what the stock vzic produces. I intend to commit this back to either libical which has its own fork of vzic or back to the "original". You can see an example of what it outputs here: https://cyrus-test.andrew.cmu.edu/tzdist/zones/America/New_York Note that DTSTART is accurate per tzdata. Also note that TZUNTIL will only appear if the tzdata itself has a termination date or the tzdist client asks for a cropped version of the time zone: https://cyrus-test.andrew.cmu.edu/tzdist/zones/America/New_York?start=201601...
Do we need to reinvent that wheel?
For some reason vzic author had to reinvent the wheel and write his own zone file parser. This part does not need to be reinvented. Since zic already knows how to translate rules to POSIX TZ string format, translating to iCalendar RRULE does not seem to require a lot of code. However, what I had in mind was that zic could implement the RFC 7808 "expand" action <https://tools.ietf.org/html/rfc7808#section-5.4> which is not implemented in vzic.
"expand" is closer to what zdump does than vzic. In fact, I implemented this format with the expand action in my server to compare against zdump -V to make sure that what my copy of vzic generates matches what is expected: https://cyrus-test.andrew.cmu.edu/tzdist/zones/America/New_York/observances?... -- Kenneth Murchison Principal Systems Software Engineer Carnegie Mellon University
On 9/7/16 16:03, Alexander Belopolsky wrote:
On Wed, Sep 7, 2016 at 1:56 PM, Paul Eggert <eggert@cs.ucla.edu <mailto:eggert@cs.ucla.edu>> wrote:
IANA doesn't have the resources to serve billions of clients.
That's clear, but adding an option to zic to generate timezone data in tzdist format may not be out of reach. Ideally, tzcode would include a reference implementation of an RFC 7808 "Time Zone Data Server". Wasn't such a reference implementation created by the tzdist group?
Also the assumption was always that relatively few servers would contact the root service and provide a service to their clients. We built support into the protocol so that we could have primary and secondary servers. The assumption was something like vendors providing their own service for their products with only a few - perhaps registered - servers contacting the root.
participants (7)
-
Alexander Belopolsky -
Ken Murchison -
Lester Caine -
Michael Douglass -
Paul Eggert -
Paul.Koning＠dell.com -
Tim Parenti