May 2005 - tz - lists.iana.org

Re: Back-of-the-envelope cost of extra data :-)
by kennykb＠crd.ge.com May 9, 2005

May 9, 2005

olsona(a)lecserver.nci.nih.gov said: > One way to reduce data file growth is to store just 5 bytes of time_t > values rather than 8. Better still would be, in the 64-bit version, to keep all the timezones in a single file rather than using the file system as a database. You lose an incredible abount of space to internal fragmentation. On my Linux machine: % find /usr/share/zoneinfo -type f -print | xargs cat | wc -c 873941 % du -k -s /usr/share/zoneinfo 5136 /usr/share/zoneinfo So there's a little over 5 megabytes of zoneinfo files (much of it in the 'right' rules), but less than 900 kilobytes of actual data. -- 73 de ke9tv/2, Kevin KENNY GE Corporate Research & Development kennykb(a)crd.ge.com P. O. Box 8, Bldg. K-1, Rm. 5B36A Schenectady, New York 12301-0008 USA

1 0

Re: Back-of-the-envelope cost of extra data :-)
by kennykb＠crd.ge.com May 4, 2005

May 4, 2005

kre(a)munnari.oz.au said: > If at some future point the code is going to switch from the tables, > to an algorithmic conversion, why bloat the tables by including so > much speculative future data? Exactly what I've been saying - except that for some reason my messages haven't been getting to the mailing list. I'm perfectly comfortable with today's tables that go out to Y2038 and resorting to a single rule for the fictitious timezone that exists thereafter. (We need to expand the table format so that as Y2038 approaches, we can extend the data past it, but that's by no means urgent.) Really, I find the whole argument ridiculous. Incidentally, for whatever it's worth, todays zoneinfo files occupy about 5.1 megabytes on my Linux box. The actual data are only about 450 kilobytes; the rest results from internal fragmentation. A gzip'ped tarball of the zoneinfo files compresses down to a bit over 250 kilobytes. (This include the 'right' files; Tcl's selected set compresses to 190 kilobytes, even though its information files are text files. Tcl doesn't need the 'right' files, ever, because its model of time has no leap seconds - it's similar to Markus Kuhn's UTS.) -- 73 de ke9tv/2, Kevin KENNY GE Corporate Research & Development kennykb(a)crd.ge.com P. O. Box 8, Bldg. K-1, Rm. 5B36A Schenectady, New York 12301-0008 USA

1 0

Back-of-the-envelope cost of extra data :-)
by Olson, Arthur David (NIH/NCI) May 4, 2005

May 4, 2005

This output... Script started on Wed 04 May 2005 10:15:00 AM EDT lecserver$ du -s tz*/tmp/*/zoneinfo 489 tz/tmp/etc/zoneinfo 1709 tzexp/tmp/etc/zoneinfo lecserver$ exit script done on Wed 04 May 2005 10:15:08 AM EDT ...indicates that "old-format" data eats up about half a megabyte of disk space total while "new-format" data eats up about 2 megabytes. (Some individual files grow by a factor of 9; others don't grow at all; the final result lies between these extremes.) At a dollar per gigabyte of disk space, the old stuff costs one twentieth of a cent per computer while the new stuff costs four twentieths of a cent per computer, for a difference of three twentieths of a cent. Making a wild estimate of one computer per person, there are about 6.5 billion computers in the world. In this case I'm happy if 90% are running Windows, since Microsoft doesn't ship time zone files. That leaves at most 650 million computers where the time zone files might show up. Maximum total cost: 650 million computers * three twentieths of a cent: $975,000 (ulp!) Frightening corollary: 1024 bytes of Microsoft code bloat costs the world economy $5850. --ado

3 2

Re: Proposed 64-bit changes
by kennykb＠crd.ge.com May 4, 2005

May 4, 2005

mark.davis(a)jtcsv.com said: > A. Extend by last year. > B. No Daylight > C. Others? I find both (A) and (B) acceptable. The politicians routinely diddle with the Daylight Saving Time rules. Even the year after the next is conjectural - what the rules will be in a century, or 400 years, is pure speculation. As long as something resembling the correct time of day is returned, the results should be entirely adequate. In short, Y2038 bugs in Daylight Saving Time processing are the least of our worries. As I said, in Tcl's analogue to zic (http://tip.tcl.tk/173) I put in an arbitrary cutoff (easily changed) of 2100. I suspect it ought to be shorter. (And I do get complaints about the footprint taken up with the zoneinfo files, owing to the places where they wind up ROMmed.) -- 73 de ke9tv/2, Kevin KENNY GE Corporate Research & Development kennykb(a)crd.ge.com P. O. Box 8, Bldg. K-1, Rm. 5B36A Schenectady, New York 12301-0008 USA

1 0

RE: Proposed 64-bit changes
by Robbin Kawabata May 4, 2005

May 4, 2005

> From: "Olson, Arthur David (NIH/NCI)" <olsona(a)dc37a.nci.nih.gov> > To: "'Robbin Kawabata'" <Robbin.Kawabata(a)eng.sun.com>, "Tz (tz(a)elsie.nci.nih.g ov)" <tz(a)elsie.nci.nih.gov> > Subject: RE: Proposed 64-bit changes > Date: Mon, 2 May 2005 09:46:29 -0400 > > > Is it feasible for zic to encode variable information in the data file for > > the last set of rules, that would be used for times past the last entry in > > the transition table? Then localtime() > > would use the variables algorithmically rather than using table-driven > > data, for dates past the last table entry. Perhaps set a minimum size for > > the table entries (ie, have table entries > > at least up to year xxxx.) > > This approach isn't as general as the extend-400-years approach. Take, for > example, the proposal a few years back to extend DST into November in the > Pacific Time Zone in presidential election years (as a way of minimizing > announced results from New York affecting voting in California). The > irregularities that would have resulted can't be handled by a POSIX-style > rule string; they can be handled by the extend-400-years approach. To address these types of timezones, zic could write a second set of variables. One set of variables would be used (for example) for nonpres years, and one set of variables would be used for uspres years. As a side note, our standards representative comments that POSIX TZ rules have been extended in recent years to be more flexible than they had been in the past, and would likely be extended again if uspres, even, odd, or other rules are actually put into practice. After more discussion here, we feel the extend-400-year approach would be acceptable. We would be OK with either the extend-400-year approach or the variable approach. Robbin

4 3

Re: Proposed 64-bit changes
by kennykb＠crd.ge.com May 3, 2005

May 3, 2005

To me, "extend 400 years" seems actually a trifle excessive, since Daylight Saving Time rules appear to have a mean lifespan of a decade or so. The meaning of a given time zone that far in the future is, to put it mildly, conjectural. In Tcl's 'zic' counterpart (http://tip.tcl.tk/173) I decided somewhat arbitrarily to go out 100 years as a compromise between Y2038 bugs and disc footprint. If we all settle on 400 years as the standard, I could of course change that, but I'm reluctant to spend the space on it. (Tcl has a way of getting ported to strange places like routers and X-ray machines.) As I said, it sounds a bit excessive, because civil time zones are not stable for anywhere near that length of time. Messing around with the clocks seems to be one favorite activity of governments that want to give the impression that they're doing something. -- 73 de ke9tv/2, Kevin KENNY GE Corporate Research & Development kennykb(a)crd.ge.com P. O. Box 8, Bldg. K-1, Rm. 5B36A Schenectady, New York 12301-0008 USA

1 0

Postponed code pushes
by Olson, Arthur David (NIH/NCI) May 3, 2005

May 3, 2005

Given comments about increased data file size and about storing rules in the data files, I'm holding off on incorporating the proposed 64-bit changes in the ftp version of the time zone package. I welcome additional comments on these matters; in particular, is adding a "short-output" option to zic a way of addressing the concern, and is it a good idea (given that it will slow the spread of new-format data file)? I'm also holding off on handling TZ strings such as "<UTC+10>-10", with security in mind (as mentioned by other folks on the time zone mailing list). On the one hand, a string such as "<UTC+10>10\nX-Arbitrary-Mail-Header: Arbitrary-Value" could be used to forge a mail header. On the other hand, there are far easier ways forge mail headers. Is there anything that could be done by a carefully selected TZ value that can't be done more easily in other ways? --ado

2 1

TZ corrections
by Jesper Norgaard Welen May 2, 2005

May 2, 2005

For the Palmer station in Antarctica, the rule ArgAQ has been declared. But the rule ArgAQ is not used until 1965, while the rule is defined for year 1964. Therefore it would be more reasonable to redefine it the following way: -Rule ArgAQ 1964 1966 - Mar 1 0:00 0 - -Rule ArgAQ 1964 1966 - Oct 15 0:00 1:00 S +Rule ArgAQ 1965 1966 - Mar 1 0:00 0 - +Rule ArgAQ 1965 1966 - Oct 15 0:00 1:00 S Zone Antarctica/Palmer 0 - zzz 1965 -4:00 ArgAQ AR%sT 1969 Oct 5 In the Argentina timezones, it seems that apart from the initial LMT (which is of course different), the Catamarca timezone and the Chubut timezone are in fact identical, so I suggest: # # Catamarca (CT) Zone America/Argentina/Catamarca -4:23:08 - LMT 1894 Oct 31 -4:16:48 - CMT 1920 May -4:00 - ART 1930 Dec -4:00 Arg AR%sT 1969 Oct 5 -3:00 Arg AR%sT 1991 Mar 3 -4:00 - WART 1991 Oct 20 -3:00 Arg AR%sT 1999 Oct 3 -4:00 Arg AR%sT 2000 Mar 3 -3:00 - ART 2004 Jun 1 -4:00 - WART 2004 Jun 20 -3:00 - ART # # Chubut (CH) # The name "Comodoro Rivadavia" exceeds the 14-byte POSIX limit. -Zone America/Argentina/ComodRivadavia -4:30:00 - LMT 1894 Oct 31 - -4:16:48 - CMT 1920 May - -4:00 - ART 1930 Dec - -4:00 Arg AR%sT 1969 Oct 5 - -3:00 Arg AR%sT 1991 Mar 3 - -4:00 - WART 1991 Oct 20 - -3:00 Arg AR%sT 1999 Oct 3 - -4:00 Arg AR%sT 2000 Mar 3 - -3:00 - ART 2004 Jun 1 - -4:00 - WART 2004 Jun 20 - -3:00 - ART +Link America/Argentina/Catamarca America/Argentina/ComodRivadavia In the 'asia' file, the EUAsia rule is defined from 1981 to max. In fact, it is not used for any timezones before 1996, so I suggest: -Rule EUAsia 1981 max - Mar lastSun 1:00u 1:00 S +Rule EUAsia 1996 max - Mar lastSun 1:00u 1:00 S Rule EUAsia 1996 max - Oct lastSun 1:00u 0 - 2:00 Cyprus EE%sT 1998 Sep 2:00 EUAsia EE%sT 4:00 - AZT 1996 # Azerbaijan time 4:00 EUAsia AZ%sT 1997 The E-EurAsia rule is defined from 1979, but not used until 1992. I suggest: -Rule E-EurAsia 1981 max - Mar lastSun 0:00 1:00 S -Rule E-EurAsia 1979 1995 - Sep lastSun 0:00 0 - +Rule E-EurAsia 1992 max - Mar lastSun 0:00 1:00 S +Rule E-EurAsia 1992 1995 - Sep lastSun 0:00 0 - Rule E-EurAsia 1996 max - Oct lastSun 0:00 0 - 3:00 RussiaAsia GE%sT 1992 # Georgia Time 3:00 E-EurAsia GE%sT 1994 Sep lastSun 4:00 E-EurAsia GE%sT 1996 Oct lastSun 4:00 1:00 GEST 1997 Mar lastSun 4:00 E-EurAsia GE%sT 2004 Jun 27 3:00 RussiaAsia GE%sT As far as I can see, the 1949-part of the PRC rule doesn't actually have any effect, so it could be deleted: -Rule PRC 1949 only - Jan 1 0:00 0 S Rule PRC 1986 only - May 4 0:00 1:00 D As we see below, only in Shanghai it is inside a time range where it is activated, but as far as I can see, before and after the rule, GMT offset will be +8: Zone Asia/Harbin 8:26:44 - LMT 1928 # or Haerbin 8:30 - CHAT 1932 Mar # Changbai Time 8:00 - CST 1940 9:00 - CHAT 1966 May 8:30 - CHAT 1980 May 8:00 PRC C%sT # Zhongyuan Time ("Central plain Time") Zone Asia/Shanghai 8:05:52 - LMT 1928 8:00 Shang C%sT 1949 8:00 PRC C%sT # Long-shu Time (probably due to Long and Shu being two names of that area) Zone Asia/Chongqing 7:06:20 - LMT 1928 # or Chungking 7:00 - LONT 1980 May # Long-shu Time 8:00 PRC C%sT # Xin-zang Time ("Xinjiang-Tibet Time") Zone Asia/Urumqi 5:50:20 - LMT 1928 # or Urumchi 6:00 - URUT 1980 May # Urumqi Time 8:00 PRC C%sT # Kunlun Time Zone Asia/Kashgar 5:03:56 - LMT 1928 # or Kashi or Kaxgar 5:30 - KAST 1940 # Kashgar Time 5:00 - KAST 1980 May 8:00 PRC C%sT In my home country Denmark, it seems that there is a superfluous rule for 2.nd of April 1945, which I suggest to delete: Rule Denmark 1940 only - May 15 0:00 1:00 S -Rule Denmark 1945 only - Apr 2 2:00s 1:00 S Rule Denmark 1945 only - Aug 15 2:00s 0 - Unles my interpretation is wrong, both before and after the rule takes effect in April 2, 1945, the time in Denmark would be GMT+2 (e.g. GMT+1 +1), or does it mean that time is GMT+3? As a side-question, I haven't been able to check this with any Time Zone Converter (the web version has a cutoff date of 1970). Is there any way to check this by a program, and which? Concerning the original definition for Denmark: Zone Europe/Copenhagen 0:50:20 - LMT 1890 - 0:50:20 - CMT 1894 Apr # Copenhagen Mean Time + 0:50:20 - CMT 1894 Jan 1 0:00 # Copenhagen Mean Time 1:00 Denmark CE%sT 1942 Nov 2 2:00s I found some references in old on-line lawtexts in Denmark: The page http://www.hum.aau.dk/~poe/tid/tine/DanskTid.htm says that the law was in effect from 1. January 1894, not from April. The page http://www.retsinfo.dk/_GETDOCI_/ACCN/A18930008330-REGL confirms this, and states that the law was put forth 29.th. of March 1893 specifying that it would be in effect from 1.st. of January 1894. The EU treaty with effect from 1973: http://www.retsinfo.dk/_GETDOCI_/ACCN/A19722110030-REGL This provoked a new law from 1974 to make possible summer time changes in subsequenet decrees with the law http://www.retsinfo.dk/_GETDOCI_/ACCN/A19740022330-REGL It seems however that no decree was set forward until 1980. I have not found any decree, but in another related law, the effecting DST changes are stated explicitly: to be from 6.th. of April 1980 at 2.00 to 28.th. of September 1980 at 2.00. If this is true, this differs slightly from the EU rule in that DST runs to 2.00, not 3.00. We don't know when Denmark began using the EU rule correctly, but we have only confirmation of the 1980-time, so I presume it was correct in 1981: The law is about the management of the extra hour, concerning working hours reported and effect on obligatory-rest rules (which was suspended on that night): http://www.retsinfo.dk/_GETDOCI_/ACCN/C19801120554-REGL I suggest the following changes: Rule Denmark 1948 only - Aug 8 2:00s 0 - +Rule Denmark 1980 only - Apr Sun>=1 2:00 1:00 S +Rule Denmark 1980 only - Sep lastSun 2:00 0 - # Whitman also gives 1949 Apr 9 to 1949 Oct 1, and disagrees in minor ways # about many of the above dates; go with Shanks. # -# For 1894, Shanks says Jan, Whitman Apr; go with Whitman. +# For 1894, Shanks says Jan, Whitman Apr; go with Shanks. # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Europe/Copenhagen 0:50:20 - LMT 1890 - 0:50:20 - CMT 1894 Apr # Copenhagen Mean Time + 0:50:20 - CMT 1894 Jan 1 0:00 # Copenhagen Mean Time 1:00 Denmark CE%sT 1942 Nov 2 2:00s 1:00 C-Eur CE%sT 1945 Apr 2 2:00 - 1:00 Denmark CE%sT 1980 + 1:00 Denmark CE%sT 1981 1:00 EU CE%sT In Norway, I accidentally tripped over some new information from the Metheorological Institute in Norway, citing information from The Theoretical Astrophysics Institute of the University in Oslo: http://met.no/met/met_lex/q_u/sommertid.html This in effect confirms the information from Shanks, although there is a discrepancy for 1940-1942. I suggest the following changes: # Norway # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S # Whitman gives 1916 May 21 - 1916 Oct 21; go with Shanks. Rule Norway 1916 only - May 22 1:00 1:00 S Rule Norway 1916 only - Sep 30 0:00 0 - # Whitman says DST observed 1935-08-11/1942-11-01, then 1943-03-29/10-04, -# 1944-04-03/10-02, and 1945-04-01/10-01; go with Shanks. +# 1944-04-03/10-02, and 1945-04-01/10-01; Shanks says C-Eur rules from +# August 10, 1940. Go with Oslo University, Institute for Astrophysics, +# see http://met.no/met/met_lex/q_u/sommertid.html. +Rule Norway 1940 only - Aug 11 2:00s 1:00 S +Rule Norway 1942 only - Nov 2 2:00s 0 - Rule Norway 1945 only - Apr 2 2:00s 1:00 S Rule Norway 1945 only - Oct 1 2:00s 0 - Rule Norway 1959 1964 - Mar Sun>=15 2:00s 1:00 S Rule Norway 1959 1965 - Sep Sun>=15 2:00s 0 - Rule Norway 1965 only - Apr 25 2:00s 1:00 S # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Europe/Oslo 0:43:00 - LMT 1895 Jan 1 - 1:00 Norway CE%sT 1940 Aug 10 23:00 + 1:00 Norway CE%sT 1943 1:00 C-Eur CE%sT 1945 Apr 2 2:00 1:00 Norway CE%sT 1980 1:00 EU CE%sT Regards, Jesper Nørgaard Welen Jesper Nørgaard Welen Email: jnorgard(a)Prodigy.Net.mx Project Leader (Líder de Proyecto) Software CIMMYT - Centro Internacional de Mejoramiento de Maíz y Trigo Dirección: CIMMYT Int. c/o Jesper Nørgaard Km. 45, Carretera México-Veracruz El Batán Texcoco, Edo. de México CP 56130 MEXICO Tel.: +52 (5) 58-04-20-04 ext. 1374 Fax: +52 (5) 58-04-75-58 Tel. Casa: 53-10-05-95 ó 53-10-97-78 Download the free program World Time Explorer, I made: http://www.cimmyt.org/timezone/index.htm

6 7

RE: Proposed 64-bit changes
by Olson, Arthur David (NIH/NCI) May 2, 2005

May 2, 2005

> Is it feasible for zic to encode variable information in the data file for the last set of rules, that would be used for times past the last entry in the transition table? Then localtime() > would use the variables algorithmically rather than using table-driven data, for dates past the last table entry. Perhaps set a minimum size for the table entries (ie, have table entries > at least up to year xxxx.) This approach isn't as general as the extend-400-years approach. Take, for example, the proposal a few years back to extend DST into November in the Pacific Time Zone in presidential election years (as a way of minimizing announced results from New York affecting voting in California). The irregularities that would have resulted can't be handled by a POSIX-style rule string; they can be handled by the extend-400-years approach. > I'm not sure yet, but with 9x larger timezone binaries, Sun might retain the old-style timezone binaries for timezone patch updates. > If so, it might be useful if zic has an option to produce old-style timezone binaries. That's certainly doable. Note, though, that the new binary files start with information in the old format, and should be usable on old systems. --ado

1 0

Re: Proposed 64-bit changes
by Robbin Kawabata May 2, 2005

May 2, 2005

Some feedback - 1. This is another idea for supporting dates into the far future. Is it feasible for zic to encode variable information in the data file for the last set of rules, that would be used for times past the last entry in the transition table? Then localtime() would use the variables algorithmically rather than using table-driven data, for dates past the last table entry. Perhaps set a minimum size for the table entries (ie, have table entries at least up to year xxxx.) zic would effectively map the last set of transition rules of an Olson timezone to an equivalent POSIX timezone. 2. I'm not sure yet, but with 9x larger timezone binaries, Sun might retain the old-style timezone binaries for timezone patch updates. If so, it might be useful if zic has an option to produce old-style timezone binaries. 3. POSIX timezone 64-bit time_t support It seems the new localtime.c does not support 64-bit dates for POSIX timezones. Will this be supported? Thanks, Robbin > Date: Thu, 21 Apr 2005 15:24:10 -0400 (EDT) > From: Arthur David Olson <olsona(a)lecserver.nci.nih.gov> > To: tz(a)lecserver.nci.nih.gov > Subject: Proposed 64-bit changes > > Below are diffs to the time zone code to cope with 64-bit time_t systems; > these reflect suggestions I've received to date. Absent negative feedback, > I'll update the ftp versions on May 2, 2005.

3 2