[PROPOSED] zic option for including data from a certain year onward
From 4093e426a42421fd2d8579b07e8edb3fb116f561 Mon Sep 17 00:00:00 2001 From: Christopher Wong <christwo@axis.com> Date: Tue, 5 Feb 2019 15:02:57 +0100 Subject: [PATCH] zic.c: Add -k option to keep specified year and onward
The -k option will take a year as input. It will strip away historical transition data before the specified year. The TZif files will be truncated. --- zic.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 69 insertions(+), 3 deletions(-) diff --git a/zic.c b/zic.c index 2ebc66a..9a3b1f5 100644 --- a/zic.c +++ b/zic.c @@ -178,6 +178,7 @@ static void mkdirs(char const *, bool); static void newabbr(const char * abbr); static zic_t oadd(zic_t t1, zic_t t2); static void outzone(const struct zone * zp, ptrdiff_t ntzones); +static zic_t ytime(zic_t year); static zic_t rpytime(const struct rule * rp, zic_t wantedy); static void rulesub(struct rule * rp, const char * loyearp, const char * hiyearp, @@ -610,6 +611,10 @@ static const char * leapsec; static const char * tzdefault; static const char * yitcommand; +/* This is used to truncate tzfiles. Only year specified by keepyear and + onward will be kept. Don't expect year to be outside range 0-9999. */ +static zic_t keepyear; + int main(int argc, char **argv) { @@ -640,7 +645,8 @@ main(int argc, char **argv) } else if (strcmp(argv[k], "--help") == 0) { usage(stdout, EXIT_SUCCESS); } - while ((c = getopt(argc, argv, "d:l:L:p:st:vy:")) != EOF && c != -1) + keepyear = ZIC_MIN; + while ((c = getopt(argc, argv, "d:k:l:L:p:st:vy:")) != EOF && c != -1) switch (c) { default: usage(stderr, EXIT_FAILURE); @@ -654,6 +660,22 @@ _("%s: More than one -d option specified\n"), return EXIT_FAILURE; } break; + case 'k': + if (keepyear == ZIC_MIN) { + keepyear = atoi(optarg); + /* Don't expect year to be outside range 0-9999. */ + if (keepyear < 0) + keepyear = 0; + if (keepyear > 9999) + keepyear = 9999; + } + else { + fprintf(stderr, +_("%s: More than one -k option specified\n"), + progname); + return EXIT_FAILURE; + } + break; case 'l': if (lcltime == NULL) lcltime = optarg; @@ -1749,7 +1771,7 @@ writezone(const char *const name, const char *const string, char version, register FILE * fp; register ptrdiff_t i, j; register int leapcnt32, leapi32; - register ptrdiff_t timecnt32, timei32; + register ptrdiff_t timecnt32, timei32, timei; register int pass; static const struct tzhead tzh0; static struct tzhead tzh; @@ -1865,6 +1887,37 @@ writezone(const char *const name, const char *const string, char version, --leapcnt32; ++leapi32; } + + timei = 0; + if (keepyear != ZIC_MIN) { + /* Discard transitions that happened before the keepyear. */ + int nbr_removes; + + nbr_removes = 0; + for (i = timei32; i < timecnt; ++i) + if (ats[i] < ytime(keepyear)) + ++nbr_removes; + timecnt32 -= nbr_removes; + timei32 += nbr_removes; + + nbr_removes = 0; + for (i = 0; i < timecnt; ++i) + if (ats[i] < ytime(keepyear)) + ++nbr_removes; + + if (nbr_removes > 0) { + /* Override the last removed transition time with the first transition + time that is the start point of the truncation range. Set defaulttype + to this first transition time. */ + --nbr_removes; + ats[nbr_removes] = ytime(keepyear); + defaulttype = types[nbr_removes]; + } + + timecnt -= nbr_removes; + timei += nbr_removes; + } + /* ** Remove old file, if any, to snap links. */ @@ -1910,7 +1963,7 @@ writezone(const char *const name, const char *const string, char version, thisleapi = leapi32; thisleapcnt = leapcnt32; } else { - thistimei = 0; + thistimei = timei; thistimecnt = timecnt; toomanytimes = thistimecnt >> 31 >> 31 >> 2 != 0; thisleapi = 0; @@ -3107,6 +3160,19 @@ tadd(zic_t t1, zic_t t2) } /* +** Given a year, compute the date (in seconds since January 1, +** 1970, 00:00). +*/ + +static zic_t +ytime(zic_t year) +{ + struct rule xr = { 0 }; + xr.r_dayofmonth = 1; + return rpytime(&xr, year); +} + +/* ** Given a rule, and a year, compute the date (in seconds since January 1, ** 1970, 00:00 LOCAL time) in that year that the rule refers to. */ -- 2.11.0 ?
Thanks for the suggestion. It'd be helpful to have a truncation option for zic. However, the option should allow the user to select an arbitrary contiguous region of time: that is, not necessarily at year boundaries, and perhaps including an end time as well as a start time. This would be useful for people trying to implement the time truncation feature of Internet RFC 8536.
Hi Paul, I think it would be possible for me to do some minor change to allow seconds in UNIX time as a start time. I try to send a new patch later. Regarding the end time, we currently don't have a use case for it and it will requires a bigger effort. However, the changes I have made will be a base for further development of the time truncation functionality mentioned in RFC 8536. BR, Christopher Wong ________________________________________ From: Paul Eggert <eggert@cs.ucla.edu> Sent: Tuesday, March 5, 2019 20:43 To: Christopher Wong Cc: Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward Thanks for the suggestion. It'd be helpful to have a truncation option for zic. However, the option should allow the user to select an arbitrary contiguous region of time: that is, not necessarily at year boundaries, and perhaps including an end time as well as a start time. This would be useful for people trying to implement the time truncation feature of Internet RFC 8536.
From c37fe30ee3a13c98c77a4ec2cafe0671d4173e42 Mon Sep 17 00:00:00 2001 From: Christopher Wong <christwo@axis.com> Date: Tue, 5 Feb 2019 15:02:57 +0100 Subject: [PATCH] zic.c: Add -k option to truncate with specified start time
The -k option will take a start time in seconds since January 1, 1970 (Unix Epoch) as input. It will strip away historical transition data before the specified start time. The TZif files will be truncated. --- zic.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/zic.c b/zic.c index 2ebc66a..40185b2 100644 --- a/zic.c +++ b/zic.c @@ -610,6 +610,10 @@ static const char * leapsec; static const char * tzdefault; static const char * yitcommand; +/* This is used to truncate tzfiles. Only transitions starting with truncstart + and onward will be kept. */ +static zic_t truncstart; + int main(int argc, char **argv) { @@ -640,7 +644,8 @@ main(int argc, char **argv) } else if (strcmp(argv[k], "--help") == 0) { usage(stdout, EXIT_SUCCESS); } - while ((c = getopt(argc, argv, "d:l:L:p:st:vy:")) != EOF && c != -1) + truncstart = ZIC_MIN; + while ((c = getopt(argc, argv, "d:k:l:L:p:st:vy:")) != EOF && c != -1) switch (c) { default: usage(stderr, EXIT_FAILURE); @@ -654,6 +659,19 @@ _("%s: More than one -d option specified\n"), return EXIT_FAILURE; } break; + case 'k': + if (truncstart == ZIC_MIN) { + truncstart = atoi(optarg); + if (truncstart > ZIC_MAX) + truncstart = ZIC_MAX; + } + else { + fprintf(stderr, +_("%s: More than one -k option specified\n"), + progname); + return EXIT_FAILURE; + } + break; case 'l': if (lcltime == NULL) lcltime = optarg; @@ -1749,7 +1767,7 @@ writezone(const char *const name, const char *const string, char version, register FILE * fp; register ptrdiff_t i, j; register int leapcnt32, leapi32; - register ptrdiff_t timecnt32, timei32; + register ptrdiff_t timecnt32, timei32, timei; register int pass; static const struct tzhead tzh0; static struct tzhead tzh; @@ -1865,6 +1883,37 @@ writezone(const char *const name, const char *const string, char version, --leapcnt32; ++leapi32; } + + timei = 0; + if (truncstart != ZIC_MIN) { + /* Discard transitions that happened before the truncstart. */ + int nbr_removes; + + nbr_removes = 0; + for (i = timei32; i < timecnt; ++i) + if (ats[i] < truncstart) + ++nbr_removes; + timecnt32 -= nbr_removes; + timei32 += nbr_removes; + + nbr_removes = 0; + for (i = 0; i < timecnt; ++i) + if (ats[i] < truncstart) + ++nbr_removes; + + if (nbr_removes > 0) { + /* Override the last removed transition time with the first transition + time that is the start point of the truncation range. Set defaulttype + to this first transition time. */ + --nbr_removes; + ats[nbr_removes] = truncstart; + defaulttype = types[nbr_removes]; + } + + timecnt -= nbr_removes; + timei += nbr_removes; + } + /* ** Remove old file, if any, to snap links. */ @@ -1910,7 +1959,7 @@ writezone(const char *const name, const char *const string, char version, thisleapi = leapi32; thisleapcnt = leapcnt32; } else { - thistimei = 0; + thistimei = timei; thistimecnt = timecnt; toomanytimes = thistimecnt >> 31 >> 31 >> 2 != 0; thisleapi = 0; -- 2.11.0
On Mar 8, 2019, at 3:38 AM, Christopher Wong <christopher.wong@axis.com> wrote:
+ case 'k': + if (truncstart == ZIC_MIN) { + truncstart = atoi(optarg); + if (truncstart > ZIC_MAX) + truncstart = ZIC_MAX; + } + else { + fprintf(stderr, +_("%s: More than one -k option specified\n"), + progname); + return EXIT_FAILURE; + } + break;
I don’t see a lower bounds test for the -k argument; is that intentional? Scott Nelson Systems Analyst/Programmer | Thermeon Ltd | Company No 11538126
Hi Scott, Thanks for the comment! The atoi() wasn't really the right function to use. I have changed that to atol() instead. Since truncstart is declared as zic_t it can only store values between ZIC_MIN and ZIC_MAX and atol() will also return within that range. So I have removed the bounds test. BR, Christopher Wong ________________________________________ From: Scott Nelson <sbnelson@thermeon.com> Sent: Friday, March 8, 2019 15:07 To: Christopher Wong Cc: Paul Eggert; Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward
On Mar 8, 2019, at 3:38 AM, Christopher Wong <christopher.wong@axis.com> wrote:
+ case 'k': + if (truncstart == ZIC_MIN) { + truncstart = atoi(optarg); + if (truncstart > ZIC_MAX) + truncstart = ZIC_MAX; + } + else { + fprintf(stderr, +_("%s: More than one -k option specified\n"), + progname); + return EXIT_FAILURE; + } + break;
I don’t see a lower bounds test for the -k argument; is that intentional? Scott Nelson Systems Analyst/Programmer | Thermeon Ltd | Company No 11538126
From 326a7b1078f62e7f0daf3a54b30a58474d22103a Mon Sep 17 00:00:00 2001 From: Christopher Wong <christwo@axis.com> Date: Tue, 5 Feb 2019 15:02:57 +0100 Subject: [PATCH] zic.c: Add -k option to truncate with specified start time
The -k option will take a start time in seconds since January 1, 1970 (Unix Epoch) as input. It will strip away historical transition data before the specified start time. The TZif files will be truncated. --- zic.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 49 insertions(+), 3 deletions(-) diff --git a/zic.c b/zic.c index 2ebc66a..ea5164f 100644 --- a/zic.c +++ b/zic.c @@ -610,6 +610,10 @@ static const char * leapsec; static const char * tzdefault; static const char * yitcommand; +/* This is used to truncate tzfiles. Only transitions starting with truncstart + and onward will be kept. */ +static zic_t truncstart; + int main(int argc, char **argv) { @@ -640,7 +644,8 @@ main(int argc, char **argv) } else if (strcmp(argv[k], "--help") == 0) { usage(stdout, EXIT_SUCCESS); } - while ((c = getopt(argc, argv, "d:l:L:p:st:vy:")) != EOF && c != -1) + truncstart = ZIC_MIN; + while ((c = getopt(argc, argv, "d:k:l:L:p:st:vy:")) != EOF && c != -1) switch (c) { default: usage(stderr, EXIT_FAILURE); @@ -654,6 +659,16 @@ _("%s: More than one -d option specified\n"), return EXIT_FAILURE; } break; + case 'k': + if (truncstart == ZIC_MIN) + truncstart = atol(optarg); + else { + fprintf(stderr, +_("%s: More than one -k option specified\n"), + progname); + return EXIT_FAILURE; + } + break; case 'l': if (lcltime == NULL) lcltime = optarg; @@ -1749,7 +1764,7 @@ writezone(const char *const name, const char *const string, char version, register FILE * fp; register ptrdiff_t i, j; register int leapcnt32, leapi32; - register ptrdiff_t timecnt32, timei32; + register ptrdiff_t timecnt32, timei32, timei; register int pass; static const struct tzhead tzh0; static struct tzhead tzh; @@ -1865,6 +1880,37 @@ writezone(const char *const name, const char *const string, char version, --leapcnt32; ++leapi32; } + + timei = 0; + if (truncstart != ZIC_MIN) { + /* Discard transitions that happened before the truncstart. */ + int nbr_removes; + + nbr_removes = 0; + for (i = timei32; i < timecnt; ++i) + if (ats[i] < truncstart) + ++nbr_removes; + timecnt32 -= nbr_removes; + timei32 += nbr_removes; + + nbr_removes = 0; + for (i = 0; i < timecnt; ++i) + if (ats[i] < truncstart) + ++nbr_removes; + + if (nbr_removes > 0) { + /* Override the last removed transition time with the first transition + time that is the start point of the truncation range. Set defaulttype + to this first transition time. */ + --nbr_removes; + ats[nbr_removes] = truncstart; + defaulttype = types[nbr_removes]; + } + + timecnt -= nbr_removes; + timei += nbr_removes; + } + /* ** Remove old file, if any, to snap links. */ @@ -1910,7 +1956,7 @@ writezone(const char *const name, const char *const string, char version, thisleapi = leapi32; thisleapcnt = leapcnt32; } else { - thistimei = 0; + thistimei = timei; thistimecnt = timecnt; toomanytimes = thistimecnt >> 31 >> 31 >> 2 != 0; thisleapi = 0; -- 2.11.0
Thanks for suggesting that. I looked into the issue, and it can be a trickier to truncate the output because truncation also affects things like the default time type and the TZ string, and truncation might affect the 32-bit data (which is already truncated) differently from the 64-bit data. I wrote a different implementation, which supports truncating newer timestamps as well, as per Internet RFC 8536. Please give the attached proposed patch a try.
Hi Paul, I rebased to the latest, since your patch was already merged so I used it to test a bit. I am a bit confused about the result, maybe you can explain to me: ### Rule from africa: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Africa/Abidjan -0:16:08 - LMT 1912 0:00 - GMT ### Compiled data using @220924800 should correspond to 1977 for tz in etcetera southamerica northamerica europe africa antarctica \ asia australasia backward systemv factory; do ./zic -L /dev/null -d $ZONEINFO -r @220924800 -y "sh yearistype.sh" ${tz} done ./zdump -v $ZONEINFO/Africa/Abidjan ### Got the following output: /home/christwo/dists/tz/test_zic_github/Africa/Abidjan -9223372036854775808 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan -9223372036854689408 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan Fri Dec 31 23:59:59 1976 UT = Fri Dec 31 23:43:51 1976 LMT isdst=0 gmtoff=-968 /home/christwo/dists/tz/test_zic_github/Africa/Abidjan Sat Jan 1 00:00:00 1977 UT = Sat Jan 1 00:00:00 1977 GMT isdst=0 gmtoff=0 /home/christwo/dists/tz/test_zic_github/Africa/Abidjan 9223372036854689407 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan 9223372036854775807 = NULL ### Compiled data using @441763200 should correspond to 1984 for tz in etcetera southamerica northamerica europe africa antarctica \ asia australasia backward systemv factory; do ./zic -L /dev/null -d $ZONEINFO -r @441763200 -y "sh yearistype.sh" ${tz} done ./zdump -v $ZONEINFO/Africa/Abidjan ### Got the following output: /home/christwo/dists/tz/test_zic_github/Africa/Abidjan -9223372036854775808 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan -9223372036854689408 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan Sat Dec 31 23:59:59 1983 UT = Sat Dec 31 23:43:51 1983 LMT isdst=0 gmtoff=-968 /home/christwo/dists/tz/test_zic_github/Africa/Abidjan Sun Jan 1 00:00:00 1984 UT = Sun Jan 1 00:00:00 1984 GMT isdst=0 gmtoff=0 /home/christwo/dists/tz/test_zic_github/Africa/Abidjan 9223372036854689407 = NULL /home/christwo/dists/tz/test_zic_github/Africa/Abidjan 9223372036854775807 = NULL Is it correct that the switch between LMT to GMT is set to the specified start of the time range? I also tried with the time range "-r @220924800/@441763200" then I encountered some segmentation fault error prints when I ran that. I haven't investigated any further on that. BR, Christopher Wong ________________________________________ From: Paul Eggert <eggert@cs.ucla.edu> Sent: Saturday, March 9, 2019 06:34 To: Christopher Wong; Scott Nelson Cc: Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward Thanks for suggesting that. I looked into the issue, and it can be a trickier to truncate the output because truncation also affects things like the default time type and the TZ string, and truncation might affect the 32-bit data (which is already truncated) differently from the 64-bit data. I wrote a different implementation, which supports truncating newer timestamps as well, as per Internet RFC 8536. Please give the attached proposed patch a try.
On 3/11/19 4:18 AM, Christopher Wong wrote:
Is it correct that the switch between LMT to GMT is set to the specified start of the time range?
Yes. If you tell zic's new -r option to discard info for timestamps before time T, then the first transition in the output TZif file will be at time T. This is so that a TZif reader can determine the range of timestamps that the TZif file covers. See Internet RFC 8536 section 5.1.
I also tried with the time range "-r @220924800/@441763200" then I encountered ome segmentation fault error prints when I ran that. I haven't investigated any further on that.
I couldn't reproduce the problem. Can you give a recipe for reproducing it on your end, along with your system configuration? Here's how I tried to reproduce it, using gcc 8.3.1 20190223 (Red Hat 8.3.1-2) on Fedora 29 x86-64: make clean make TOPDIR=/tmp/tz CFLAGS='-fsanitize=address -g3 -O2' ZFLAGS='-r @220924800/@441763200' install ./zdump -i Africa/Abidjan America/Los_Angeles
On Mon, 11 Mar 2019 at 18:30, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 3/11/19 4:18 AM, Christopher Wong wrote:
Is it correct that the switch between LMT to GMT is set to the specified start of the time range?
Yes. If you tell zic's new -r option to discard info for timestamps before time T, then the first transition in the output TZif file will be at time T. This is so that a TZif reader can determine the range of timestamps that the TZif file covers. See Internet RFC 8536 section 5.1.
Yes, but in the examples given which truncate from 1977 and 1984, that shouldn't be a transition from LMT, as LMT was not observed in that Zone since 1912: "As with untruncated TZif files, time type 0 indicates local time immediately before the start point, and the time type of the first transition indicates local time thereafter." So if my truncation range starts with POSIX timestamp N, I would expect time type zero to correctly and accurately represent timestamp N−1 (though, of course, not necessarily N−2 or any earlier). In each of these examples, that's GMT, not LMT. The file would then have the required transition at time N, though in these cases, that would be a no-op transition to the same time type, as it's still GMT. -- Tim Parenti
On 3/11/19 8:35 PM, Tim Parenti wrote:
So if my truncation range starts with POSIX timestamp N, I would expect time type zero to correctly and accurately represent timestamp N−1 (though, of course, not necessarily N−2 or any earlier).
Thanks for catching that bug. I installed the attached patch.
On Thu, 14 Mar 2019 at 12:26, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 3/11/19 8:35 PM, Tim Parenti wrote:
So if my truncation range starts with POSIX timestamp N, I would expect time type zero to correctly and accurately represent timestamp N−1 (though, of course, not necessarily N−2 or any earlier).
Thanks for catching that bug. I installed the attached patch.
Thanks, though I should note I have a quibble with a small assertion in your patch notes: It’s nicer (though not required by RFC 8536) that when the start of a TZif
file is truncated, the default time type is that of timestamps just before the first transition.
Section 5.1 of RFC 8536 requires that "[a]ll represented information that falls inside the truncation range MUST be the same as that represented by a corresponding untruncated TZif file." By not making the default time type that of timestamps just before the range starts at time N, this creates a false and inaccurate transition at time N. By ensuring that timestamp N−1 is accurately represented, either a correct transition is inserted at time N (if a transition does indeed normally exist at time N), or a fictitious-though-accurate no-op transition is inserted at time N. Either way, the data representing the transition at time N would then be correct. Therefore, it is certainly my understanding that such behavior is indeed required by RFC 8536. And yes, that is very much because it's nicer. -- Tim Parenti
On 3/14/19 5:31 PM, Tim Parenti wrote:
I have a quibble with a small assertion in your patch notes:
It’s nicer (though not required by RFC 8536) that when the start of a TZif file is truncated, the default time type is that of timestamps just before the first transition.
Section 5.1 of RFC 8536 requires that "[a]ll represented information that falls inside the truncation range MUST be the same as that represented by a corresponding untruncated TZif file."
Yes, my comment was intended to refer only to the default time type in the 32-bit data. Sorry, my comment did not make that intent clear - and I don't now recall whether the default time type in the 64-bit area was also buggy, but it shouldn't matter now since it should be fixed now. You're right about the default time type in the 64-bit data, which is what RFC 8536 section 5.1 is talking about. Section 5.1 deliberately does not place a similar requirement on the 32-bit data; this is so that zic can still cater to buggy old 32-bit clients.
My gcc version is: gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 Copyright (C) 2016 Free Software Foundation, Inc. I just used a script to compile the database without any flags: #! /bin/sh ZONEINFO=/home/christwo/dists/tz/test_zic_github gcc zic.c -o zic for tz in etcetera southamerica northamerica europe africa antarctica \ asia australasia backward systemv factory; do ./zic -L /dev/null -d $ZONEINFO -r @220924800/@441763200 -y "sh yearistype.sh" ${tz} done ./zdump -v $ZONEINFO/Africa/Abidjan If I do: make clean make ... install as you specified, then I don't see the segmentation faults either. I guess, I just didn't do it the right way. BR, Christopher Wong ________________________________________ From: Paul Eggert <eggert@cs.ucla.edu> Sent: Monday, March 11, 2019 23:29 To: Christopher Wong Cc: Scott Nelson; Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward On 3/11/19 4:18 AM, Christopher Wong wrote:
Is it correct that the switch between LMT to GMT is set to the specified start of the time range?
Yes. If you tell zic's new -r option to discard info for timestamps before time T, then the first transition in the output TZif file will be at time T. This is so that a TZif reader can determine the range of timestamps that the TZif file covers. See Internet RFC 8536 section 5.1.
I also tried with the time range "-r @220924800/@441763200" then I encountered ome segmentation fault error prints when I ran that. I haven't investigated any further on that.
I couldn't reproduce the problem. Can you give a recipe for reproducing it on your end, along with your system configuration? Here's how I tried to reproduce it, using gcc 8.3.1 20190223 (Red Hat 8.3.1-2) on Fedora 29 x86-64: make clean make TOPDIR=/tmp/tz CFLAGS='-fsanitize=address -g3 -O2' ZFLAGS='-r @220924800/@441763200' install ./zdump -i Africa/Abidjan America/Los_Angeles
On 3/12/19 5:01 AM, Christopher Wong wrote:
If I do: make clean make ... install as you specified, then I don't see the segmentation faults either. I guess, I just didn't do it the right way.
No, your approach should have worked, even though -y is obsolescent and your use of it had no effect and we should remove -y one of these days. I installed the attached patch to fix the bug that you found. Thanks for reporting it.
I rebased to the latest and did some testing. I didn't got any more segmentation fault, so that is fixed. However, I encountered something else related to the truncation. I compiled with the following range 1977-1984 : ./zic -L /dev/null -d $ZONEINFO -r @220924800/@441763200 ${tz} ./zdump -v $ZONEINFO/Australia/Perth The result I got was: /home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854775808 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854689408 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 17:59:59 1983 UT = Sun Oct 30 01:59:59 1983 AWST isdst=0 gmtoff=28800 /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 18:00:00 1983 UT = Sun Oct 30 03:00:00 1983 AWDT isdst=1 gmtoff=32400 /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854689407 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854775807 = NULL I was expecting many dates before 1983. I made a hexdump from the Australia/Perth TZif file: 00000000 54 5a 69 66 32 00 00 00 00 00 00 00 00 00 00 00 |TZif2...........| 00000010 00 00 00 00 00 00 00 02 00 00 00 02 00 00 00 00 |................| 00000020 00 00 00 03 00 00 00 02 00 00 00 0a 0d 2b 0b 80 |.............+..| 00000030 1a 01 62 a0 1a 54 c5 81 00 01 01 00 00 70 80 00 |..b..T.......p..| 00000040 00 00 00 7e 90 01 05 41 57 53 54 00 41 57 44 54 |...~...AWST.AWDT| 00000050 00 01 01 00 00 54 5a 69 66 32 00 00 00 00 00 00 |.....TZif2......| 00000060 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 |................| 00000070 02 00 00 00 00 00 00 00 03 00 00 00 02 00 00 00 |................| 00000080 0a 00 00 00 00 0d 2b 0b 80 00 00 00 00 1a 01 62 |......+........b| 00000090 a0 00 00 00 00 1a 54 c5 81 00 01 01 00 00 70 80 |......T.......p.| 000000a0 00 00 00 00 7e 90 01 05 41 57 53 54 00 41 57 44 |....~...AWST.AWD| 000000b0 54 00 01 01 00 00 0a 0a |T.......| 000000b8
From it I extracted the timestamps below, should the last transition be 1s after range end?
0d 2b 0b 80 220924800 Saturday, January 1, 1977 0:00:00 1a 01 62 a0 436298400 Saturday, October 29, 1983 18:00:00 1a 54 c5 81 441763201 Sunday, January 1, 1984 0:00:01 According to RFC 8536: When truncating the end of a TZif file, the service MUST supply in the version 2+ data a last transition time that is the end point of the truncation range I was kind of expecting the last transition to be within my supplied range or am I misinterpreting the RFC? BR, Christopher Wong ________________________________________ From: Paul Eggert <eggert@cs.ucla.edu> Sent: Thursday, March 14, 2019 17:30 To: Christopher Wong Cc: Scott Nelson; Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward On 3/12/19 5:01 AM, Christopher Wong wrote:
If I do: make clean make ... install as you specified, then I don't see the segmentation faults either. I guess, I just didn't do it the right way.
No, your approach should have worked, even though -y is obsolescent and your use of it had no effect and we should remove -y one of these days. I installed the attached patch to fix the bug that you found. Thanks for reporting it.
On 3/14/19 7:13 PM, Christopher Wong wrote:
I rebased to the latest and did some testing. I didn't got any more segmentation fault, so that is fixed.
However, I encountered something else related to the truncation. I compiled with the following range 1977-1984 :
./zic -L /dev/null -d $ZONEINFO -r @220924800/@441763200 ${tz}
./zdump -v $ZONEINFO/Australia/Perth
The result I got was:
/home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854775808 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854689408 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 17:59:59 1983 UT = Sun Oct 30 01:59:59 1983 AWST isdst=0 gmtoff=28800 /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 18:00:00 1983 UT = Sun Oct 30 03:00:00 1983 AWDT isdst=1 gmtoff=32400 /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854689407 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854775807 = NULL
I was expecting many dates before 1983.
But the range you specified (@220924800/@441763200) is from 1977-01-01 00:00 UTC through 1984-01-01 00:00 UTC. Perth saw only one transition during that period, namely, the 1983-10-29 transition listed above. So this zdump output is correct, no? zdump by design does not output no-op transitions like the transitions at 220924800 and at 441763200 in that TZif file.
I extracted the timestamps below, should the last transition be 1s after range end?
0d 2b 0b 80 220924800 Saturday, January 1, 1977 0:00:00 1a 01 62 a0 436298400 Saturday, October 29, 1983 18:00:00 1a 54 c5 81 441763201 Sunday, January 1, 1984 0:00:01
This was due to an off-by-one discrepancy between the zic -r syntax and the RFC 8536 truncation rules. 'zic -r /@T' meant to truncate *after* time T, whereas RFC 8536 means to truncate *before* time T (this is as per RFC 7808 section 3.9). Thanks for pointing out this off-by-one discrepancy. It's better for zic to be consistent with the RFCs, so I installed the attached patch to remove the discrepancy, and also to check more carefully for overflow and for empty ranges.
Hi Paul, Regarding the range 1977-1984. Sorry, I thought the start range was 1877, so you are absolutely right that the transition is correct. Hope I didn't make you double check this for too long. Thanks for adding this feature and fixing related bugs so quickly! BR, Christopher Wong ________________________________________ From: Paul Eggert <eggert@cs.ucla.edu> Sent: Friday, March 15, 2019 17:45 To: Christopher Wong Cc: Scott Nelson; Time Zone Mailing List Subject: Re: [tz] [PROPOSED] zic option for including data from a certain year onward On 3/14/19 7:13 PM, Christopher Wong wrote:
I rebased to the latest and did some testing. I didn't got any more segmentation fault, so that is fixed.
However, I encountered something else related to the truncation. I compiled with the following range 1977-1984 :
./zic -L /dev/null -d $ZONEINFO -r @220924800/@441763200 ${tz}
./zdump -v $ZONEINFO/Australia/Perth
The result I got was:
/home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854775808 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth -9223372036854689408 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 17:59:59 1983 UT = Sun Oct 30 01:59:59 1983 AWST isdst=0 gmtoff=28800 /home/christwo/dists/tz/test_zic_github/Australia/Perth Sat Oct 29 18:00:00 1983 UT = Sun Oct 30 03:00:00 1983 AWDT isdst=1 gmtoff=32400 /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854689407 = NULL /home/christwo/dists/tz/test_zic_github/Australia/Perth 9223372036854775807 = NULL
I was expecting many dates before 1983.
But the range you specified (@220924800/@441763200) is from 1977-01-01 00:00 UTC through 1984-01-01 00:00 UTC. Perth saw only one transition during that period, namely, the 1983-10-29 transition listed above. So this zdump output is correct, no? zdump by design does not output no-op transitions like the transitions at 220924800 and at 441763200 in that TZif file.
I extracted the timestamps below, should the last transition be 1s after range end?
0d 2b 0b 80 220924800 Saturday, January 1, 1977 0:00:00 1a 01 62 a0 436298400 Saturday, October 29, 1983 18:00:00 1a 54 c5 81 441763201 Sunday, January 1, 1984 0:00:01
This was due to an off-by-one discrepancy between the zic -r syntax and the RFC 8536 truncation rules. 'zic -r /@T' meant to truncate *after* time T, whereas RFC 8536 means to truncate *before* time T (this is as per RFC 7808 section 3.9). Thanks for pointing out this off-by-one discrepancy. It's better for zic to be consistent with the RFCs, so I installed the attached patch to remove the discrepancy, and also to check more carefully for overflow and for empty ranges.
participants (4)
-
Christopher Wong -
Paul Eggert -
Scott Nelson -
Tim Parenti