So it looks as if this has nothing to do with non-normalized values and, actually, nothing to do with mktime but everything with to do with how the TZ variable is being handled. At the bottom of this message there's output from "zdump." It shows that, for TZ=EST5EDT,M4.1.0/02:00,M10.5.0/02:00, the first detectable transition is on 1970-10-25, from EDT to EST; that means that daylight saving is applied to all instants before 1970-10-25, leading to the reported results. The small fix is to ensure that a transition to DST shows up in the table that's generated at run time when the given TZ environment variable is used. The philosophical question: for such environment variables, in what year should application of daylight saving start (or should it apply to all years, no matter how far in the past)? @dashdashado Script started on Wed, Jan 11, 2017 2:24:59 PM $ zdump -v EST5EDT,M4.1.0/02:00,M10.5.0/02:00 | head EST5EDT,M4.1.0/02:00,M10.5.0/02:00 -9223372036854775808 = NULL EST5EDT,M4.1.0/02:00,M10.5.0/02:00 -9223372036854689408 = NULL EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 25 05:59:59 1970 UT = Sun Oct 25 01:59:59 1970 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 25 06:00:00 1970 UT = Sun Oct 25 01:00:00 1970 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 4 06:59:59 1971 UT = Sun Apr 4 01:59:59 1971 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 4 07:00:00 1971 UT = Sun Apr 4 03:00:00 1971 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 31 05:59:59 1971 UT = Sun Oct 31 01:59:59 1971 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 31 06:00:00 1971 UT = Sun Oct 31 01:00:00 1971 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 2 06:59:59 1972 UT = Sun Apr 2 01:59:59 1972 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 2 07:00:00 1972 UT = Sun Apr 2 03:00:00 1972 EDT isdst=1 gmtoff=-14400 $ exit exit Script done on Wed, Jan 11, 2017 2:25:31 PM On Wed, Jan 11, 2017 at 11:27 AM, Kees Dekker <Kees.Dekker@infor.com> wrote:

Hi,

Can anyone help me?

In short, I set TZ to EST5EDT,M4.1.0/02:00,M10.5.0/02:00, which means daylight saving for EST (=GMT-5 or GMT-4 if DST is active), with switching DST at first Sunday of April and move to standard time at the last Sunday of October.

The input to mktime is (See also attached source code) 1-1-1970, 19:00h (tm_min=1140 = 1140/60 = 19h) local EST time and tm_isdst set to -1. Because of this value, the tz code should figure out that 1-1-1970 is not in DST so the time difference with GMT/UTC is 5 hours. I would exect then mktime() to return 0. This is true if I use the system functions, but mktime() returns -3600 when using the tz library.

The output of running the test case with system libraries is:

./tztestcase

enter mktime with: 1970/Jan/00 00:1140:00, week day: -858993460, year day: -858993460, dst: -1

the return value of mktime is: 0

normalized times: 1969/Dec/31 19:00:00, week day: Wed, year day: 364, dst: 0

The output of the test case with the tz lib is:

./tztestcase2016j

enter mktime with: 1970/Jan/00 00:1140:00, week day: -858993460, year day: -858993460, dst: -1

the return value of mktime is: -3600

normalized times: 1969/Dec/31 19:00:00, week day: Wed, year day: 364, dst: 1

Please correct me if I’m wrong. Is this a bug in the tz code? Please note the differences.

*From:* Kees Dekker *Sent:* Thursday, January 05, 2017 15:03 *To:* 'tz@iana.org' <tz@iana.org> *Subject:* question about mktime_tzname()

Hi,

I’m running into a potential bug. Please also see the attached C-source file for a reproduction.

Reproduction steps:

1. Build tz code, in such way that a libtz.a exists

2. Compile attached source with (e.g. on Linux): gcc -g -o tztestcase2016j tztestcase.c -L <tzlibdir> -ltz

3. Checking the same test case source code with a system library (instead of IANA libtz.a) can be achieved by omitting -L and -l flags in previous command (see #2).

It does not make much sense which version of tzcode is used, I tried 2013i and 2016j and both show the same behavior.

The mktime() code returns -3600, while I expect 0. Because the TZ zone is explicitly specified, EST5EDT is GMT-5. So the provided time stamp (31-Jan-1969, 19:00) is exactly the same as UTC value 0 (1-Jan-1970, 00:00 GMT). Both Linux and Windows with system libraries return 0 in this specific case (I used these for reference).

The remainder of this email is based on the tzcode2016j source code (e.g. for used line numbers).

The provided TZ value is parsed in tzparse() and results in sp->ttis[0] set as DST value, and sp->ttis[1] as standard time value. This is at least contrary to in the else part of the if (*name == ',' || *name == ';') block (there DST is saved in sp->ttis[1]). Just grep on init_ttinfo() calls in localtime.c. However, in localsub(), around line 1428, the i becomes sp->defaulttype (which is zero), i.e. points to the DST value of sp->ttis[], which is assigned to ttisp. As result, because t is below sp->ats[0], DST takes into effect, which is incorrect, as everything after the last Sunday in October and before the first Sunday in April (see attachment, TZ=”EST5EDT,M4.1.0/02:00,M10.5.0/02:00”) is assumed to be standard time.

I’m not sure whether I draw the right conclusion, but because the incorrect assumption of using DST, mktime() returns -3600 (on hour off) instead of the expected 0. Is this conclusion correct?

I’m not sure how to fix it. When changing the call to init_ttinfo() and effectively swapping DST and standard time (also for symmetry with the else part) in sp-ttis[] (0 to 1 and 1 to 0) + keeping sp->defaulttype = 0 solved this problem, but introduces other conversion errors around DSTchanges (I don’t have yet an reproduction, but can make one if needed). The sp->default type is used (in localsub ()) when if (sp->timecnt == 0 || t < sp->ats[0]). The other approach is to assign sp->defaulttype = 1 when the init_ttinfo() calls are not swapped. Anyhow, there is a tight relationship between the value of sp->defaulttype and the order in which DST/standard times are written in sp->ttis[].

If the previous idea is incorrect (and I guess it is), additional code may be needed in time2sub() if yourtm.tm_isdst == -1. In that case, the provided data in yourtm should be checked whether DST is (or is not) into effect. The found mytm struct should match. Otherwise a DST correction is needed (one-hour shift). The current code mistakenly uses DST, and that is why (if I understand all well) mktime() now returns -3600.

If I was not able to explain the issue correctly, please let me know. I’m not sure to understand what’s going wrong now, but may be, the provided attachment will help in giving better understanding. Because the provided date is near 1970, quite few steps are needed to see what’s happing in localtime.c.

Regards,

Kees Dekker

PS. This email has been sent assuming that attachments will not get lost. Otherwise, just copy paste the code below and save to e.g. tztestcase.c.

#ifdef _WIN32

#define _CRT_SECURE_NO_WARNINGS

#endif

#include <time.h>

#include <stdio.h>

#include <stdlib.h>

int main(void)

{

int ret = 0;

const char *months[] = { "Jan", "Feb", "Mar", "April", "May", "June", "July", "Aug", "Sep", "Oct", "Nov", "Dec" };

const char *days[] = { "Sun", "Mon", "Tue", "Wed", "Tur", "Fri", "Sat" };

/* DST switch in April in first Sunday and October, last Sunday */

putenv("TZ=EST5EDT,M4.1.0/02:00,M10.5.0/02:00");

/* January, 1970, invalid input for tm_mday (0 instead 1..31) */

/* non-normalized input for minutes: 1140/60 = 19, so 19:00 EST5EDT time, which is 5 hours before GMT, and should result in t == 0 below */

/* let system figure out whether DST is effect */

struct tm tmp = { 0, 1140, 0, 0, 0, 70, -858993460, -858993460, -1};

printf("enter mktime with: %04d/%s/%02d %02d:%02d:%02d, week day: %d, year day: %d, dst: %d\n",

tmp.tm_year + 1900, months[tmp.tm_mon], tmp.tm_mday,

tmp.tm_hour, tmp.tm_min, tmp.tm_sec,

tmp.tm_wday, tmp.tm_yday, tmp.tm_isdst);

time_t t = mktime(&tmp);

/* expect: t == 0, DST is off, time is exactly at the boundary of lowest possible UTC value */

printf("the return value of mktime is: %lld\n", (long long)t);

printf("normalized times: %04d/%s/%02d %02d:%02d:%02d, week day: %s, year day: %d, dst: %d\n",

tmp.tm_year + 1900, months[tmp.tm_mon], tmp.tm_mday,

tmp.tm_hour, tmp.tm_min, tmp.tm_sec,

days[tmp.tm_wday], tmp.tm_yday, tmp.tm_isdst);

if (t >= 0) {

printf("tz=%s, %s\n", tzname[0], tzname[1]);

}

return 0;

}

Many thanks for reply. I will check your reply with a colleague tomorrow. He is knowing more details than I. I only don't understand why system libraries (including Linux, that also uses AFAIK tz code for the system libraries) think that DST is not into effect. Based on my debugging tests, the initialization order resulted in the assumption that DST was on (sp->defaulttype). If EST is at the northern part of the world, I would expect that DST starts in april and end in 1970-10-25. But I have to do more checking tomorrow (I'm typing this email from home). I've some feeling, but I've to do more research, that the detection of DST is incorrect at the boundary (and probably theoretical cases). The reason why we use this old dates (it is not very old, otherwise, I'm also very old :-)) is to check our own software. Our own software provides interfaces to programmer's, and checking the boundaries of the UTC ranges is part of a test. I agree that these tests are probably beyond any practical case. But our customers may have used it in the past, and so we have to remain compatible. That's why we have tests that prove the past behavior, even if I update the tz sour code and/or data/rules in our product. I've much more failing tests in our test-suite, that worked well before (with older tz code + most probably own modification). My goal is just to eliminate our own modifications and just the IANA code as is. Any other changes that are probably useful for IANA will be sent to this mailing list one I have isolated them (and still find them useful, probably a faster algorithm for the binary search). ________________________________ Van: Arthur David Olson [arthurdavidolson@gmail.com] Verzonden: woensdag 11 januari 2017 20:39 Aan: Kees Dekker CC: tz@iana.org Onderwerp: Re: [tz] question about mktime_tzname() So it looks as if this has nothing to do with non-normalized values and, actually, nothing to do with mktime but everything with to do with how the TZ variable is being handled. At the bottom of this message there's output from "zdump." It shows that, for TZ=EST5EDT,M4.1.0/02:00,M10.5.0/02:00, the first detectable transition is on 1970-10-25, from EDT to EST; that means that daylight saving is applied to all instants before 1970-10-25, leading to the reported results. The small fix is to ensure that a transition to DST shows up in the table that's generated at run time when the given TZ environment variable is used. The philosophical question: for such environment variables, in what year should application of daylight saving start (or should it apply to all years, no matter how far in the past)? @dashdashado Script started on Wed, Jan 11, 2017 2:24:59 PM $ zdump -v EST5EDT,M4.1.0/02:00,M10.5.0/02:00 | head EST5EDT,M4.1.0/02:00,M10.5.0/02:00 -9223372036854775808 = NULL EST5EDT,M4.1.0/02:00,M10.5.0/02:00 -9223372036854689408 = NULL EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 25 05:59:59 1970 UT = Sun Oct 25 01:59:59 1970 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 25 06:00:00 1970 UT = Sun Oct 25 01:00:00 1970 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 4 06:59:59 1971 UT = Sun Apr 4 01:59:59 1971 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 4 07:00:00 1971 UT = Sun Apr 4 03:00:00 1971 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 31 05:59:59 1971 UT = Sun Oct 31 01:59:59 1971 EDT isdst=1 gmtoff=-14400 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Oct 31 06:00:00 1971 UT = Sun Oct 31 01:00:00 1971 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 2 06:59:59 1972 UT = Sun Apr 2 01:59:59 1972 EST isdst=0 gmtoff=-18000 EST5EDT,M4.1.0/02:00,M10.5.0/02:00 Sun Apr 2 07:00:00 1972 UT = Sun Apr 2 03:00:00 1972 EDT isdst=1 gmtoff=-14400 $ exit exit Script done on Wed, Jan 11, 2017 2:25:31 PM On Wed, Jan 11, 2017 at 11:27 AM, Kees Dekker <Kees.Dekker@infor.com<mailto:Kees.Dekker@infor.com>> wrote: Hi, Can anyone help me? In short, I set TZ to EST5EDT,M4.1.0/02:00,M10.5.0/02:00, which means daylight saving for EST (=GMT-5 or GMT-4 if DST is active), with switching DST at first Sunday of April and move to standard time at the last Sunday of October. The input to mktime is (See also attached source code) 1-1-1970, 19:00h (tm_min=1140 = 1140/60 = 19h) local EST time and tm_isdst set to -1. Because of this value, the tz code should figure out that 1-1-1970 is not in DST so the time difference with GMT/UTC is 5 hours. I would exect then mktime() to return 0. This is true if I use the system functions, but mktime() returns -3600 when using the tz library. The output of running the test case with system libraries is: ./tztestcase enter mktime with: 1970/Jan/00 00:1140:00, week day: -858993460, year day: -858993460, dst: -1 the return value of mktime is: 0 normalized times: 1969/Dec/31 19:00:00, week day: Wed, year day: 364, dst: 0 The output of the test case with the tz lib is: ./tztestcase2016j enter mktime with: 1970/Jan/00 00:1140:00, week day: -858993460, year day: -858993460, dst: -1 the return value of mktime is: -3600 normalized times: 1969/Dec/31 19:00:00, week day: Wed, year day: 364, dst: 1 Please correct me if I’m wrong. Is this a bug in the tz code? Please note the differences. From: Kees Dekker Sent: Thursday, January 05, 2017 15:03 To: 'tz@iana.org<mailto:tz@iana.org>' <tz@iana.org<mailto:tz@iana.org>> Subject: question about mktime_tzname() Hi, I’m running into a potential bug. Please also see the attached C-source file for a reproduction. Reproduction steps: 1. Build tz code, in such way that a libtz.a exists 2. Compile attached source with (e.g. on Linux): gcc -g -o tztestcase2016j tztestcase.c -L <tzlibdir> -ltz 3. Checking the same test case source code with a system library (instead of IANA libtz.a) can be achieved by omitting -L and -l flags in previous command (see #2). It does not make much sense which version of tzcode is used, I tried 2013i and 2016j and both show the same behavior. The mktime() code returns -3600, while I expect 0. Because the TZ zone is explicitly specified, EST5EDT is GMT-5. So the provided time stamp (31-Jan-1969, 19:00) is exactly the same as UTC value 0 (1-Jan-1970, 00:00 GMT). Both Linux and Windows with system libraries return 0 in this specific case (I used these for reference). The remainder of this email is based on the tzcode2016j source code (e.g. for used line numbers). The provided TZ value is parsed in tzparse() and results in sp->ttis[0] set as DST value, and sp->ttis[1] as standard time value. This is at least contrary to in the else part of the if (*name == ',' || *name == ';') block (there DST is saved in sp->ttis[1]). Just grep on init_ttinfo() calls in localtime.c. However, in localsub(), around line 1428, the i becomes sp->defaulttype (which is zero), i.e. points to the DST value of sp->ttis[], which is assigned to ttisp. As result, because t is below sp->ats[0], DST takes into effect, which is incorrect, as everything after the last Sunday in October and before the first Sunday in April (see attachment, TZ=”EST5EDT,M4.1.0/02:00,M10.5.0/02:00”) is assumed to be standard time. I’m not sure whether I draw the right conclusion, but because the incorrect assumption of using DST, mktime() returns -3600 (on hour off) instead of the expected 0. Is this conclusion correct? I’m not sure how to fix it. When changing the call to init_ttinfo() and effectively swapping DST and standard time (also for symmetry with the else part) in sp-ttis[] (0 to 1 and 1 to 0) + keeping sp->defaulttype = 0 solved this problem, but introduces other conversion errors around DSTchanges (I don’t have yet an reproduction, but can make one if needed). The sp->default type is used (in localsub ()) when if (sp->timecnt == 0 || t < sp->ats[0]). The other approach is to assign sp->defaulttype = 1 when the init_ttinfo() calls are not swapped. Anyhow, there is a tight relationship between the value of sp->defaulttype and the order in which DST/standard times are written in sp->ttis[]. If the previous idea is incorrect (and I guess it is), additional code may be needed in time2sub() if yourtm.tm_isdst == -1. In that case, the provided data in yourtm should be checked whether DST is (or is not) into effect. The found mytm struct should match. Otherwise a DST correction is needed (one-hour shift). The current code mistakenly uses DST, and that is why (if I understand all well) mktime() now returns -3600. If I was not able to explain the issue correctly, please let me know. I’m not sure to understand what’s going wrong now, but may be, the provided attachment will help in giving better understanding. Because the provided date is near 1970, quite few steps are needed to see what’s happing in localtime.c. Regards, Kees Dekker PS. This email has been sent assuming that attachments will not get lost. Otherwise, just copy paste the code below and save to e.g. tztestcase.c. #ifdef _WIN32 #define _CRT_SECURE_NO_WARNINGS #endif #include <time.h> #include <stdio.h> #include <stdlib.h> int main(void) { int ret = 0; const char *months[] = { "Jan", "Feb", "Mar", "April", "May", "June", "July", "Aug", "Sep", "Oct", "Nov", "Dec" }; const char *days[] = { "Sun", "Mon", "Tue", "Wed", "Tur", "Fri", "Sat" }; /* DST switch in April in first Sunday and October, last Sunday */ putenv("TZ=EST5EDT,M4.1.0/02:00,M10.5.0/02:00"); /* January, 1970, invalid input for tm_mday (0 instead 1..31) */ /* non-normalized input for minutes: 1140/60 = 19, so 19:00 EST5EDT time, which is 5 hours before GMT, and should result in t == 0 below */ /* let system figure out whether DST is effect */ struct tm tmp = { 0, 1140, 0, 0, 0, 70, -858993460, -858993460, -1}; printf("enter mktime with: %04d/%s/%02d %02d:%02d:%02d, week day: %d, year day: %d, dst: %d\n", tmp.tm_year + 1900, months[tmp.tm_mon], tmp.tm_mday, tmp.tm_hour, tmp.tm_min, tmp.tm_sec, tmp.tm_wday, tmp.tm_yday, tmp.tm_isdst); time_t t = mktime(&tmp); /* expect: t == 0, DST is off, time is exactly at the boundary of lowest possible UTC value */ printf("the return value of mktime is: %lld\n", (long long)t); printf("normalized times: %04d/%s/%02d %02d:%02d:%02d, week day: %s, year day: %d, dst: %d\n", tmp.tm_year + 1900, months[tmp.tm_mon], tmp.tm_mday, tmp.tm_hour, tmp.tm_min, tmp.tm_sec, days[tmp.tm_wday], tmp.tm_yday, tmp.tm_isdst); if (t >= 0) { printf("tz=%s, %s\n", tzname[0], tzname[1]); } return 0; }

For the immediate question:

I'm not quite sure what 'goback' and 'goahead' are supposed to do.

"goback" indicates that there's a 400-year pattern to the stuff at the start of the transition table that can be extrapolated backward through time; "goahead" indicates that there's a 400-year pattern to the stuff at the end that can be extrapolated forward. When a table generated from an environment variable does indeed cover 400 years, both goahead and goback should be set. @dashdashado On Wed, Jan 11, 2017 at 3:55 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 01/11/2017 11:39 AM, Arthur David Olson wrote:

...

for such environment variables, in what year should application of daylight saving start (or should it apply to all years, no matter how far in the past)?

Surely it should apply to all years. POSIX doesn't require this (POSIX specifies only behavior starting in 1970) but it would be odd for a permanent rule to spring into action only after 1969.

The attached proposed patch does that, and fixes the problem for Kees Dekker's case. Does this look like the right thing in general? I'm not quite sure what 'goback' and 'goahead' are supposed to do.

Unfortunately now that I think of it, I see that this can't work on hosts with 32-bit time_t, as they can't cover the 400-year span. So the code is still broken on 32-bit hosts. Is there a better way to fix the code?

Since 32-bit time_t values can only cover (about) 140 years, not covering a 400-year span's not a bug, it's a given.-) (An aside: in the future, my guess is that there'll either be a new, 64-bit-only time zone binary format or that time zone source will be interpreted at run time with no binary format; at that point, a lot of code simplification happens.) @dashdashado On Wed, Jan 11, 2017 at 4:51 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 01/11/2017 01:28 PM, Arthur David Olson wrote:

...

When a table generated from an environment variable does indeed cover 400 years, both goahead and goback should be set.

The localtime.c code in question tries to do that sort of thing, as it says:

yearlim = EPOCH_YEAR + YEARSPERREPEAT; for (year = EPOCH_YEAR; year < yearlim; year++) {

and YEARSPERREPEAT is 400.

Unfortunately now that I think of it, I see that this can't work on hosts with 32-bit time_t, as they can't cover the 400-year span. So the code is still broken on 32-bit hosts. Is there a better way to fix the code?

Also, I'm puzzled as to why the first transition in 1970 was being omitted, as that leads me to worry that the above code will generate one transition shy of 400 years' worth even on 64-bit hosts.

Kees.Dekker@infor.com said: | I don?t understand how 32-bit time_t systems can survive 2038 There are proposals (even implemented, though I am not sure how widely) to alter the range of 32 bit time_t, 0 is still Jan 1, 1970, but from that positive values range from (say) 1..BFFFFFFF and C0000000..FFFFFFFF are negative. (The actual cutover point seems to vary). That one would extend the usefulness of 32 bit time_t values until 2072 (or perhaps beyond, depending upon the switch point) and limiting the earliest value to 1936 (or whatever) isn't likely to seriously bother anyone. If the switch point were F0000000 (so EFFFFFFF is the biggest positive value) the range of timestamps would be from mid 1961 to mid 2097, which would probably be OK, and gives lots of time for everything to be converted. Note that processing isn't the issue (the 32 bit time_t values can be moved to 64 bit variables, represented as regular 2's complement binary numbers, for that), but converting all the places that time_t's get stored is hard. Certainly everything new should really be using 64 bit values by now. NetBSD was (I believe) one of the earliest systems to switch to a 64 bit time_t (years ago now) - yet we still have to support people with filesystems created even longer ago, that still have only 32 bit spaces for the seconds part of the timestamps, and which are still in use (new/updated timestamps being created every day.) There is also lots of old application code around, some in binary only form, which has to keep running, and in that, time_t would have been 32 bits. How much of that will still be a problem in another 20 years, when expiry of a current format 32 bit time_t is imminent, I have no idea. But just switching time_t to 64 bits doesn't solve all the problems by itself. kre

On 01/11/2017 02:06 PM, Arthur David Olson wrote:

...

Unfortunately now that I think of it, I see that this can't work on hosts with 32-bit time_t, as they can't cover the 400-year span. So the code is still broken on 32-bit hosts. Is there a better way to fix the code?

Since 32-bit time_t values can only cover (about) 140 years, not covering a 400-year span's not a bug, it's a given.-)

Yes, but unfortunately the code in localtime.c filled in the table only for the years 1970-2038, and this the 68-year filled-in period was not enough to make the goback code work correctly (it needs 400 years). I installed the attached patch to try to fix this; it handles Kees Dekker's test case correctly on hosts with 32-bit time_t.

Date: Thu, 12 Jan 2017 08:05:59 -0800 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <368610ff-5501-d18d-46e8-486588574e2a@cs.ucla.edu> | I haven't seen that; what implementations are those? Sorry, don't know, that was something I was told about, I have not seen one myself. That's why I don't know for sure what the switchover point is - from what I was told I suspect it might have been arranged so that the earliest representable time is a sensible human date (like Jan 1, 1950 or something) regardless of that not being a nice looking bit pattern (it makes no real difference to the implementation.) | They would not conform to POSIX, Perhaps not, but there are times when functioning rationally so users' systems behave as they should, and conforming to POSIX simply clash, and you can guess which wins then. What's more, POSIX is (supposed to be) documenting what systems actually do, so new systems can be compatible, and applications know what to expect. If this technique became common, posix would be updated to allow it. | which requires that time_t be an integer type that | counts non-leap seconds since 1970. The sane way to use this technique would be to make time_t 64 bits, and use the "off-centre" 32 bit values just when a timestamp is required in a 32 bit field for compatibility. That would be the same way, the same as the sane way for the internet would be to be (almost exclusively) using IPv6 with its bigger addresses, rather than IPv4 which ran out of addresses years ago. But pragmatics aren't always sane. | There are systems with unsigned 32-bit time_t, BSD was almost like that, back around 4.1/4.2 (I forget when exactly). I switched time_t to be unsigned at Berkeley... Unfortunately it broke stuff (particularly the localtime() (of that era) for people west of Greenwich, and caused (time_t)0 to print as something in the 22nd century, rather than Dec 31, 1969 (local time). It never bothered me, as I've (almost exclusively) lived east of Greenwich. Rather than just fix the bugs (it was unclear how much else would break), they reverted the change (it was never in anything released). Pity that. | Microsoft and NetBSD are not the only 32-bit systems with 64-bit time_t; Yes, I know that. The point was just that NetBSD switched a long time ago (6 or 7 years, perhaps more) and there are certainly still places where converting from 32 bit time data into 64 bit time_t's is required (and back again on writes.) The need to support old 32 bit time_t's contiues, both for that, and to retain ABI compatibility for programs last compiled before the switch (the gettimeofday(), stat(), utimes() ... sys calls all require 32 bit time_t interface versions to support that ... NetBSD has all of that.) | I expect these | platforms to supplant their older 32-bit time_t cousins on 32-bit | systems before 2038 rolls around. Once again, the hard problem isn't the system and what it uses as time_t for programs compiled today, it is how to cope with data repositories (like filesystems) that have just 32 bit fields for the time. Do you really believe that all the ext2 filesystems in the world will be gone by then? I know I have disks with filesystems on them that were created in the 1990's (they haven't been spun up in a while, but probably still work). They would (assuming the drives still spin) work just fine on a modern NetBSD (they contain NetBSD filesystems) - perhaps on other *BSDs. Most likely by 2038 I won't care about them (more likely by 2038 there will be no me to care or not care, but that is beside the point) but I cannot help but believe that there will still be people with running filesystems (and similar) which would be annoying to be forced to upgrade. And certainly vendor types (even vendors of free systems) do not like to annoy their users by breaking things needlessly. kre

On 01/12/2017 09:06 AM, Kees Dekker wrote:

Note that we even don't have 32-bit time_t hosts anymore (since a few years). For me, 64-bit is the only thing to deal with.

On my Fedora 25 x86-64 platform, I can test 32-bit code by compiling with 'gcc -m32'. Perhaps something similar would work for you.

Does this patch replace the previous one sent?

It's in addition. You can get the current version from GitHub, e.g., via the shell command: |git <https://git-scm.com> clone https://github.com/eggert/tz |

Many thanks. I now used latest source code as retrieved from mentioned git repository. First impression is very good (on RedHat linux, 64-bit). This fix solves a large number of tests (400+) in our own test case. I will do more testing on more platforms. There are still some failing tests in our test suite. I will send a new mail when I have find out the root cause and was able to make a small reproduction. At the final end, I will try to contribute back my changes (Windows related and more). Will be continued. Thanks again. -----Original Message----- From: Paul Eggert [mailto:eggert@cs.ucla.edu] Sent: Thursday, January 12, 2017 18:48 To: Kees Dekker <Kees.Dekker@infor.com>; Arthur David Olson <arthurdavidolson@gmail.com> Cc: Time Zone Mailing List <tz@iana.org> Subject: Re: [tz] question about mktime_tzname() On 01/12/2017 09:47 AM, Paul Eggert wrote:

|git <https://git-scm.com> clone https://github.com/eggert/tz |

Sorry, my email client mangled that one. The actual shell command should be: git clone https://github.com/eggert/tz

On 01/13/2017 05:24 AM, Kees Dekker wrote:

I discovered that zdump.c was updated, and now got additional code from private.h. Why?

The command "git log zdump.c" can help you answer questions like that. It points to: http://mm.icann.org/pipermail/tz/2016-December/024721.html which was fixed as described in: http://mm.icann.org/pipermail/tz/2016-December/024722.html

I'm busy porting the code to Windows, and now see duplicate stuff, but private.h is already included in zdump.c. It is not a big problem (but in general, I like to avoid duplicating stuff);

zdump is intended to be buildable without the rest of the tz source code, and so does not assume any internals of private.h. That being said, it uses some of the same style as the tz source code and so needs some of private.h's definitions when these definitions work on any time_t implementation. Perhaps private.h should be split into two (a portable part, and a non-portable part), but it's not high priority.

I just have to apply some changes that I already applied myself to private.h. In this case: Visual Studio is not setting STDC_VERSION but adheres more or less to STDC (at least, has stdbool.h).

The tz code should work fine the way it is, no? That is, Visual Studio will work just fine with the stdbool.h substitute that is in private.h and zdump.c. If so, there's no need to insist that Visual Studio include stdbool.h and I'm inclined to leave the code alone. The goal is to run on nonstandard platforms, not to cater to their every nook and cranny.

Another point of attention is mixing size_t (e.g. result of strlen()) and int. Complaining compilers, especially in 64-bit mode, will raise an warning or error when mixing these types (regarless the tz code will never return a strlen() value that will exceed a signed 32-bit value).

There are many such false alarms, and pacifying every compiler's false alarms would take too much maintenance effort and contort the code too much. Instead, please ignore the false alarms, or use compile-time options that suppress them.

I will try to resolve these 64-bit warnings (the Makefile provided with the tzcode still uses a default mode of the compiler. As far as I know, this is 32-bit on most platforms. Changing CFLAGS to -m64 -Wall -Wextra will also show a lot of warnings on e.g. Linux).

Yes, and those are all false alarms too. On GNU/Linux I suggest using these compile-time flags instead: make CFLAGS='$(GCC_DEBUG_FLAGS)' This does not generate false alarms with recent GCC, which is the only platform for which I worry about false alarms.

One example: printf("%*s", strlen(str), str) width specifier requires int type for the width (according printf() definition for width specifiers), but if strlen or sizeof is used as input, then a 64-bit value is passed to printf. Depending on whether a platform is big or little endian, the 4 bytes read by printf may or may not show the intended value, but even worse, the stack is out-of-sync. If strlen() returns 8 bytes, printf just reads 4 and str is not at the right place. I'm not familiar with all platform details. So I don't know whether a stack may or may not contain 64-bit values only on 64-bit platforms. A number of warnings is related to strlen()/sizeof()/size_t type, but used as int. -----Original Message----- From: Kees Dekker Sent: Friday, January 13, 2017 22:03 To: Paul Eggert <eggert@cs.ucla.edu>; Arthur David Olson <arthurdavidolson@gmail.com> Cc: Time Zone Mailing List <tz@iana.org> Subject: RE: [tz] question about mktime_tzname() Thanks for your explanation and sharing this git knowledge (I'm a newbie regarding git). My surprise was mainly because zdump.c includes private.h, its code is thus shipped together with zdump.c, and still some code of private.h is duplicated. Anyhow, I'm a guest user, so I will off course respect your choice. I just shared my own surprise (no offence or disrespect intended). It is probably also somewhat a personal choice to choose between a false alarm warning on int a = strlen(str) (warning in 64-bit mode) and to use int a = (int)strelen(str) if we no for sure that str will never exceed max-signed-int. In our own code, we often use a code like if strlen(str) exceeds max-signed-int, then call assert(). Still not perfect, but if a value unintentionally exceeds an expected limit, you will get an abort/core dump. Even in the case of not using a cast, at runtime an overflow is silently ignored and may result in unexpected behavior. But again, I'm 'guest user' of your code. I have to live with more platforms, as our product runs on it. In much regards, gcc is a 'perfect' compiler, but sometimes other platforms complain about other things (not covered by gcc). Each platform has its own odd things. I like to limit these changes to an absolute minimum, but may ask you to solve some. The world is not limited to Linux/gcc only :-). ________________________________________ Van: Paul Eggert [eggert@cs.ucla.edu] Verzonden: vrijdag 13 januari 2017 16:55 Aan: Kees Dekker; Arthur David Olson CC: Time Zone Mailing List Onderwerp: Re: [tz] question about mktime_tzname() On 01/13/2017 05:24 AM, Kees Dekker wrote:

I discovered that zdump.c was updated, and now got additional code from private.h. Why?

The command "git log zdump.c" can help you answer questions like that. It points to: http://mm.icann.org/pipermail/tz/2016-December/024721.html which was fixed as described in: http://mm.icann.org/pipermail/tz/2016-December/024722.html

I'm busy porting the code to Windows, and now see duplicate stuff, but private.h is already included in zdump.c. It is not a big problem (but in general, I like to avoid duplicating stuff);

zdump is intended to be buildable without the rest of the tz source code, and so does not assume any internals of private.h. That being said, it uses some of the same style as the tz source code and so needs some of private.h's definitions when these definitions work on any time_t implementation. Perhaps private.h should be split into two (a portable part, and a non-portable part), but it's not high priority.

I just have to apply some changes that I already applied myself to private.h. In this case: Visual Studio is not setting STDC_VERSION but adheres more or less to STDC (at least, has stdbool.h).

The tz code should work fine the way it is, no? That is, Visual Studio will work just fine with the stdbool.h substitute that is in private.h and zdump.c. If so, there's no need to insist that Visual Studio include stdbool.h and I'm inclined to leave the code alone. The goal is to run on nonstandard platforms, not to cater to their every nook and cranny.

Another point of attention is mixing size_t (e.g. result of strlen()) and int. Complaining compilers, especially in 64-bit mode, will raise an warning or error when mixing these types (regarless the tz code will never return a strlen() value that will exceed a signed 32-bit value).

There are many such false alarms, and pacifying every compiler's false alarms would take too much maintenance effort and contort the code too much. Instead, please ignore the false alarms, or use compile-time options that suppress them.

I will try to resolve these 64-bit warnings (the Makefile provided with the tzcode still uses a default mode of the compiler. As far as I know, this is 32-bit on most platforms. Changing CFLAGS to -m64 -Wall -Wextra will also show a lot of warnings on e.g. Linux).

Yes, and those are all false alarms too. On GNU/Linux I suggest using these compile-time flags instead: make CFLAGS='$(GCC_DEBUG_FLAGS)' This does not generate false alarms with recent GCC, which is the only platform for which I worry about false alarms.

On 01/13/2017 07:55 AM, Paul Eggert wrote:

Perhaps private.h should be split into two (a portable part, and a non-portable part)

Instead of doing that, I installed the attached proposed patches, in which zdump.c always includes private.h and which omits the duplicated code. Since private.h is independent of tzfile format, this should be OK.

On 03/14/2017 04:41 AM, Kees Dekker wrote:

I assume that the previously mentioned patch is already in it, or do you something expect from me to test? I saw that zdump.c removed some code, and relies more on private.h, which is ok for me.

Yes, the patch is in there. Sounds like it's working for you.