tzfile.5 says:
After the second header and data comes a newline-enclosed, POSIX-TZ-environment-variable-style string for use in handling instants after the last transition time stored in the file
I had imagined that the reference implementation would apply the future rule for all of the future, but I currently observe that UTC-offset changes get stuck after a while for many (but not all) zones. For example, for *"Australia/Sydney"*, whose POSIX TZ string is *"AEST-10AEDT,M10.1.0,M4.1.0/3"*, tz_localtime_rz() produces the following civil times at the given instants: 14745254399 = 2437-04-05T02:59:59+1100 14745254400 = 2437-04-05T02:00:00+1000 14760979199 = 2437-10-04T01:59:59+1000 14760979200 = 2437-10-04T03:00:00+1100 That looks good ... daylight time from the first Sunday in Oct through the first Sunday in Apr, shifting at 03:00. But once we hit 2438, the UTC offset is stuck at +1100. For example, in July we would expect to see standard time, but instead get: 14784296400 = 2438-07-01T00:00:00+1100 And it never seems to recover after that. I'm pretty sure this used to work.
On 10/13/22 14:58, Bradley White via tz wrote:
For example, for *"Australia/Sydney"*, whose POSIX TZ string is *"AEST-10AEDT,M10.1.0,M4.1.0/3"*, tz_localtime_rz() produces the following civil times at the given instants:
14745254399 = 2437-04-05T02:59:59+1100 14745254400 = 2437-04-05T02:00:00+1000 14760979199 = 2437-10-04T01:59:59+1000 14760979200 = 2437-10-04T03:00:00+1100
That looks good ... daylight time from the first Sunday in Oct through the first Sunday in Apr, shifting at 03:00. But once we hit 2438, the UTC offset is stuck at +1100. For example, in July we would expect to see standard time, but instead get:
14784296400 = 2438-07-01T00:00:00+1100
And it never seems to recover after that.
Thanks for reporting the issue. I'm not seeing that problem when I build on Fedora 36 x86-64, using this command: make TOPDIR=/tmp/tz CFLAGS='$(GCC_DEBUG_FLAGS)' -j5 install in that when I run this command: ./zdump -i -c 2437,2440 Australia/Sydney the output is: TZ="Australia/Sydney" - - +11 AEDT 1 2437-04-05 02 +10 AEST 2437-10-04 03 +11 AEDT 1 2438-04-04 02 +10 AEST 2438-10-03 03 +11 AEDT 1 2439-04-03 02 +10 AEST 2439-10-02 03 +11 AEDT 1 which looks OK. I get the same results if I set CFLAGS='$(GCC_DEBUG_FLAGS) -m32 -D_TIME_BITS=64' instead. To help debug this, it'd be nice to know the platform and a recipe for reproducing the problem. The existence of tz_localtime_tz indicates that you're using non-default CFLAGS of some sort, and this may be at issue here.
On Thu, Oct 13, 2022 at 6:32 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
On 10/13/22 14:58, Bradley White via tz wrote:
For example, for *"Australia/Sydney"*, whose POSIX TZ string is *"AEST-10AEDT,M10.1.0,M4.1.0/3"*, tz_localtime_rz() produces the following civil times at the given instants:
14745254399 = 2437-04-05T02:59:59+1100 14745254400 = 2437-04-05T02:00:00+1000 14760979199 = 2437-10-04T01:59:59+1000 14760979200 = 2437-10-04T03:00:00+1100
That looks good ... daylight time from the first Sunday in Oct through the first Sunday in Apr, shifting at 03:00. But once we hit 2438, the UTC offset is stuck at +1100. For example, in July we would expect to see standard time, but instead get:
14784296400 = 2438-07-01T00:00:00+1100
And it never seems to recover after that.
Thanks for reporting the issue. I'm not seeing that problem when I build on Fedora 36 x86-64, using this command:
make TOPDIR=/tmp/tz CFLAGS='$(GCC_DEBUG_FLAGS)' -j5 install
Try that with ZFLAGS='-b fat' and observe the difference. $ make TOPDIR=/tmp/tz CFLAGS='$(GCC_DEBUG_FLAGS)' ZFLAGS='-b fat' -j5 install : $ /tmp/tz/usr/bin/zdump -i -c 2437,2440 /tmp/tz/usr/share/zoneinfo/Australia/Sydney TZ="/tmp/tz/usr/share/zoneinfo/Australia/Sydney" - - +11 AEDT 1 2437-04-05 02 +10 AEST 2437-10-04 03 +11 AEDT 1
in that when I run this command:
./zdump -i -c 2437,2440 Australia/Sydney
the output is:
TZ="Australia/Sydney" - - +11 AEDT 1 2437-04-05 02 +10 AEST 2437-10-04 03 +11 AEDT 1 2438-04-04 02 +10 AEST 2438-10-03 03 +11 AEDT 1 2439-04-03 02 +10 AEST 2439-10-02 03 +11 AEDT 1
which looks OK. I get the same results if I set CFLAGS='$(GCC_DEBUG_FLAGS) -m32 -D_TIME_BITS=64' instead.
To help debug this, it'd be nice to know the platform and a recipe for reproducing the problem. The existence of tz_localtime_tz indicates that you're using non-default CFLAGS of some sort, and this may be at issue here.
On Thu, Oct 13, 2022 at 6:32 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
To help debug this, it'd be nice to know the platform and a recipe for
reproducing the problem.
OK, here is something using only unmodified tz code: $ uname -om x86_64 GNU/Linux $ rm -fr /tmp/tz/ $ make --silent TOPDIR=/tmp/tz CFLAGS='$(GCC_DEBUG_FLAGS)' -j5 install $ TZ=Australia/Sydney ./date -r 14784296400 +%Y-%m-%dT%H:%M:%S%z 2438-06-30T23:00:00+1000 $ rm -fr /tmp/tz/ $ make --silent TOPDIR=/tmp/tz CFLAGS='$(GCC_DEBUG_FLAGS)' ZFLAGS='-b fat' -j5 install $ TZ=Australia/Sydney ./date -r 14784296400 +%Y-%m-%dT%H:%M:%S%z 2438-07-01T00:00:00+1100
On 2022-10-13 17:17, Bradley White wrote:
Try that with ZFLAGS='-b fat' and observe the difference.
Thanks, that's the part of the recipe I was missing. I rarely test fat TZif files, relying on downstream users like you for that. I hope downstream users cast away these fat crutches sooner rather than later. Please try the attached patch, which I've installed in the development repository.
On Fri, Oct 14, 2022 at 2:44 AM Paul Eggert <eggert@cs.ucla.edu> wrote:
Please try the attached patch, which I've installed in the development repository.
Yes, that worked for me. Thanks. I rarely test fat TZif files Perhaps a goal should be a public (in the repo), comprehensive test suite that verifies all the options, in all their combinations?
On 2022-10-14 00:10, Bradley White wrote:
Perhaps a goal should be a public (in the repo), comprehensive test suite that verifies all the options, in all their combinations?
Although the tests could certainly be more extensive, I doubt whether it would be practical to test all option combinations as I count about four dozen options now, many with more than two possibilities. One step we can easily take is to improve the comments to warn about this particular problem, so I installed the attached.
On Fri, Oct 14, 2022 at 12:37 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
On 2022-10-14 00:10, Bradley White wrote:
Perhaps a goal should be a public (in the repo), comprehensive test suite that verifies all the options, in all their combinations?
Although the tests could certainly be more extensive, I doubt whether it would be practical to test all option combinations as I count about four dozen options now, many with more than two possibilities.
That's a problem, isn't it. I'd proffer that too many options to test means too many options, period. And the non-visibility of "the tests" means that we "downstream users" are never going to be as effective in catching problems as you might be expecting. We also can't contribute to the tests.
One step we can easily take is to improve the comments to warn about this particular problem, so I installed the attached.
*Fat TZif files work around incompatibilities and bugs in some* *TZif readers, notably older ones that ignore or otherwise* *mishandle 64-bit data in TZif files; however, fat TZif files* *may trigger bugs in newer TZif readers.* [I'm not sure I would characterize the inability of an old reader to handle a new format as a "bug", but OK.] So, to handle old readers I need fat TZif files, and to handle new readers I need to avoid fat TZif files. That places folks who want to handle all readers in an untenable position.
On 2022-10-14 14:05, Bradley White wrote:
And the non-visibility of "the tests" means that we "downstream users" are never going to be as effective in catching problems as you might be expecting. We also can't contribute to the tests.
All the tests I use are in the standard distribution; they are neither invisible nor secret, and you can contribute to them the same way you can contribute to any other part of the distribution. Unfortunately, writing patches to improve the tests is a boring and thankless task and I expect few will want to do it.
to handle old readers I need fat TZif files, and to handle new readers I need to avoid fat TZif files. That places folks who want to handle all readers in an untenable position.
This topic is discussed at some length in Internet RFC 8536 Appendix A. You're correct that no single format will cater to every buggy TZif reader for every possible timestamp. However, these problems are inherent to the situation and has been true ever since TZif files were introduced: the problems occurred with TZif files before slim files were introduced, and I expect they'll occur with TZif files long after fat files die out (assuming that ever happens). By design, these compatibility problems and bugs have been rare in practice. That being said, I do expect quite a few more bugs to shake out around the year 2038, due to the large number of currently-produced systems that continue to mishandle timestamps after 2038. So it'd be helpful if we had a testing strategy that would help people discover and report these bugs in other systems sooner than 15 years and 3 months from now, when it'll be too late.
participants (2)
-
Bradley White -
Paul Eggert