Joda-Time compiler broken again
The latest 2020e release broke the Joda-Time compiler again. https://github.com/JodaOrg/joda-time/runs/1607329443?check_suite_focus=true#... Presumably, some rule now contains a 31 in a 30 day month now? Stephen
On 2020-12-27 02:20, Stephen Colebourne wrote:
The latest 2020e release broke the Joda-Time compiler again. https://github.com/JodaOrg/joda-time/runs/1607329443?check_suite_focus=true#... Presumably, some rule now contains a 31 in a 30 day month now?
Maybe enhance your error logging and reporting and email yourself the details on errors? You can also go on to github repo, select the latest release, select compare, select previous release, select Files changed tab to check: many 24:00 added/changed. comments at end of Rule lines, Save <= 0030, Abbr +0030, other? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]
On 12/27/20 1:20 AM, Stephen Colebourne wrote:
https://github.com/JodaOrg/joda-time/runs/1607329443?check_suite_focus=true#...
Could you arrange for a buildbot to run Joda-Time on the tzdb development repository on GitHub too? That would help catch problems like this earlier.
Presumably, some rule now contains a 31 in a 30 day month now? I just ran 'git diff 2020d..2020e' and saw that "31"s were introduced only for Mar, May, Jul, Aug, Oct, Dec, so I don't see such a calendar error introduced in tz2020e.
It's hard to tell from Joda-Time's log file what the problem is, as it doesn't report the tz input file name or line number. The only useful info the exception handler gives is "Value 31 for dayOfMonth must be in the range [1,30]". However, I did see some "31"s in 2020e that were not in 2020d, namely, lines that I've manually marked with "*" in the following listing generated by "grep -n -E '[^#].*[[:space:]]31[[:space:]]+24' $(git ls-files) | sed -E 's/[[:space:]]+/ /g; s/^/ /'": africa:355:Rule Egypt 2014 only - Jul 31 24:00 1:00 S *africa:533: 2:30 - +0230 1936 Dec 31 24:00 *africa:534: 2:45 - +0245 1942 Jul 31 24:00 asia:232:Rule Dhaka 2009 only - Dec 31 24:00 0 - asia:412:Rule Shang 1947 only - Oct 31 24:00 0 S *asia:1870:Rule Zion 1940 only - May 31 24:00u 1:00 D *asia:1873:Rule Zion 1942 1946 - Oct 31 24:00u 0 S *asia:1874:Rule Zion 1943 1944 - Mar 31 24:00u 1:00 D *asia:1877:Rule Zion 1948 only - Aug 31 24:00u 1:00 D *asia:1878:Rule Zion 1948 1949 - Oct 31 24:00u 0 S *asia:1882:Rule Zion 1951 only - Mar 31 24:00u 1:00 D *asia:1918:Rule Zion 1985 only - Aug 31 24:00 0 S *asia:1964:Rule Zion 1991 only - Aug 31 24:00 0 S europe:1756:Rule Italy 1917 only - Mar 31 24:00 1:00 S *northamerica:2966:Rule Bahamas 1944 only - Dec 31 24:00 0 S Given that the last Joda-Time output before it crashed was "Fixed negative save values for rule 'Namibia'", one possibility is that Joda-Time mishandles 24:00 at month or year end in continuation lines like africa:533. Less likely possibilities include mishandling 24:00u at month end in Rule lines like asia:1870, or mishandling Dec 31 24:00 in the western hemisphere in Rule lines like northamerica:2966. From the "Fixed negative save values" diagnostic it appears that Joda-Time is operating on main data not rearguard data, so this is not an issue that can be fixed by futzing with ziguard.awk. Not that I'm a fan of ziguard.awk - it's a hack and the latest problem with Apple suggests that it might now be more trouble than it's worth.
The code was taking a rule like May 31 24:00 (Asia/Jerusalem) and moving May 31 to Jun 1 because of the 24:00, then trying to set it to the 31st. I've hacked the code to avoid changing the day until later in the logic, but ultimately the code is quite brittle because of the complexity of the input format. I would note that this code has existed unchanged for 17 years - the seemingly innocuous change from Jun 1 00:00 to May 31 24:00 caused the issue (and similar changes). A reminder that there is no such thing as a completely innocuous change. Stephen On Sun, 27 Dec 2020 at 18:54, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 12/27/20 1:20 AM, Stephen Colebourne wrote:
https://github.com/JodaOrg/joda-time/runs/1607329443?check_suite_focus=true#...
Could you arrange for a buildbot to run Joda-Time on the tzdb development repository on GitHub too? That would help catch problems like this earlier.
Presumably, some rule now contains a 31 in a 30 day month now? I just ran 'git diff 2020d..2020e' and saw that "31"s were introduced only for Mar, May, Jul, Aug, Oct, Dec, so I don't see such a calendar error introduced in tz2020e.
It's hard to tell from Joda-Time's log file what the problem is, as it doesn't report the tz input file name or line number. The only useful info the exception handler gives is "Value 31 for dayOfMonth must be in the range [1,30]". However, I did see some "31"s in 2020e that were not in 2020d, namely, lines that I've manually marked with "*" in the following listing generated by "grep -n -E '[^#].*[[:space:]]31[[:space:]]+24' $(git ls-files) | sed -E 's/[[:space:]]+/ /g; s/^/ /'":
africa:355:Rule Egypt 2014 only - Jul 31 24:00 1:00 S *africa:533: 2:30 - +0230 1936 Dec 31 24:00 *africa:534: 2:45 - +0245 1942 Jul 31 24:00 asia:232:Rule Dhaka 2009 only - Dec 31 24:00 0 - asia:412:Rule Shang 1947 only - Oct 31 24:00 0 S *asia:1870:Rule Zion 1940 only - May 31 24:00u 1:00 D *asia:1873:Rule Zion 1942 1946 - Oct 31 24:00u 0 S *asia:1874:Rule Zion 1943 1944 - Mar 31 24:00u 1:00 D *asia:1877:Rule Zion 1948 only - Aug 31 24:00u 1:00 D *asia:1878:Rule Zion 1948 1949 - Oct 31 24:00u 0 S *asia:1882:Rule Zion 1951 only - Mar 31 24:00u 1:00 D *asia:1918:Rule Zion 1985 only - Aug 31 24:00 0 S *asia:1964:Rule Zion 1991 only - Aug 31 24:00 0 S europe:1756:Rule Italy 1917 only - Mar 31 24:00 1:00 S *northamerica:2966:Rule Bahamas 1944 only - Dec 31 24:00 0 S
Given that the last Joda-Time output before it crashed was "Fixed negative save values for rule 'Namibia'", one possibility is that Joda-Time mishandles 24:00 at month or year end in continuation lines like africa:533. Less likely possibilities include mishandling 24:00u at month end in Rule lines like asia:1870, or mishandling Dec 31 24:00 in the western hemisphere in Rule lines like northamerica:2966.
From the "Fixed negative save values" diagnostic it appears that Joda-Time is operating on main data not rearguard data, so this is not an issue that can be fixed by futzing with ziguard.awk. Not that I'm a fan of ziguard.awk - it's a hack and the latest problem with Apple suggests that it might now be more trouble than it's worth.
On 12/27/20 5:31 PM, Stephen Colebourne wrote:
the seemingly innocuous change from Jun 1 00:00 to May 31 24:00 caused the issue
Thanks for looking into the problem. As I understand it the relevant law said May 31 at the end of the UT day not June 1 at the start of the day, which is why we changed the data. It is odd that Joda-Time handles "Jul 31 24:00" correctly (as this has been in tzdb since 2014e for the Egypt rule), but it mishandles "May 31 24:00u" in 2020e. I guess the trailing "u" is part of what triggers the bug. Does your fix to Joda-Time mean that we need not worry about this issue in tzdb? That is, can we just tell people to upgrade to the latest Joda-Time release? If that's not realistic, I can look into modifying ziguard.awk to avoid using 24:00u in the rearguard format. 24:00u is used only in the Zion rule, and this rule is used only in Zones where STDOFF is 2:00, so we can change all the "24:00u"s to "2:00s"s the next day. This would be a bit of a hack as I'd have to modify ziguard.awk to do some calendar arithmetic (e.g., it'd need to know that the next day after Sep 30 is Oct 1), which is why I'm hoping that we can just ask people to upgrade Joda-Time.
Date: Sun, 27 Dec 2020 18:52:09 -0800 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <380c4727-0aac-c7c5-7914-5e045442f44e@cs.ucla.edu> | It is odd that Joda-Time handles "Jul 31 24:00" correctly (as this has | been in tzdb since 2014e for the Egypt rule), but it mishandles "May 31 | 24:00u" in 2020e. I guess the trailing "u" is part of what triggers the bug. More likely that the actual problem is a failed/misplaced error check, and that it worked before because August has 31 days, and not this time, because June does not. kre
On Mon, 28 Dec 2020 at 02:52, Paul Eggert <eggert@cs.ucla.edu> wrote:
It is odd that Joda-Time handles "Jul 31 24:00" correctly (as this has been in tzdb since 2014e for the Egypt rule), but it mishandles "May 31 24:00u" in 2020e. I guess the trailing "u" is part of what triggers the bug.
The month after July has 31 days. The month after May has 30 days. You've obviously never had Mar 31, May 31, Aug 31 or Oct 31 before.
Does your fix to Joda-Time mean that we need not worry about this issue in tzdb? That is, can we just tell people to upgrade to the latest Joda-Time release?
Yes, no change in tzdb Stephen
Stephen Colebourne said:
The month after July has 31 days. The month after May has 30 days. You've obviously never had Mar 31, May 31, Aug 31 or Oct 31 before.
Or Jan 31. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646
On 12/28/20 2:23 AM, Stephen Colebourne wrote:
On Mon, 28 Dec 2020 at 02:52, Paul Eggert <eggert@cs.ucla.edu> wrote:
It is odd that Joda-Time handles "Jul 31 24:00" correctly (as this has been in tzdb since 2014e for the Egypt rule), but it mishandles "May 31 24:00u" in 2020e. I guess the trailing "u" is part of what triggers the bug.
The month after July has 31 days. The month after May has 30 days. You've obviously never had Mar 31, May 31, Aug 31 or Oct 31 before.
Actually tzdb has had a line "Rule Shang 1947 only - Oct 31 24:00 0 S" since tzdb 2018f. So the problem can't be simply 24:00 on the 31st of a month that is followed by a shorter month. (Though this doesn't matter much for tzdb, given your comment below.)
Does your fix to Joda-Time mean that we need not worry about this issue in tzdb? That is, can we just tell people to upgrade to the latest Joda-Time release?
Yes, no change in tzdb
Thanks, that's good as it means we needn't change ziguard.awk.
participants (5)
-
Brian Inglis -
Clive D.W. Feather -
Paul Eggert -
Robert Elz -
Stephen Colebourne