I verified that introducing a rule to set zone to empty when encountering “End of rearguard section" fixes the problem, and does not introduce any other changes to the output. Debbie
On Dec 22, 2020, at 10:59 PM, Deborah Goldsmith via tz <tz@iana.org> wrote:
OK, I think I (mostly) figured it out. On Darwin (macOS) the default value of FS is “ “ (space).
For these two lines: # Interpretation Ordinance (Cap 2) # 1919, s. 2.)"
I believe the second one is hitting this rule: || in_comment + 3 == NF)) and I’m not sure which rule the first one is hitting.
I think a safe fix for this issue (and future such issues) might be to check for “End of rearguard section.” and if found, set the variable zone to an empty string. However, I’ll leave this to more experienced awk aficionados. This seems like a pretty dangerous set of awk rules to leave active over a large section of the input.
I suspect that these failures will occur on any system, not just Darwin, but I don’t have access to a non-Darwin system with a working awk at the moment.
Debbie
On Dec 22, 2020, at 10:10 PM, Deborah Goldsmith via tz <tz@iana.org> wrote:
I think I’m getting closer.
In the africa input file, the Zone directive most nearly preceding the new comment block is Africa/Windhoek, so the “zone” variable contains that. Meanwhile, ziguard.awk contains this:
# If this line should differ due to Namibia using negative SAVE values, # uncomment the desired version and comment out the undesired one. Rule_Namibia = /^#?Rule[\t ]+Namibia[\t ]/ Zone_using_Namibia_rule \ = (zone == "Africa/Windhoek" \ && ($(in_comment + 2) == "Namibia" \ || (1994 <= $(in_comment + 4) && $(in_comment + 4) <= 2017) \ || in_comment + 3 == NF)) if (Rule_Namibia || Zone_using_Namibia_rule) { if ((Rule_Namibia \ ? ($(in_comment + 9) ~ /^-/ \ || ($(in_comment + 9) == 0 && $(in_comment + 10) == "CAT")) \ : $(in_comment + 1) == "2:00" && $(in_comment + 2) == "Namibia") \ == vanguard) { uncomment = in_comment } else { comment_out = !in_comment } }
So these rules are in effect while the comment block is being processed. I tried reproducing this on a raspberry pi, but in Raspbian awk is crashing with a malloc heap corruption, so no luck there:
pi@raspberrypi:~/source/tz $ make rearguard_tarballs awk -v DATAFORM=`expr main.zi : '\(.*\).zi'` -f ziguard.awk \ africa antarctica asia australasia europe northamerica southamerica etcetera factory backward >main.zi.out malloc(): unsorted double linked list corrupted Aborted make: *** [Makefile:604: main.zi] Error 134
It’s not clear to me why these rules would match the comment lines that are being altered, but this seems like where it must be coming from. In the previous commit, a new Zone directive for Africa/Lagos follows almost immediately, which would change the value of the zone variable. In the previous commit there were very few comment lines between Zone Africa/Windhoek and Zone Africa/Lagos, so these awk rules have never been tested on a large comment block.
Debbie
On Dec 22, 2020, at 9:48 PM, Deborah Goldsmith via tz <tz@iana.org> wrote:
In the 2020e release, the build process produces a rearguard zic file for africa that does not compile. This is because the rearguard.zi file has that same grammatical problem: missing comment markers.
source africa file:
# In 1919, standard time was changed to GMT+1. # Interpretation Ordinance (Cap 2) # The Laws of Nigeria, Containing the Ordinances of Nigeria, in Force on the # 1st Day of January, 1923, Vol.I [p 16] # https://books.google.com/books?id=BOMrAQAAMAAJ&pg=PA16 # "The expression 'Standard time' means standard time as used in Nigeria: # namely, 60 minutes in advance of Greenwich mean time. (As amended by 18 of # 1919, s. 2.)" # From Tim Parenti (2020-12-10):
output rearguard.zi (and africa zic file):
# In 1919, standard time was changed to GMT+1. Interpretation Ordinance (Cap 2) # The Laws of Nigeria, Containing the Ordinances of Nigeria, in Force on the # 1st Day of January, 1923, Vol.I [p 16] # https://books.google.com/books?id=BOMrAQAAMAAJ&pg=PA16 # "The expression 'Standard time' means standard time as used in Nigeria: # namely, 60 minutes in advance of Greenwich mean time. (As amended by 18 of 1919, s. 2.)" # From Tim Parenti (2020-12-10):
This happens on macOS; I don’t know if it happens on other platforms. It looks like something is going awry in ziguard.awk. The only thing I can think of is that the africa file in the source repository has a big new comment block from Tim Parenti (commit 316c1598e166e15c27fe611cacd81aeada2a836d) and the problem is occurring inside that. Apparently there’s something in that comment block that ziguard.awk is mistaking for zone rules that it needs to change. I’m trying to see if I can reproduce this on a non-Apple system.
Debbie
On Dec 22, 2020, at 6:03 PM, Paul Eggert <eggert@CS.UCLA.EDU> wrote:
The 2020e release of the tz code and data is available. It reflects the following changes, which were either circulated on the tz mailing list or are relatively minor technical or administrative changes:
Briefly: Volgograd switches to Moscow time on 2020-12-27 at 02:00.
Changes to future timestamps
Volgograd changes time zone from +04 to +03 on 2020-12-27 at 02:00. (Thanks to Alexander Krivenyshev and Stepan Golosunov.)
Changes to past timestamps
Correct many pre-1986 transitions, fixing entries originally derived from Shanks. The fixes include: - Australia: several 1917 through 1971 transitions - Bahamas: several 1941 through 1945 transitions - Bermuda: several 1917 through 1956 transitions - Belize: several 1942 through 1968 transitions - Ghana: several 1915 through 1956 transitions - Israel and Palestine: several 1940 through 1985 transitions - Kenya and adjacent: several 1908 through 1960 transitions - Nigeria and adjacent: correcting LMT in Lagos, and several 1905 through 1919 transitions - Seychelles: the introduction of standard time in 1907, not 1906 - Vanuatu: DST in 1973-1974, and a corrected 1984 transition (Thanks to P Chan.)
Because of the Australia change, Australia/Currie (King Island) is no longer needed, as it is identical to Australia/Hobart for all timestamps since 1970 and was therefore created by mistake. Australia/Currie has been moved to the 'backward' file and its corrected data moved to the 'backzone' file.
Changes to past time zone abbreviations and DST flags
To better match legislation in Turks and Caicos, the 2015 shift to year-round observance of -04 is now modeled as AST throughout before returning to Eastern Time with US DST in 2018, rather than as maintaining EDT until 2015-11-01. (Thanks to P Chan.)
Changes to documentation
The zic man page now documents zic's coalescing of transitions when a zone falls back just before DST springs forward.
Here are links to the release files:
https://www.iana.org/time-zones/repository/releases/tzcode2020e.tar.gz https://www.iana.org/time-zones/repository/releases/tzdata2020e.tar.gz https://www.iana.org/time-zones/repository/releases/tzdb-2020e.tar.lz
As usual, links to the latest release files are here:
https://www.iana.org/time-zones/repository/tzcode-latest.tar.gz https://www.iana.org/time-zones/repository/tzdata-latest.tar.gz https://www.iana.org/time-zones/repository/tzdb-latest.tar.lz
Links are also available via plain HTTP, and via FTP from ftp://ftp.iana.org/tz/releases with the same basenames as above.
Each release file has a GPG signature, which can be retrieved by appending ".asc" to the above URLs. Copies of these signatures are appended to this message.
This release corresponds to commit efc5c061920f1d5820d66b7ec88ebff144c55b3f dated 2020-12-22 15:14:34 -0800 and tagged '2020e' in the development GitHub repository at <https://github.com/eggert/tz>.
Here are the SHA-512 checksums for the release files:
37656ee4400f6e7ac8b3d4b515ea2ae940de05e8a95873112a4ec08afc11227214f269e4ef1bedb0389497958dd07a6d4721191e441920bc45c235b029a8a885 tzcode2020e.tar.gz 1e64b5c91b9e56923cf8e3e079781c59c8afb6c379b38b9b91ef493929814d50c29a6368cfcf77db08a7af3b6876387bac5617f64ac965a5bddab436d17862c4 tzdata2020e.tar.gz 77e62a3717d179b13f35710dedd87d780649edf6c338c056e41ba58d49225e512074a5fbfa3ae52242bea6b28d1a08bb0e2b07ff7a3aed031dc9d357dac34170 tzdb-2020e.tar.lz
Here are the GPG digital signatures for the release files:
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEfjeSqdis99YzvBWI7ZfpDmKqfjQFAl/ifpwACgkQ7ZfpDmKq fjQrYA//Z3ky92YK4p8GAejA/69W92B2n+uATB7YPOVJXjWOieo+nmlJihEaY447 bxsLfrExy/AwM5wUoqu9JPQmjUGQYxQR0MjIdpzLeWNCGx+NM8n2j+9K00VXxIXX 356rhSNclm5a/1AmkeeDeYK1C8Fw3QIQnaqeTMHDZ/Txv2UncAMshHR2P+9c2i/k N5A+XK0X3+YNHs/0AHbFobRonynw68cogTeSdslMxFtjWV1iTNNR7c8hHHCDXMgJ sjF7hJ7d6RHBRDQn53XgJdR/OKPaj1RJXncP1Be9j6J1M5wepkHJFVS7/66dDl+g 7TyimxnE/y5gxNVk1JWF9ssz31ksGBRbwq1P8IdF3/EsNF3tTB3eqmKM7PB55Oiu k9MWSDo5/fjfdhH24oYUhocJmMN+n2iiBjAh+mWOq1KNPBGtugbztSZPNXuHDHn9 vsAVuuOdK7FvZcbzE0kMqh3gCmp1YBPcXUSnlqLGaowwlykfoYYR0MGz8AdvJreE 5Su2wfiPlI/B2zvVmJ1i2LFr+9nfVFWlmXGAwFmHunpEKTHB9b0HKXVdsJL9qZC2 Gm7SXPMzHEjZ1G420jT8Bx0qfGLC62jTq98QSK6BdcfLd0AsU+rstmQviFZOtSAv qEPrAXPXUraEwMs2/dR+eXR0DWQMEbUWgtmxJM3jHOGnWRseAQc= =xaDL -----END PGP SIGNATURE----- -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEfjeSqdis99YzvBWI7ZfpDmKqfjQFAl/ifpwACgkQ7ZfpDmKq fjSC+w/9HzW/PbI7gfxfbPkF0fCyHCl2OWWrLlJi1m0G72balJ+BmkhI3vvnBGwK 3OKhvN/HOxA4d11JpXiYmPuQ7iuCDO8nRTDWbtg5nuEdP3FOJqUBD0HKDjTg/UUo haXoFxJwPklPqGOiIsFklEQWgVwpN+LwZkL7b2yjjadL0k5ukx6z/G2lJ5qF2i63 f3Gbe/oG/UYd3ERlDpjvgn2tJWfmoS0S3qU3Y9TpraL3NrmunnbtIFDW7SIEM2oP W4XmtkNgBQGlHS1/x/vocNo31h6KyzqVg6OSyvk88Fzp8vE+gHGOVTLR2RdNh/0H NQDi+Pj65KlIJUsjcLmZo+Huv+eNUfzwBcG+LGwUhay0q+2kaeGTs6epohYFNZen A/0/nzXJBohqcBnbLcBYVf+AZUHuPs5lTkG3OZpdRSvkmw6lELlJbBQMEf/HP/kG l43x9DEe0Dr2oDIEr1mfjszFJqKvSg43QJnTgUJtd0SqITazC1N9XuIZj0uXJ64g zfC1aByJ1FLNJmkMDKbl+OzVg7o53sS2gzL/EaGp43NeOPVa84w5MSWdZnuPz8xa 8KDdcaqWeY+mdLonfq5K/DZhOcV6UanBEXFFrmGZymSSqF2zYPpzPBB/7+B29S5F 68v8ScTY8a3fKjhQqrqzNOOKbVMSwM7m8dZ5o69nIW/knvdmxeg= =eekA -----END PGP SIGNATURE----- -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEfjeSqdis99YzvBWI7ZfpDmKqfjQFAl/ifp0ACgkQ7ZfpDmKq fjQnxA//ZYROGBW1B1b7EY+j8b8z8oBsrzuN+PHkpmXqXre/rVJfIU7NoRoOoZ27 U3JDYPhQ6jy3fOCkW8Xa/eBwEe9PXfBnlpjAYbQ4d7KUvxzDauNG7Sch/aOKuQqE LIM9B+Gdyojn1dAimKcVh8z+lSH3f+gaL3/kCDAPDCJqxsDpbnzQ5rsgpf7OlkG8 6yBRo4wNsW6D49nEVScCPwtmlMQUBFesV0J49rZ74NL9eArIjjdHgwSSc0RtSTLx VT8yDmV1ZW93W4+ZKwfnyPb/u8w8O5NP42SoWHOdXV40UpyU/jWfioKPhbLr+D1R AU+uPE/GBpIPEd6dL75WNMQWvSPkER+QEcTYdzJIMiqOpwUXEADn5/miH3JiJ1Cx rgkQuukOzmmshL3YiKxWM2WT41NdVZn4MrXx3TfS2gw0Am4zElkRwcNanFc7Ksyl PNoJrypAKPnArG3+P0LSFxq9nQUsaajlYcAqYthRtzTicPA13iXWml/ATh1sw36H rJAEAVBg2TmIHo4/zlEuV7alrnBT0tIz38RtGvGA7lVB0w8ZLrh3BqyezucoK3Zc GsleX8wkJrrWMcdVPnRUCfCosx8IN57349mnL2g5v1B3Ij/ytYy4yDr8H7/vAX2c ddaxW1aluGBzL48ZKKJedDP/vZgRWINXMAyEyqdXW2G33aMsFlw= =IS+h -----END PGP SIGNATURE-----
PS. If your tzdata parser does not yet support negative DST offsets or times past 24:00, or if it insists on a 'pacificnew' file that is no longer present, this release's data entries can be turned into a rearguard-format tarball that should work even with these older parsers. This is intended to be a temporary transition aid for these parsers. To generate a rearguard-format tarball, obtain the full distribution as described above, and run the command 'make rearguard_tarballs' on a development host. Or you can run 'make rearguard.zi' to generate a single file that can be fed directly to a parser that works like 'zic'.