broken compression in archive
I'm doing some work on how to access historical tzdb versions, and I noticed that tz{code,data}94d.tar.Z are, despite their names, uncompressed tar files. To be precise, I refer to the files with these SHA-512 hashes: 9872b757ee3a8bf0ed3de1f0875f878b977b34bd2b60440b23a8c7308a7514d12a5d5d283387973e61b75e1a8eebd27009aab78eee4e12f1f1a5593f71941328 tzcode94d.tar.Z e1524951a5c261cc8eef1f3adadf5ebf8143293745fda9411a0090d6bb6e19f899dd7a6e55fe2f8f490a167a3da392020e49b369984bc1f5cbed87d664eb58de tzdata94d.tar.Z They're unique, in the present archive <ftp://ftp.iana.org/tz/releases/>, in having a mismatch between format and filename suffix. I think these files should be replaced, in the archive, with actual compressed (with compress(1)) versions of these tars, under the same name, so that the names are no longer misleading and they are in the same format as the surrounding files. The other way to make the names no longer misleading is to rename these files, dropping the ".Z" suffix, but this approach has the problem that these files would (still) have a different format from all the other tarballs. Note that this mismatch is not a mistake made in the original release of 94d, and the replacement that I'm proposing would not erase a record of the original release files. The original 94d release was of *gzipped* tar files, which are not in the present archive. Actual releases switched from .Z to .gz at 93g, but the archive files switch at tzdata95c and tzcode95d. Obviously someone recompressed the intervening tarballs somewhere along the chain of custody that led to the present archive, and it appears that the failure to (re)compress 94d happened in that process. -zefram
Date: Wed, 1 Nov 2017 03:11:07 +0000 From: Zefram <zefram@fysh.org> Message-ID: <20171101031107.GE6716@fysh.org> | I'm doing some work on how to access historical tzdb versions, and I | noticed that tz{code,data}94d.tar.Z are, despite their names, uncompressed | tar files. That is probably my doing - I maintain the ~ftp/pub/oldtz archive on munnari.oz.au and for most of its life. manually. It was, once, the only available set of old data files (aside from what was in ado's sccs files.) I will compress those 2 files there so the names match reality (and all the other files of the same era.) The change from gzip to compress format was probably (I don't really remember why, or what) that I did not trust that new fangled gzip thing, or that it would last very long, so was keeping the archive in a consistent format. kre
Robert Elz wrote:
The change from gzip to compress format was probably (I don't really remember why, or what) that I did not trust that new fangled gzip thing
Ironically, two weeks ago I fixed a bug where gzip mishandled some old .Z (compress-format) files! See: http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=79f88bd1e54d6042fbe50c2... These old file formats sure are taking a long time to die....
Zefram wrote:
Actual releases switched from .Z to .gz at 93g, but the archive files switch at tzdata95c and tzcode95d. Obviously someone recompressed the intervening tarballs somewhere along the chain of custody that led to the present archive, and it appears that the failure to (re)compress 94d happened in that process.
Thanks for reporting the problem. I propose that we fix the file formats to match those of the announcements. Although this won't give us the exact files that Arthur released, it will be closer than what we have now. Proposed shell script attached; the idea is to execute this shell script on the archive of rel3eased files at iana.org.
Paul Eggert wrote:
I propose that we fix the file formats to match those of the announcements.
That sounds OK.
script attached;
You missed a couple: tzdata94h.tar.Z and tzdata95b.tar.Z. Otherwise the script looks good to me. When it's done, please post to the mailing list the SHA-512 hashes of the revised files. Hashes of archived and new files have previously been posted here, and I've found that useful. -zefram
On 2017-11-01 15:19, Zefram wrote:
I propose that we fix the file formats to match those of the announcements. That sounds OK. script attached; You missed a couple: tzdata94h.tar.Z and tzdata95b.tar.Z. Otherwise the
Paul Eggert wrote: script looks good to me. When it's done, please post to the mailing list the SHA-512 hashes of the revised files. Hashes of archived and new files have previously been posted here, and I've found that useful.
How about creating/updating a sha512.sum file in releases/, and maybe in other directories too, to allow easily checking that downloads are complete, even if gpg signatures are not available, or if users choose not to use PGP/GPG? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
Paul Eggert wrote:
I propose that we fix the file formats to match those of the announcements.
This has now been done. For the record, here follow some SHA-512 hashes. Via both FTP and HTTP I no longer see the .tar.Z files in question: a1c801ed4a1bf242cecf92d59c3fb64b49fa998e60b0a3629d8a20a2f68ed29a691a7bce091e298073b6c184fb610bc4157b6ecfe5d638b8fd5aa1257f2a330b tzcode93g.tar.Z 1303e720e59b608bfd4f3cc9df6f22f773e4e07c70d8dd7a45cd731b312706c620b70341ef304689487a0834528e5c71dcb70f9cf82e2f00f275562f9f23aeef tzcode94b.tar.Z 8419ad1dd75bf424a3cdedd89e626cba2f178ff2d6c46e4fd2f1aad726ba9337eec70939397f4079383e7bf6c4903cd91d2298d678038c6d03845d19685b4596 tzcode94c.tar.Z 9872b757ee3a8bf0ed3de1f0875f878b977b34bd2b60440b23a8c7308a7514d12a5d5d283387973e61b75e1a8eebd27009aab78eee4e12f1f1a5593f71941328 tzcode94d.tar.Z 4105c2862e74c0e40418874735b7c26a5e9797a8b4c98ca9d73cc4a22872313ae0f6121b0cd376494bb3e86e438db7468cf605c577d459aa1fe80debd5ef9760 tzcode94e.tar.Z 88ba97a4576b9e49ee557d6b9c8c3154e0ff528ba34a2d69bb8a2cb36662404ec2420519dbfb7763dde8c390c1bded7597a6886f615956e342fc1542f48cefbb tzcode94g.tar.Z 58f82147be3c74d4b26cfb2ec64db9401d7e2cf3734dae50777143e210e0f3669d6d9799625023bce30c28b0a60b08691986562d353c3264fc8953a1f9ba664b tzcode94h.tar.Z 405f4bf43bdc921cfceec2c438964fe875fca4b7188b83c1923ffa4bf59912e41a16f80e95d3c8ed717e6ded344b312f4115b8377c9c5ba72f628ed0524224f7 tzcode95b.tar.Z 8761094f2551a812da0d657e2531c979df126e4e6e50e0eb69f33e8e163577886a7ce21c990f0f1bc31c57b3f66e53feebe7f8099254600088d09c225665dcef tzcode95c.tar.Z 9f8bcb29e39ed959a321cc7a91ede02f0dc4ad7fd203ebc178dea6d4be362fac399a8275ef2ed016c2394ba27a336b54630be3feccb333da3bc2f06960358e82 tzdata93g.tar.Z 5aad836757522d1416d1047026160888c5eb8fba829afd6678d38630af7bf84ec24a48fe93727d10be619670d76abacf81b496b0e8442c918c75fe048a6082b4 tzdata94a.tar.Z 1c7e59fb22ab079a8e3da94c6cbe7bb6795e68adfb1d37b966a0f4d0056f339c84183b6a6fa7a5f6224a2ab1950e8bf71512441689009455a7e1870b27cc4000 tzdata94b.tar.Z e1524951a5c261cc8eef1f3adadf5ebf8143293745fda9411a0090d6bb6e19f899dd7a6e55fe2f8f490a167a3da392020e49b369984bc1f5cbed87d664eb58de tzdata94d.tar.Z 5a78a918ebc69918ce3b32811b0a885fa374a4b8d98eaa61f5127139d7fc44fd69a9dcbe2e0676a953c33f86fd749468af3d1690d2f0d3ee0873c2e87660b8fd tzdata94e.tar.Z d1435e50f1fdbe4fa00f824b5ea206ace7a8fc00b2693f155f1418958febb9be423ce02acdd12cb39272ad3f530d50a778b59830a516facafe417f4d6afcbe00 tzdata94f.tar.Z 4409de26f7cbb4d6d827cc069c7c7a884e704e61a5ea165ab989ad99628aafed7c559269ee6f23cc84053369a0da17f7faa2a4b1efc8a4c72099397ae791cb32 tzdata94h.tar.Z 33c65d20e8efa9c76cd7641b85c05ea11abe4724686b6f7f85edff9db3317bb40db532405e7832cb419ee9775311c02a0d04bc0d5d482fb77f5cbb5bc69d619b tzdata95b.tar.Z Instead I see these reconstructed .tar.gz files: fdeec7df164487832df6c0bf58e872a0938656a73fc2cbe6a6319dd5252769d24d5ff7336a21886e9f584bab78631d0c2b64db2eb79227e5789ba3bdb742e800 tzcode93g.tar.gz d68cf00b782019334ed810303379b1612cddcd2ed6b5b5a2044739968147ddc6fca4437fa3b538ba3130177ba08ffffc0112edac0815aebcab83fa1ec3ea3b1e tzcode94b.tar.gz ea0a3d5fbbd01119089a5407d168ef9c3dc2d3d887332cac713a376acc5a8358182e73fb8350195c388c6483b2b9b671c28221db6153c84231b5c7f4f73d8d74 tzcode94c.tar.gz 8e9f65e059f35167cc3ba6c1aa52f533cb9d6b4c373377c11e4454444822498eef56e6247d8a08456a7e000fa65bf2fe5999a4dceeaf67a8b9866e15014b6b95 tzcode94d.tar.gz 8dcb1c19c108bf80b40e34826706abe7e646726a77324608e265b66fd16b4fb02db346e6d0667c7e856e279d50352aeb15eb8be21a0782ea0cf128750560489a tzcode94e.tar.gz 8d6411c8ec4ba52b2be853b8009685cd8568c21623011f5349add0d0130572df15fe1ee9bc192ac4e868df56591960a0083d523b6fe378749caefca34f53989a tzcode94g.tar.gz 4e8f733dc51f57c101f59032cc4b4211664b2ba17ec7283c39c16e20447731937afdd76e0271ef553cf76b671797d48fc1f8922722f0f092fbefe9a3c6e9b4a4 tzcode94h.tar.gz 12b10af8046ac313a36ff2a36ac536c380be315ccef8f34286474458b54ad463023c186fb2d7768e86874feb9bca33b89d6c17723755152f5631002d4b1124fa tzcode95b.tar.gz b41462d17eadda2a9bd969189832b7ccea4b1a5328afd7e76eeef56684bdbb834071cca28a990a4a3043e256a22c276fa785236ffc5c262a53c9687c2d22c87e tzcode95c.tar.gz d8f3230e16d32e6168b0d106ae643712bf8088bb2b58f0b8df1639ab2f417a19ffbdf40a2cfb1d44300f091ec13ead08713c2f798d7d6ecad47f2e6d9891a121 tzdata93g.tar.gz d37d3c982e682e8b11f16b0520ef094b88709c3a90dd5970911dded34c601965c43a42df0f20e946265406838d9ea2730c721596554d6d167f3a791ade6d7b4d tzdata94a.tar.gz d93e0288fb717b62fafb0f7cb329e9e90821eb39c66288a9ca012b582cb45e66a1f6b920be815bf36de81fede60150bd6dcc84dc9e10fc39c1df0a1d2df17b54 tzdata94b.tar.gz 53154a1c04355fb6341f0881eab434b3199c8912a699e0a16247884a0d4b43cfa2aa59c2cee0bfa230863bcd92516464bc1ea70497eb4ee1929fa1286380edb1 tzdata94d.tar.gz 046496e3195dc974e66fa2eefa5e96edcdcc01f7b7987e99647f6374fedc95f516acb77e52b6098eac7c278c2dc64d51621222ff71c28752f17bd1fe2423ea86 tzdata94e.tar.gz ba0ee21d62a528303fcdfdbc6681e0ae3b0f342e1280dff45228bf13a9994ac91a5ff55861f21e76cf656df470581d7a279d670265344de64b11352bebb6fb74 tzdata94f.tar.gz f3308a08a0bdcca214a8a7d00017eaacaa93da212b79aa07293a4b89230f30f7f289ba93b3d9cfcf900be79be7869ddad1e05b61dc2e71df4f45e28180a7a5e3 tzdata94h.tar.gz c45771bb48023d248a0612b1fee61a61906249dc9c078e2133407ad655870a546b04076078bcb20e7757485f56e19b1953c04a7264f39d5a0d41b69b3ba74291 tzdata95b.tar.gz The .tar.gz files are all actually gzipped, and they decompress to the same .tar files that the .tar.Z files did: 722da97b88f58247e96874609ec6b4edbf8931167c97a56a8fd48aff01d883a5414327709602c248cbf02b2dea58f87a098c2e4c67dc45432225e7113a45189b tzcode93g.tar 7686bfa24e4a357dec0ec1a96961367bce5b2a54daabfcdbd369a7a61f2a77b4ab4783ec2337246fe63e905fdcea440b42451190ae753683930654993d83f233 tzcode94b.tar 86e037def6afdf12b3fa3c39c7b59a4fb38c2c8d576c1147063637aac05026ae1d727ad4434107fc00dc312b3f1f7d0c521c744182ad1a8f582c429e2c91cabe tzcode94c.tar 9872b757ee3a8bf0ed3de1f0875f878b977b34bd2b60440b23a8c7308a7514d12a5d5d283387973e61b75e1a8eebd27009aab78eee4e12f1f1a5593f71941328 tzcode94d.tar c3e68db0c13ce59937998a0d625bbfa90d06b70bca491f0b5618b9d0f59c6e249f1aae51167cf81607a3a2599f105bb22807d91264112cfa274d0fa397d5ba83 tzcode94e.tar 6c1954cd82eb77f78ca84e5e68d0a81603f928cb071640e5aafd6b463696e431ccdbda1b74cc5c13ce087f437e86a4172324caaf78285fc486e1263b412e54fe tzcode94g.tar ecfdea8487f1ca77fc7deda3ab5d83331fd0315418a8701e5501e6ad1d7760014e9e8308df00a9c6178e22077aa58d0f86f924845ecd7c8f12f9a7d3ce2121ae tzcode94h.tar 78faa3c7d59486ca1005b71af510321cc6feaac8c524eb59c32297e90ab607433cc09948297a8dd080b108c15eedc981d8b03e0359b0c2dce113a14a554f8c5f tzcode95b.tar f0a8c9c564800e8e5b310b0e404997968dfdc641026159e12d0d1fab336c2b573fadf1cc07e8b6a50af3eeb025443dc7137a969c731fc7f0e4475890f6763e19 tzcode95c.tar 2c2be092934d1919d3ef79324acf64a380c3e44cce532973cf30bed2fee9f7a01176bebc7a0a9408efc64822507698d134c3cd94cf4603ffa7b6cb8b5cc56452 tzdata93g.tar eefe885e5e13e22a09cc867e449539e70a1181b9e28c4530049f6a781f5796d4b7145b57183df4ecd12752f1ad8d7b0299c4c77129069da5702d0a9ea7c85640 tzdata94a.tar 8258bc3c560717b2cc7432fcdf6ceea4ea26fdd4a6b31513d3c80551be75ae7af2c5bea4b3b31ba61fb21ab57f75b41b3e9a1844dd931e8ab0e990a832b437c0 tzdata94b.tar e1524951a5c261cc8eef1f3adadf5ebf8143293745fda9411a0090d6bb6e19f899dd7a6e55fe2f8f490a167a3da392020e49b369984bc1f5cbed87d664eb58de tzdata94d.tar 831e8cddff91d2254d7a8846f5e7c48f5555deb7ea6d039b9efe33e977eaa47f26afb34ef835a782d60a85c72b4f32ea68cbe36fcf8c37eaaf1e32ad9539c926 tzdata94e.tar ada2a6dbf5727f4ee7af81b365a0ad346ca7d960b130ffad0d50239aba10e402afad836ca8690ce6739d3ef8d3fadcee4de7c5e39b32c7f9005dc594b0084eac tzdata94f.tar 5ef593c36bf39778c3e0faf5837809acb2d5b0d183028604433abd76ea7200e3e565cfa291f26dca59247e4628c7d3e557af69cee44a30c9bdd6a22335c44846 tzdata94h.tar fc1bb67d31a9f3bd6046548219e3af3d4788f3d1e4c861768607c4e6fefe30a6f4afe804df552c2957ac66b44e32f48c71f74d03759f9f414364bbc7fec09ad4 tzdata95b.tar -zefram
participants (4)
-
Brian Inglis
-
Paul Eggert
-
Robert Elz
-
Zefram