Thanks for the info. I'm just having a difficult time understanding why both tzcode and tzdata use gzip, while tzdb uses lzip. Why not use gzip for all? Or even better Zstandard for all? Come to think of it, I think it would be better to switch to Zstandard as it's much more popular than lzip. On Thu, May 5, 2022 at 6:21 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

...

they indicated they still wanted gzip to be considered specifically to increase “availability for low end systems as lzip isn’t that widely supported”:

Unfortunately I'm not quite seeing the use case here. If it's a low-end system that supports only gzip-style compression, presumably it is an end-use system that merely needs access to the data and it should get tzdataXXXXX.tar.gz. In contrast, the tzdb tarball contains code intended for use by developers, who are expected to have access to software tools like cc and lzip that a low-end system might not have, and developers can easily gain access to these tools if they don't have them already.

If we were to start from scratch now, Zstandard (Internet RFC 8878) would be a good choice for distributing the data as it outperforms gzip on both speed and compression ratios (though it doesn't compress as well as the much-slower lzip), and my impression is that Zstandard is gaining in popularity. Perhaps we could start by encouraging the distribution of tzdata.zi in Zstandard format in addition to what we already distribute; to represent all of TZDB 2022a's data, tzdata.zi.zst needs just 24,604 bytes, which is nearly as good as tzdata.zi.lz's 22,460 bytes.

One more thing. We shouldn't assume that iana.org as a utility that billions of devices can blithely access for time zone data. If you're building an IoT application for low-end devices, please arrange to have your downstream servers serve up time zone data. And if you do that, you can use any compression format you like.

I apologize for what is certainly an off-topic rant, but:

...

Why not use gzip for all?

gzip doesn't compress as well as lzip.

But gzip compresses well enough, and storage is almost infinitely cheap.

Also, the benefits of switching to a better compression algorithm need to be weighed against the maintenance hassle of adapting to the switch.

Absolutely. And that hassle is HUGE. Every few years, I have to replace my monitor, after putting my fist through it out of frustrated rage, because I've just downloaded some package I'm interested in, but first I have to go through the yak-shaving exercise of downloading and building a decompressor for the nifty new "better" decompression algorithm it uses. (I'm kidding about the broken monitor part, but just barely.) But, clearly, I'm just a cranky old Luddite. Progress marches on. Please, let's not clutter this list with a bunch of zealous endorsements of how Very Much Better the latest compression algorithms are, and how stubborn I am for failing to appreciate them.