Re: [tz] Vanguard vs. "main" vs. rearguard
On 5/20/21 10:50 AM, Kerry Shetline wrote:
Is there a particular reason that the following situation is handled with bespoke awk processing:
The goal was to keep the data source simple, and put hacks into ziguard.awk when that's easy. I would rather not have these hacks bleed into the data source.
It would be great if the special comments would be all one needed to parse the data for rearguard, main, or vanguard.
I see where you're coming from, but that's not the goal here. I didn't want to invent a preprocessor for zic, creating technology that would have to be documented and supported and have backwards-compatibility problems of its own. The vanguard/main/rearguard stuff is just a hopefully-temporary hack that, once it evaporates, will require zero maintenance. By the way, is anybody still using rearguard format? If not, perhaps it's time to retire this stuff now. (That would save us all the hassle of conversations like this one. :-)
By the way, is anybody still using rearguard format? If not, perhaps it's time to retire this stuff now. (That would save us all the hassle of conversations like this one. :-)
Yes. We are unfortunately still dependent on the rearguard format. We are actively working to get away from that need and have been for some time now, but the size and complexity of the amount of code required to be updated here makes this a difficult and time consuming task. Unfortunately, we are dealing with years of legacy code that make a lot of bad assumptions about how DST works and the introduction of negative values greatly exposed this resulting in a substantial number of required changes on our part. So in the interim, we are still reliant upon rearguard until we can work through all the necessary changes to be fully compatible with vanguard formats. Making the switch to vanguard today absent of those changes would introduce a number of defects for us. I know this is not something this community wishes to support forever and I certainly understand that. But for the immediate time we are still dependent upon rearguard and would respectively ask for your support and patience while we continue to work through our necessary updates. Thanks, -Brandon On Thu, May 20, 2021 at 1:17 PM Paul Eggert via tz <tz@iana.org> wrote:
On 5/20/21 10:50 AM, Kerry Shetline wrote:
Is there a particular reason that the following situation is handled with bespoke awk processing:
The goal was to keep the data source simple, and put hacks into ziguard.awk when that's easy. I would rather not have these hacks bleed into the data source.
It would be great if the special comments would be all one needed to parse the data for rearguard, main, or vanguard.
I see where you're coming from, but that's not the goal here. I didn't want to invent a preprocessor for zic, creating technology that would have to be documented and supported and have backwards-compatibility problems of its own. The vanguard/main/rearguard stuff is just a hopefully-temporary hack that, once it evaporates, will require zero maintenance.
By the way, is anybody still using rearguard format? If not, perhaps it's time to retire this stuff now. (That would save us all the hassle of conversations like this one. :-)
On 5/20/21 1:47 PM, Brandon Smith wrote:
But for the immediate time we are still dependent upon rearguard and would respectively ask for your support and patience while we continue to work through our necessary updates.
Thanks for the info, and it sounds like we should keep rearguard support around for a while longer.
I personally believed rearguard data was going to be kept "forever" as a nod towards backward compatibility. Various downstream APIs and apps that use them probably assume "isDst == summer". I suspect downstream folks would value a roadmap of changes like this. Perhaps with a countdown in NEWS to keep it top of mind. More information below about why rearguard / vanguard matters to Android. Neil. ------------- ICU/CLDR also assume / require rearguard data, and (partly because of this) so does Android. i.e. Android devices will continue to want data updates using rearguard for years to come. Any API that takes a boolean to mean "isDst" / "daylight" or returns a boolean meaning "isDaylight" is affected and will return the opposite behavior after the move to vanguard. The problem remains on Android that behavior of APIs has to be supported, but they will change their behavior "overnight" if Android switches from using rearguard to vanguard with a data update. I think there were bugs I fixed that choked on negative DST so at least there won't be crashes. AOSP supports data updates for about 5 years. So, to stop relying on rearguard, a fix would have to be put in place, and then 5 years later Android probably wouldn't need to use rearguard any more. To fix this for future releases *safely*, I think downstream code like Android would need metadata at runtime (preferably mastered by TZDB) that indicates whether a zone ID is one of the ones that has flipped its behavior. Perhaps just a text file list of IDs. With that data, the function could invert the DST boolean at runtime for apps expecting rearguard behavior. This would need to be applied in the implementation of all affected APIs exposed by Android (e.g. ICU4J, java.util.TimeZone, java.time, possibly others). The alternative is to switch and hope for the best. AFAIK, nobody is currently working on this on Android, and it would take coordination with ICU to make the "rearguard / vanguard" behavior a runtime behavior change. On Fri, 21 May 2021 at 07:23, Paul Eggert via tz <tz@iana.org> wrote:
On 5/20/21 1:47 PM, Brandon Smith wrote:
But for the immediate time we are still dependent upon rearguard and would respectively ask for your support and patience while we continue to work through our necessary updates.
Thanks for the info, and it sounds like we should keep rearguard support around for a while longer.
-- Google UK Limited Registered Office: 6 Pancras Square, London, N1C 4AG Registered in England Number: 3977902
On Jun 7, 2021, at 12:21, Neil Fuller via tz <tz@iana.org> wrote:
I suspect downstream folks would value a roadmap of changes like this. Perhaps with a countdown in NEWS to keep it top of mind.
++ Cheers! |---------------------------------------------------------------------| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |---------------------------------------------------------------------| | A room without books is like a body without a soul. | | | | -- Cicero | |---------------------------------------------------------------------
On 6/7/21 9:21 AM, Neil Fuller wrote:
To fix this for future releases *safely*, I think downstream code like Android would need metadata at runtime (preferably mastered by TZDB) that indicates whether a zone ID is one of the ones that has flipped its behavior.
Surely it'd have to be more complicated than that, as a single Zone can have negative DST some years, and positive DST other years. See Africa/Windhoek, for example. But perhaps I'm not understanding the problem correctly, as I don't see why tzdb would need to maintain a separate set of data for this issue. Android libraries could have a flag (or multiple versions, or however Android folks like to do software changes) controlling whether an application can tolerate negative DST. If the flag says "negative DST is OK", the libraries can pass through data unscathed. If the flag says "I don't want to see negative DST", then whenever the underlying data says DST is negative, the Android library can swap standard and daylight time and give the swapped info to the app. As I understand it, Android libraries are already doing the latter; if so, all that needs to be added is a flag so that they can do the former. Anyway, I appreciate the info about why rearguard support will be needed for quite some time.
I think you're correct it would have to be more complicated: I was thinking about Dublin, not Windhoek. The main point I'd make about Android challenges associated with TZDB is... When TZDB makes a fundamental change (rearguard or other recent proposals), Android has to formulate a plan for: (1) how to arrange things for new apps on new devices to move things forward (which requires a good understanding of where TZDB is going) (2) how to ensure old apps still get old behavior on new devices (which requires a way to continue to provide that old behavior) (3) how to keep existing devices running with latest TZDB data (and which do not take code changes) until they drop out of the support window. This can take years (which requires a degree of stability of file formats / conventions or Android risks user-facing breakages / changes) Where possible, Android also doesn't want to master too much of its own data and would prefer to adopt changes on its own, usually yearly, major release schedule. AFAIK, this is possible with things like the "slim" format: slim *probably* can't be used on existing devices. Because Android typically doesn't get to update code for TZDB updates, only data, it will stick with the "fat" format for existing devices. Android is downstream of various open source projects that themselves are downstream of TZDB. If TZDB *doesn't* master information, and something is left to each project to work out, they could jump in different directions. e.g. if two projects want country-specific zone IDs, and they don't come from TZDB, then there's a risk of one of them "inventing" (should the need arise) Europe/Glasgow as "the zone for Scotland", and another inventing Europe/Edinburgh to fill the same role. There goes our interoperability. Different projects going in different directions was a risk with rearguard, too, but everything we currently care about (ICU and our own code) stuck with rearguard so we thankfully haven't had to deal with it. I appreciate these problems are well outside of your scope. Until recently I was handling TZDB updates on the Android side, so I may have an unusual perspective among readers of this list. As always, thanks for the hard work. Neil. On Mon, 7 Jun 2021 at 18:01, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 6/7/21 9:21 AM, Neil Fuller wrote:
To fix this for future releases *safely*, I think downstream code like Android would need metadata at runtime (preferably mastered by TZDB) that indicates whether a zone ID is one of the ones that has flipped its behavior.
Surely it'd have to be more complicated than that, as a single Zone can have negative DST some years, and positive DST other years. See Africa/Windhoek, for example.
But perhaps I'm not understanding the problem correctly, as I don't see why tzdb would need to maintain a separate set of data for this issue. Android libraries could have a flag (or multiple versions, or however Android folks like to do software changes) controlling whether an application can tolerate negative DST. If the flag says "negative DST is OK", the libraries can pass through data unscathed. If the flag says "I don't want to see negative DST", then whenever the underlying data says DST is negative, the Android library can swap standard and daylight time and give the swapped info to the app. As I understand it, Android libraries are already doing the latter; if so, all that needs to be added is a flag so that they can do the former.
Anyway, I appreciate the info about why rearguard support will be needed for quite some time.
-- Google UK Limited Registered Office: 6 Pancras Square, London, N1C 4AG Registered in England Number: 3977902
On 6/9/21 9:43 AM, Neil Fuller wrote:
Because Android typically doesn't get to update code for TZDB updates, only data, it will stick with the "fat" format for existing devices.
I can understand Android doing that, given old apps that read TZif files directly and that predate RFC 8536 or have bugs related to RFC 8536. I expect it'll take many years for RFC 8536 to proliferate.
e.g. if two projects want country-specific zone IDs, and they don't come from TZDB, then there's a risk of one of them "inventing" (should the need arise) Europe/Glasgow as "the zone for Scotland", and another inventing Europe/Edinburgh to fill the same role.
Yes, that's very much a concern. And it's not just Scotland: it could happen with the Oduduwa Republic or any number of other plausible future countries created by political disputes. This is a primary motivation for the guidelines not requiring a separate Zone per country.
participants (4)
-
Brandon Smith -
Fred Gleason -
Neil Fuller -
Paul Eggert