I'm not an expert in the native ICU APIs, but the older Java APIs have tended to simplify the complex world of time zones and there are decades of apps written against this simplified model. Apps probably also make additional assumptions like "DST is always a positive adjustment", "DST is always in the summer", "!daylight == standard time". This makes the compatibility story hard if the data we use switches convention when applying a tzdb data update, especially if the change doesn't take place on a well-defined Android release boundary. Android applications expect occasional variations in behavior but we try to make sure that larger changes tend to take place on Android release boundaries (which apps can check for) and we often try to limit behavior changes based on an applications "target API level" (which indicates the release the app was tested against and is an indicator for the behavior they expect).
As an example of API simplification, methods like TimeZone.getDisplayName(boolean, int, Locale) [1] take a boolean for "daylight". Therefore they assume two offsets a year, not many (I believe this breaks down in some places near the poles). If developers take the value from the inDaylightTime(Date) method to determine the first parameter for a given time, and everything is using the same convention, then they'll probably get their expected behavior regardless of whether it's vanguard of rearguard data being used. If they wanted "summer" or "winter", to pick the time to pass to inDaylightTime(), they'd need to know whether a zone was in the northern or southern hemisphere (not readily available but perhaps guessable from the first part of the Olson ID but that's not ideal). I suspect a lot of code just hardcodes "false" and "true" expecting the rearguard (daylight == summer, !daylight = standard) convention.
Note, also, the method and others like it don't take a parameter for "when", so tend to assume that the "standard time" convention for "daylight" and "!daylight" are also consistent for all time for a given zone.
I'm struggling to come up with a way to preserve API behavior for Android applications if we switched from rearguard to vanguard as it's a build-time decision. I think I'm right in saying there's no metadata in the zic output that would reliably allow code to selectively "reverse out" the convention change at runtime for affected zones / time periods, hence we would end up with heuristics like "are any of the dst changes (effectively) negative?". If we could determine what's going on, perhaps we could continue to present the old behavior for older apps on newer devices so they get the behavior they've always expected. I assume if rearguard stops including negative DST there would be no easily available metadata in the rules that could be extracted either. Of course, we'd still like rearguard for older devices, as mentioned above.
Ultimately I see this ending with one of: keeping rearguard as it is forever, somebody compiling their own metadata, possibly API changes, and/or a fork occurring. It all seems like a lot of pain.