On 25 January 2018 at 11:20, <Paul.Koning@dell.com> wrote:
What concerns me is that it seems to have been suggested as something that could just be added without any backward compatibility concerns, and that is not correct. If people are given adequate notice of this new thing so they can update the tools in advance, and the test mechanism is standardized so that it can be reliably filtered, then it is probably ok.
If we are so concerned about introducing any potentially-breaking changes as it seems this list is, then this may even warrant an initial out-of-band email to tz-announce@. (And I think this is a big enough procedural change to warrant that anyway.) I'd imagine a procedure something like this:
- Once there is sufficient consensus here on adding a Test namespace, we announce that the first release after a certain appointed date (e.g., ~90 days from announcement?) will have a new Test namespace.
- Until the appointed date, business as usual for real data changes only.
- On or after the appointed date, the Test namespace is created and added to the development branch, and the build process ignores it by default, but allows a flag to build it. It contains a single zone on perpetual -00.
- The first regular release after that (again, based on real data changes) will contain this single-zone test file, and will also announce a date when actual test cases will be added.
- We follow a similar procedure for adding the initial test cases, which reflect as accurately/succinctly as possible the actual limitations (and lack thereof) of the tzdb format as it exists today.
- After a year or two, we issue a separate announcement that the first release after a certain appointed date will enable building the tests by default, but there will be a flag to ignore it, and we follow through as above.
Principles we'd follow would include:
- We never roll a new release just for test cases without the need to update real data, similar to the current de facto policy for code updates. The only exception might be if we ever manage to get to the point where we're going more than ~2–3 years between data updates (ha!), it may be worthwhile to continue the habit of preparing a release with tzcode and tests only and encouraging downstream updates, just so it isn't forgotten that despite the then-current stability, these things still aren't set in stone.
- Over time, new "features" are added first as test cases (possibly even in an explicit Test/Beta sub-namespace) where development is deemed necessary — such as other calendars (which has already come-up and been worked around in the short term) or other timescales/epochs (TAI, Mars time??), which may otherwise crop up at any time.
- Likewise, if future endeavors may require dropping or fiddling with certain aspects of POSIX-compliance, we test them first.
This project has always and will necessarily continue to evolve, so let's take some steps in the short-term to ensure that we and our downstream consumers are ready to achieve those longer-term necessities.