Lester Caine wrote:
the change that pulls support for specifically 32 bit systems
That change was not about adding or removing support for 32-bit systems, which were (and still are) the most commonly-used tz platform. That change was about improving support for 64-bit systems.
so up until that time data HAS been gathered for the whole of time not just post 1970....
Two things here. First, once we establish the need for a new zone, we attempt to gather high-quality pre-1970 data for it. This doesn't override the guideline that we don't bother to create new zones merely because their timestamps differ before 1970. Second, although the Theory file attempts to document the practices, it often lags because we don't get around to writing everything down. For example my recent proposed patch "Document some longstanding unwritten guidelines." <http://mm.icann.org/pipermail/tz/2013-September/019856.html> says that names cannot end in '/', a guideline that's been faithfully followed since the project began (because the POSIX implementation prohibits names ending in '/') even though nobody got around to writing it down until this month.
finally there is a formal definition of the SCOPE of the database
The guideline about not bothering to create national zones that differ only before 1970 has been in practical use throughout the maintenance of the tz database. In one case (America/Montreal) we departed from it, but that was only because we incorrectly thought that Montreal differed from Toronto in the early 1970s. That timestamp error has been corrected in the data; the extra zone survives only because we didn't want to discard existing data. We would like to allow zones to be added even if they exceed the current guideline. These zones are currently out of scope, and most users won't want them (as they'll be presented with unnecessary choices when selecting TZs, which makes their jobs harder), but a few will want them and if the data are high quality I don't see any reason to exclude from the database. It's just that we need a mechanism for winnowing out these extra zones, as I expect we don't want them installed by default on POSIX platforms. Zefram has proposed one mechanism, we're currently evaluating it, and I hope and expect that it or something like it will appear in the tz database soon. Once that happens, we'll have the mechanism we need, and a place for these new zones.
POSIX is just an implementation detail
Yes and no. Yes, because there are tz parsers and runtimes that run on non-POSIX systems. No, because the restrictions that POSIX imposes are realistic restrictions if we want the code and data to be portable.