Adding verified historic details
Paul I sat down and worked my way through the UK details to see what is documented and what not. I'm normally using the results rather than the raw data, but I think I understand how this works now. I'm still using my own code in parallel with the version of TZ that Derick has built into the PHP clock tools so the main thing here is now comparing results. I've even got the original emails still archived. I've been using the 1880 'legal' date for the adoption of GMT and calling 'the other time' Railway time, but I can accept combining them. However the documents are all legal ones so using 'legal' time so midnight for a change was legal midnight not GMT until 1880 - that's politics for you :) As far as I am concerned, the Isle of Man was using LMT until 1883 and we now have a date of 30th March to go with that. I've always worked with 18 minutes offset, although Tynwald which is always quoted as the reference is 4.4814degW - 17m55.5s and that goes back 1000 years, but I think we can skip earlier solar times. So looking at the Europe data file ... # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Europe/London -0:01:15 - LMT 1847 Dec 1 0:00s 0:00 GB-Eire %s 1968 Oct 27 1:00 - BST 1971 Oct 31 2:00u 0:00 GB-Eire %s 1996 0:00 EU GMT/BST Link Europe/London Europe/Jersey Link Europe/London Europe/Guernsey Link Europe/London Europe/Isle_of_Man I think I need to add ... # AT4 of 1883 - The Statutory Time et cetera Act 1883 - Zone Europe/Isle_of_Man 0:17:55 - LMT 1883 March 30 0:00s Then the London Rules apply from then so in my simple way of thinking Link To Europe/London ( date is optional as on line above? ) Fills the picture. Europe/Isle_of_Man exists in it's own right up to 1883, and then it follows Europe/London. I'm fairly sure from the IOM documents that Jersey and Guernsey followed the same date, but their archives have not got that far back yet. So do we have to duplicate the whole zone record each timezone to add this one item of data? - - Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Paul Eggert wrote:
On 09/04/13 08:46, Lester Caine wrote:
do we have to duplicate the whole zone record each timezone to add this one item of data?
I'm afraid so. It's not fun.
http://lsces.org.uk/hg/tz/rev/b30b17f4191a -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Lester Caine wrote:
Your LMT offset has the wrong sign. -zefram
Zefram wrote:
Lester Caine wrote:
http://lsces.org.uk/hg/tz/rev/b30b17f4191a Your LMT offset has the wrong sign.
I was looking at the Dublin time as well ;) Fixed in my repo - TA -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Wed, 4 Sep 2013, Lester Caine wrote:
Fills the picture. Europe/Isle_of_Man exists in it's own right up to 1883, and then it follows Europe/London.
Well - subject to confirming it did so during WW2, as you drew my attention to that issue. The (UK) Emergency Powers (Defence) Act, 1939 includes provision (subsection 4(1)(a)) for it to be extended to the Isle of Man by Order in Council, but I don't have a specific reference to such an Order in Council (presumably a UK SR&O) doing so. (All the subsequent regulations then worked by modification of an Act also applying to the Isle of Man, so if they had the power to apply there then they would have done so.)
I'm fairly sure from the IOM documents that Jersey and Guernsey followed the same date, but their archives have not got that far back yet. So do we have to duplicate the whole zone record each timezone to add this one item of data?
We have dates of legal adoption of GMT in Jersey (around 4pm on Saturday 11 June 1898) and Guernsey (18 June 1913). What we don't have is full details from WW1 or WW2, beyond the quote about time being put forward under occupation in WW2 (https://en.wikipedia.org/wiki/File:Orders_of_the_German_Commandant_Evening_P... gives one transition time as 11pm on 2 July 1940 in Jersey and we do have reasonably detailed information on German time from the PTB). Certainly if the cut-off is 1945 or before, you need different data for them than for Europe/London and we don't have the full information (whereas after 1945, the links to Europe/London are probably correct). -- Joseph S. Myers jsm@polyomino.org.uk
Joseph S. Myers wrote:
On Wed, 4 Sep 2013, Lester Caine wrote:
Fills the picture. Europe/Isle_of_Man exists in it's own right up to 1883, and then it follows Europe/London.
Well - subject to confirming it did so during WW2, as you drew my attention to that issue. The (UK) Emergency Powers (Defence) Act, 1939 includes provision (subsection 4(1)(a)) for it to be extended to the Isle of Man by Order in Council, but I don't have a specific reference to such an Order in Council (presumably a UK SR&O) doing so. (All the subsequent regulations then worked by modification of an Act also applying to the Isle of Man, so if they had the power to apply there then they would have done so.)
I've been slowly reading through the IOM Hansard documents but not found any discrepancies yet, but I have a few years to go ...
I'm fairly sure from the IOM documents that Jersey and Guernsey followed the same date, but their archives have not got that far back yet. So do we have to duplicate the whole zone record each timezone to add this one item of data?
We have dates of legal adoption of GMT in Jersey (around 4pm on Saturday 11 June 1898) and Guernsey (18 June 1913). Can I take those dates as facts we can add to the tz data? Is there any indication of the islands using a different time prior or like the main land they were already using 'GMT' and this just formalised it? Typically I can't find the reference to Channel Islands again, but it was probably the later Summer Time changes I'm mixing up.
What we don't have is full details from WW1 or WW2, beyond the quote about time being put forward under occupation in WW2 (https://en.wikipedia.org/wiki/File:Orders_of_the_German_Commandant_Evening_P... gives one transition time as 11pm on 2 July 1940 in Jersey and we do have reasonably detailed information on German time from the PTB). Certainly if the cut-off is 1945 or before, you need different data for them than for Europe/London and we don't have the full information (whereas after 1945, the links to Europe/London are probably correct).
I was on Jersey earlier in the year and certainly read up about that but I've misplaced the paper notes I made at the time :( Paul can I use rules from another zone to fill this gap? I presume so as we have different rules already, but I don't think I've digested enough to know how. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Paul Eggert wrote:
On 09/04/13 13:09, Lester Caine wrote:
Paul can I use rules from another zone to fill this gap?
Sure, just note down your assumptions in a comment. I do that all the time.
Following the style of the patch I've already created? Is that good enough to merge? If so I'll look to expand the Channel Island data ... -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On 09/04/13 13:43, Lester Caine wrote:
Is that good enough to merge?
I'm afraid we can't add zones that differ only in pre-1970 dates until we can figure out how to filter them out in the default case, as we aren't prepared for the hundreds or thousands of new zones that would be feasible if we opened the floodgates. Zefram has one prototype for filtering, which I really must get to at some point.
On 4 September 2013 21:53, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 09/04/13 13:43, Lester Caine wrote:
Is that good enough to merge?
I'm afraid we can't add zones that differ only in pre-1970 dates until we can figure out how to filter them out in the default case, as we aren't prepared for the hundreds or thousands of new zones that would be feasible if we opened the floodgates. Zefram has one prototype for filtering, which I really must get to at some point.
The Isle of Man is already in the main tzdb, so no new ID is being created and nothing changes for the selection code. Changing a Link to a Zone because more data is now available is perfectly reasonable and must not be a blocker here. Were there a request to create a new zone ID that is not currently present that would be an entirely different matter. Stephen
Paul Eggert wrote:
On 09/04/13 13:43, Lester Caine wrote:
Is that good enough to merge? I'm afraid we can't add zones that differ only in pre-1970 dates until we can figure out how to filter them out in the default case, as we aren't prepared for the hundreds or thousands of new zones that would be feasible if we opened the floodgates. Zefram has one prototype for filtering, which I really must get to at some point.
Paul - this is the point of contention ! Deleting any data is wrong. If users have a reason for only viewing part of the data then fair enough, but we have the opportunity here to get this right before it becomes embedded in every browser. That does not want a broken set of timezones with this arbitrary cutoff - THAT is the very thing some of us want to get away from. If you want a 1970+ only database then it should be advertised as that, and we can create a second one which has all of time ... any distribution and API should be using the full range, just as their calendar should be accurate for historic dates. The only time hiding data is acceptable is when the user has good reason to do so. The distributed data does not have that luxury! -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Sep 4, 2013, at 2:15 PM, Lester Caine <lester@lsces.co.uk> wrote:
If you want a 1970+ only database then it should be advertised as that,
As far as I know, he wants the ability for packagers of the database to create, from the tzdb, 1970+-only databases if they deem pre-1970 data not worth keeping, not to permanently expunge pre-1970 differences from the tzdb itself.
and we can create a second one which has all of time
If we're not going to be adding new tzids by splitting zones that differ only pre-1970, and there are currently tzids that cover a single region that has different pre-1970 behavior in different parts of the region, we're not going to have "a second one that has all of time" accurately.
any distribution and API should be using the full range, just as their calendar should be accurate for historic dates.
Any distribution and API has the option to use whatever they choose; some particular UN*X might well themselves choose to merge tzids that differ only pre-1970. That's not for us to dictate. (Heck, as somebody noted, QNX has an *unsigned* seconds-since-the-Epoch time_t, so it's literally *impossible* to specify a pre-January 1, 1970, 00:00:00 UTC date as a time_t in QNX, so anything in QNX that use the tzdb to convert time_t's will never look at the pre-1970 data in the tzdb.)
Guy Harris wrote:
On Sep 4, 2013, at 2:15 PM, Lester Caine <lester@lsces.co.uk> wrote:
If you want a 1970+ only database then it should be advertised as that, As far as I know, he wants the ability for packagers of the database to create, from the tzdb, 1970+-only databases if they deem pre-1970 data not worth keeping, not to permanently expunge pre-1970 differences from the tzdb itself. Then it's up to the distributions to do that - in a manor that is compatible wth the rest of their infrastructure ... such as not having a pre-1970 calendar.
and we can create a second one which has all of time If we're not going to be adding new tzids by splitting zones that differ only pre-1970, and there are currently tzids that cover a single region that has different pre-1970 behavior in different parts of the region, we're not going to have "a second one that has all of time" accurately. A proposal has been put forward to prevent the creation of new timezones just for different locations prior to standard time being adopted. Pre-standard time LMT is assumed and calculated independent of TZ.
If there is documentary evidence of a different time pattern such has been added for the Isle of Man and is about to appear for the channel islands then it should be allowed not blocked. Where different locations adopt standard time at different documented times then the source file format needs to be change to accommodate that. I proposed that the location and date is recorded and then links to the following timezone for the rest of the data. The Isle of man is a perfect example of this since all it differs in is the date of the start of using GMT.
any distribution and API should be using the full range, just as their calendar should be accurate for historic dates. Any distribution and API has the option to use whatever they choose; some particular UN*X might well themselves choose to merge tzids that differ only pre-1970. That's not for us to dictate. (Heck, as somebody noted, QNX has an *unsigned* seconds-since-the-Epoch time_t, so it's literally *impossible* to specify a pre-January 1, 1970, 00:00:00 UTC date as a time_t in QNX, so anything in QNX that use the tzdb to convert time_t's will never look at the pre-1970 data in the tzdb.) So why are you dictating that pre 1970 data will be stripped? "That's not for us to dictate."
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Lester Caine <lester@lsces.co.uk> writes:
A proposal has been put forward to prevent the creation of new timezones just for different locations prior to standard time being adopted.
Just to be clear, this isn't really a proposal; rather, this is the existing maintenance practice for the tz database that's been followed fairly consistently for many years now. In other words, it's the existing standard, not the proposal. Therefore, to be pedantically correct, the proposal (i.e., the change from existing practice) is to permit introduction of new zones that differ only in pre-1970 dates, something that the tz project has not previously done.
If there is documentary evidence of a different time pattern such has been added for the Isle of Man and is about to appear for the channel islands then it should be allowed not blocked.
There are certainly several people here advocating for this (new!) practice, and the packrat in me that likes having as much data as possible approves of the idea of incorporating whatever data people are willing to do the work to research, at least if it's practical to do so. (I question whether it's practical to represent some of the historic time practices, such as using one time zone inside railway stations and a different one for municipal time, but we can cross that bridge when someone has time to put forward a complete proposal for how it would be done. If anyone ever does.) However, this is a divergence from existing practice, and it sounds like Paul would prefer to have a winnowing mechanism in place before changing that maintenance rule. That seems like an entirely reasonable position to me. We haven't had or tracked that data for years; waiting a few more months (if that) until a winnowing process can be properly reviewed and adopted isn't going to hurt anything. And, in the meantime, the data that people are gathering can be documented in the form of patches here or Git branches or a number of other methods, so it won't be lost when there is a project in a position to adopt it.
Where different locations adopt standard time at different documented times then the source file format needs to be change to accommodate that. I proposed that the location and date is recorded and then links to the following timezone for the rest of the data. The Isle of man is a perfect example of this since all it differs in is the date of the start of using GMT.
This sounds like a good idea to me, although I don't know what difficulties it poses for the rest of the software. It would indeed be nice to not have to duplicate all the rules into a separate zone when the zones are identical after a certain date, and it also seems like it would make some possible winnowing strategies more straightforward. -- Russ Allbery (rra@stanford.edu) <http://www.eyrie.org/~eagle/>
Russ Allbery wrote:
Where different locations adopt standard time at different documented
times then the source file format needs to be change to accommodate that. I proposed that the location and date is recorded and then links to the following timezone for the rest of the data. The Isle of man is a perfect example of this since all it differs in is the date of the start of using GMT. This sounds like a good idea to me, although I don't know what difficulties it poses for the rest of the software. It would indeed be nice to not have to duplicate all the rules into a separate zone when the zones are identical after a certain date, and it also seems like it would make some possible winnowing strategies more straightforward.
Fix the current holes in the system so that we can handle all the missing data before writing software to take it way again? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Lester Caine <lester@lsces.co.uk> writes:
Russ Allbery wrote:
This sounds like a good idea to me, although I don't know what difficulties it poses for the rest of the software. It would indeed be nice to not have to duplicate all the rules into a separate zone when the zones are identical after a certain date, and it also seems like it would make some possible winnowing strategies more straightforward.
Fix the current holes in the system so that we can handle all the missing data before writing software to take it way again?
It sounds like you may have misunderstood the intent of winnowing. Winnowing provides a mechanism to generate selected subsets of the raw database so that, for example, user time zone selection is not confused by multiple time zones that, from the perspective of the average user, have no interesting differences. It doesn't take any data away; the raw data is still present. It just provides ways of generating reduced views on that data. The specific concern here as I understand it, and the concern I would have when looking to deploy software based on the tz database, is that the addition of more correct pre-1970 data will, if done properly, result in an explosion of new zones that are not of interest to the vast majority of users of the tz database. Given that, we should provide some mechanism to present them with the collapsed post-1970 view (at least) prior to creating that explosion of new zones so that they can choose the level of granularity at which they want to operate. -- Russ Allbery (rra@stanford.edu) <http://www.eyrie.org/~eagle/>
On Sep 4, 2013, at 2:57 PM, Lester Caine <lester@lsces.co.uk> wrote:
Guy Harris wrote:
On Sep 4, 2013, at 2:15 PM, Lester Caine <lester@lsces.co.uk> wrote:
If you want a 1970+ only database then it should be advertised as that, As far as I know, he wants the ability for packagers of the database to create, from the tzdb, 1970+-only databases if they deem pre-1970 data not worth keeping, not to permanently expunge pre-1970 differences from the tzdb itself. Then it's up to the distributions to do that - in a manor that is compatible wth the rest of their infrastructure ... such as not having a pre-1970 calendar.
Well, it's up to them to do whatever they choose, including merging tzids that correspond to regions that differed only prior to 1970 *and* having calendar software that doesn't stop at 1970. You may disapprove of that choice, but you should take that up with them.
and we can create a second one which has all of time If we're not going to be adding new tzids by splitting zones that differ only pre-1970, and there are currently tzids that cover a single region that has different pre-1970 behavior in different parts of the region, we're not going to have "a second one that has all of time" accurately. A proposal has been put forward to prevent the creation of new timezones just for different locations prior to standard time being adopted. Pre-standard time LMT is assumed and calculated independent of TZ.
What about locations that are in a single tzdb zone, so that they currently don't differ in post-standard-time, but that we discover didn't all maintain the same GMT offset subsequent to the adoption of standard time or didn't follow the same rules for "shifted time" (to avoid the term that implies that daylight is being saved and the term that implies that this shift only happens during the summer :-))? And what does the adoption of standard time consist of? Is "standard time" only specified by the actions of a government (whether national or sub-national or local), so that standard time can only be adopted by a government, or does it include some form of non-governmental standardized time (e.g., railway time) very broadly used? For the former, we can (at least for national governments, and probably sub-national governments in many nations, and perhaps even local governments) establish a fairly firm date for the adoption; for the latter, that might be more difficult to establish, unless we simply decide to say that, once the railways agreed on standardized time, "OK, these regions are on standard time" and ignore people who weren't keeping railway time.
If there is documentary evidence of a different time pattern such has been added for the Isle of Man and is about to appear for the channel islands then it should be allowed not blocked.
As, *for those locations*, that creates no new tzids (there were already tzids for the Isle of Man and, that's fine with me and, I suspect, Paul. If, however, we were to discover documentary evidence that, for example, some towns in New Jersey chose not to adopt daylight savings time in 1918: https://en.wikipedia.org/wiki/1918_Standard_Time_Act supporting *that* in the tzdb would require splitting America/New_York and creating a new tzid or tzids for those towns.
So why are you dictating that pre 1970 data will be stripped?
Mu! I'm *not* dictating that pre-1970 data will be stripped from either the tzdb or from the version of the tzdb some supplier distributes, I'm just saying I won't get in the way of somebody stripping pre-1970 tzid differences from the copy of the tzid they distribute.
Guy Harris wrote:
What about locations that are in a single tzdb zone, so that they currently don't differ in post-standard-time, but that we discover didn't all maintain the same GMT offset subsequent to the adoption of standard time or didn't follow the same rules for "shifted time" (to avoid the term that implies that daylight is being saved and the term that implies that this shift only happens during the summer :-))? If the system can't currently handle a fact it needs fixing so it does. But this one is covered by the proposal below.
And what does the adoption of standard time consist of? Is "standard time" only specified by the actions of a government (whether national or sub-national or local), so that standard time can only be adopted by a government, or does it include some form of non-governmental standardized time (e.g., railway time) very broadly used? For the former, we can (at least for national governments, and probably sub-national governments in many nations, and perhaps even local governments) establish a fairly firm date for the adoption; for the latter, that might be more difficult to establish, unless we simply decide to say that, once the railways agreed on standardized time, "OK, these regions are on standard time" and ignore people who weren't keeping railway time. I already have documentary evidence for a second layer of offsets and I could make an argument for that, but it needs a more major change to the record structure to return multiple timezones so that is on my back burner.
I've listed elsewhere a couple of changes that will both save space in reusing existing timezones, and allow adding single events such as different start dates and then 'link' to the following timezone sequence. Any documented change to changes in time should be capable of being documented. The adoption of GMT related standard time is one which is fairly clear cut, but any previous local standard should be allowed, and LMT seems to be the norm prior to having clocks accurate enough to show the differences across a country so is a natural background.
If there is documentary evidence of a different time pattern such has been added for the Isle of Man and is about to appear for the channel islands then it should be allowed not blocked. As,*for those locations*, that creates no new tzids (there were already tzids for the Isle of Man and, that's fine with me and, I suspect, Paul. But why should what facts are allowed be restricted by some 'mistake' in the past? A fact is a fact ...
If, however, we were to discover documentary evidence that, for example, some towns in New Jersey chose not to adopt daylight savings time in 1918: https://en.wikipedia.org/wiki/1918_Standard_Time_Act supporting*that* in the tzdb would require splitting America/New_York and creating a new tzid or tzids for those towns.
NOT supporting a fact means that anybody who is looking for the correct historic data is not going to get it. If it is not going to be added to TZ then we need a TZ+ that does have it - end of story ... the same with every new fact that is established. We need to split the date and code to create two repositories, fork the data repository and then we can get on with TZ+ ... -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Sep 4, 2013, at 4:22 PM, Lester Caine <lester@lsces.co.uk> wrote:
Guy Harris wrote:
What about locations that are in a single tzdb zone, so that they currently don't differ in post-standard-time, but that we discover didn't all maintain the same GMT offset subsequent to the adoption of standard time or didn't follow the same rules for "shifted time" (to avoid the term that implies that daylight is being saved and the term that implies that this shift only happens during the summer :-))? If the system can't currently handle a fact it needs fixing so it does.
Even if the fix involves splitting tzdb zones and creating new tzids for the new zones?
If there is documentary evidence of a different time pattern such has been added for the Isle of Man and is about to appear for the channel islands then it should be allowed not blocked. As,*for those locations*, that creates no new tzids (there were already tzids for the Isle of Man and, that's fine with me and, I suspect, Paul. But why should what facts are allowed be restricted by some 'mistake' in the past? A fact is a fact ...
Yes, but facts can also be ignored by declaring them "out of scope" and not promising to provide correct time conversions for times prior to 1970. If you're really suggesting that there not be any restriction on adding new tzdb zones due to pre-1970 differences, then, *if* there is ever a case where correcting the historical data in the tzdb requires splitting a zone (rather than, say, just turning a link into a separate zone, as is the case for the Isle of Man and Channel Islands changes), then either 1) you'll have to convince the tzdb maintainer to allow that or 2) you'll have to fork the tzdb into your own database. It may be that, with tools to allow a packager of the tzdb to winnow out changes that they view as "out of scope", so as not to have to force users to choose between zones when, for their purposes, the two zones act the same (as I suspect would be the case for many pre-1970 differences on many UN*X systems), the tzdb can maintain the history, and add new tzdb zones as necessary, without inconveniencing all its users.
If, however, we were to discover documentary evidence that, for example, some towns in New Jersey chose not to adopt daylight savings time in 1918: https://en.wikipedia.org/wiki/1918_Standard_Time_Act supporting*that* in the tzdb would require splitting America/New_York and creating a new tzid or tzids for those towns.
NOT supporting a fact means that anybody who is looking for the correct historic data is not going to get it.
People looking for correct historical data in the tzdb need to start by reading the Theory file and, in particular, the Clock transitions before 1970 are recorded for each such location, because most POSIX-compatible systems support negative time stamps and could misbehave if data were omitted for pre-1970 transitions. However, the database is not designed for and does not suffice for applications requiring accurate handling of all past times everywhere, as it would take far too much effort and guesswork to record all details of pre-1970 civil timekeeping. As noted in the README file, the tz database is not authoritative (particularly not for pre-1970 time stamps), and it surely has errors. Corrections are welcome and encouraged. Users requiring authoritative data should consult national standards bodies and the references cited in the database's comments. part of the Theory file, and realize that there is no guarantee that you're getting correct historic data.
On Thu, 05 Sep 2013, Lester Caine wrote:
We need to split the date and code to create two repositories, fork the data repository and then we can get on with TZ+ ...
No, please let's not fork the database. Lets find a way for the tz project to maintain all the information, in a way that satisfies as many people as possible, including (a) those who want thousands of new zones to accommodate pre-1970 time differences, and (b) those who want to install or use subsets of the database that do not have those thousands of new zones. It seems to me that the following changes or additions would be sufficient: 1. Code to extract a subset of zone names according to specified criteria; for example, by selecting only one of several zones that differ pre-1970 but are identical post-1970. Zefram has written a prototype of this code. 2. A process to allow OS vendors, system vendors, packagers, sysadmins, and even end users, to install or use only a subset of the data; for example, by ignoring zones that differ from other zones only in pre-1970 data. Zefram has produced a prototype of such code. 3. (Optional) New notation in the zic input files to make it easier to express the fact that, after some date (or between some pair of dates), one zone follows exactly the same rules as another zone. 4. (Optional) Changes in zic and third party parsers to handle the input notation changes. 5. Guidelines to allow the creation of new zones that differ only for pre-1970 times. 6. Data for new zones that differ only for pre-1970 times. This could be contributed by volunteers. --apb (Alan Barrett)
Alan Barrett wrote:
Zefram has written a prototype of this code. My problem with that statement is I am not convinced that a re-factoring of the data wouldn't be helpful in both tidying up the process of adding new data moving forwards, and incorporating extra history.
The current concentration on 'timezone' is to my mind ( now that I've been through the data ) the wrong way of working. We have 'rules' that apply over periods of time, and locations switch between those rules. NAMES are about identifying which set of rules to use, and if those rules were re-factored to add the arbitrary 1970-01-01 transition, would filtering be any easier? We just need to identify a unique set of rules for a period, and then link locations to those rules. Simply number the rules ( in database terms an autoinc identifier for the id ) and then many countries can use the same rule, or set of rules, and adding 'hot potatoes' is separated from managing the raw data? As a practical example. Jersey Started using UK-GMT/BST time on a date, switched to 'German Time', switched back to UK-GMT/BST time and then to EU defined times. The bulk of the detail is in the rules which are shared, and the small amount of detail relates to the when rules changed. Once we can package rules in a way that can be filtered for valid use dates, filtering is easy, and then timezones available are those using the valid rules ... -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
I agree that changing the format of the zic input would make it considerably easier to maintain data, particularly the mass of pre-1970 data you're thinking about. But someone would need to come up with what the new input format actually is, and implement that change. I expect that we would use the new zic-input format only in the new "extended" zones that would be excluded from typical POSIX installations; this should lessen backward compatibility concerns.
Paul Eggert wrote:
I agree that changing the format of the zic input would make it considerably easier to maintain data, particularly the mass of pre-1970 data you're thinking about. But someone would need to come up with what the new input format actually is, and implement that change.
I expect that we would use the new zic-input format only in the new "extended" zones that would be excluded from typical POSIX installations; this should lessen backward compatibility concerns.
I'll put my hand up and admit to not having ever used zic and only looked at the manual today. I'm using the PHP tools for looking at 'published' data, and that is the calendar that I am expecting to scoll back through the 20th century and get correct times. Adding the Jersey and Guernsey data will ensure that for the war period is going to be correct in future. I don't think the proposals I'm making break zic. The new 'Rule1' for LMT only requires an additional piece of information from outside TZ and handling the relevant date could be moved there. An 'extended' with all of the 'timezones' you are worried about - just locations with links to existing rules mainly. Personally I don't think that there will be an unmanageable increase in 'rules' if we managed to find every documented change pre-1970. It will be things like using or not daylight saving, but the extra 'extended' data could be levered to add out of sync data. I hate to say this, but keep zic for creating 'TZ' and develop a new tool using the 'extended' data to give a 'TZ+' view. I've got to the point on my own setup that I'm looking to dump the raw data into a database and create code to work with that via SQL calls. A lot more flexible, and I already have a comprehensive set of location data in that, which links to the OSM mapping. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
On Thu, 5 Sep 2013, Lester Caine wrote:
As a practical example. Jersey Started using UK-GMT/BST time on a date, switched to 'German Time', switched back to UK-GMT/BST time and then to EU defined times. The bulk of the detail is in the rules which are shared, and
That's a simplification. Although the Channel Islands orders I found in the National Archives at Kew are far from providing complete information for 1916-1922, they do show as detailed on my webpage that Jersey used different transition times from the UK in 1916 (and Geraint Jennings found reference to this in the debate in the States). I transcribed the 1916 Channel Islands orders at <http://mm.icann.org/pipermail/tz/1999-September/010658.html>. -- Joseph S. Myers jsm@polyomino.org.uk
On 04/09/13 23:15, Lester Caine wrote:
Paul Eggert wrote:
On 09/04/13 13:43, Lester Caine wrote:
Is that good enough to merge? I'm afraid we can't add zones that differ only in pre-1970 dates until we can figure out how to filter them out in the default case, as we aren't prepared for the hundreds or thousands of new zones that would be feasible if we opened the floodgates. Zefram has one prototype for filtering, which I really must get to at some point.
Paul - this is the point of contention !
Deleting any data is wrong. If users have a reason for only viewing part of the data then fair enough, but we have the opportunity here to get this right before it becomes embedded in every browser. That does not want a broken set of timezones with this arbitrary cutoff - THAT is the very thing some of us want to get away from. I don't know why a browser would want to embed its own copy of tz, but this seems the wrong example. Actually, I think browsers should not expose the full tz rules, as that would allow to pinpoint the user quite accurately by probing old timezones that aren't actually needed by web apps.
Ángel González wrote:
Deleting any data is wrong. If users have a reason for only viewing part of the data then fair enough, but we have the opportunity here to get this right before it becomes embedded in every browser. That does not want a broken set of timezones with this arbitrary cutoff - THAT is the very thing some of us want to get away from. I don't know why a browser would want to embed its own copy of tz, but this seems the wrong example. Actually, I think browsers should not expose the full tz rules, as that would allow to pinpoint the user quite accurately by probing old timezones that aren't actually needed by web apps.
If one is looking at ONLY displaying current dates, then that is not a problem, but why have one display process for current dates and a different one for historic dates? My point is that if you ask for historic dates is should be handled the same as a current one. Time didn't start in 1970 ... even if some computers still think it did. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
You are probably looking at about 2000 zones in all if you want the same level of detail that the ACS Atlas has. On Wed, Sep 4, 2013 at 4:53 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
On 09/04/13 13:43, Lester Caine wrote:
Is that good enough to merge?
I'm afraid we can't add zones that differ only in pre-1970 dates until we can figure out how to filter them out in the default case, as we aren't prepared for the hundreds or thousands of new zones that would be feasible if we opened the floodgates. Zefram has one prototype for filtering, which I really must get to at some point.
On Wed, 4 Sep 2013, Lester Caine wrote:
We have dates of legal adoption of GMT in Jersey (around 4pm on Saturday 11 June 1898) and Guernsey (18 June 1913). Can I take those dates as facts we can add to the tz data? Is there any indication of the islands using a different time prior or like the main land they were already using 'GMT' and this just formalised it?
I don't have information about what time was being used before the legal adoption of GMT. The petition (March 1909) signed by ten members of the States of Deliberation of Guernsey called for adoption of GMT as well as for summer time (only the first part ended up entered into law), and says "As to the first, that the island time should be standardised, we apprehend that there will be little, if any, difference of opinion. Indeed, we think it likely that many will hear with surprise that there is at present no official standard time in Guernsey, and that at any moment questions might arise which would be difficult to settle in the present state of confusion.". -- Joseph S. Myers jsm@polyomino.org.uk
Joseph S. Myers wrote:
On Wed, 4 Sep 2013, Lester Caine wrote:
We have dates of legal adoption of GMT in Jersey (around 4pm on Saturday 11 June 1898) and Guernsey (18 June 1913). Can I take those dates as facts we can add to the tz data? Is there any indication of the islands using a different time prior or like the main land they were already using 'GMT' and this just formalised it?
I don't have information about what time was being used before the legal adoption of GMT. The petition (March 1909) signed by ten members of the States of Deliberation of Guernsey called for adoption of GMT as well as for summer time (only the first part ended up entered into law), and says "As to the first, that the island time should be standardised, we apprehend that there will be little, if any, difference of opinion. Indeed, we think it likely that many will hear with surprise that there is at present no official standard time in Guernsey, and that at any moment questions might arise which would be difficult to settle in the present state of confusion.".
With the Channel Islands being closer to the meridian the differences would probably just be interpreted as the poor setting of clocks? I've not found anything yet relating to a standard on the Isle of Man prior to GMT but I think it's worth going back through the records as they are very comprehensive. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Joseph S. Myers wrote:
On Wed, 4 Sep 2013, Lester Caine wrote:
Fills the picture. Europe/Isle_of_Man exists in it's own right up to 1883, and then it follows Europe/London.
Well - subject to confirming it did so during WW2, as you drew my attention to that issue. The (UK) Emergency Powers (Defence) Act, 1939 includes provision (subsection 4(1)(a)) for it to be extended to the Isle of Man by Order in Council, but I don't have a specific reference to such an Order in Council (presumably a UK SR&O) doing so. (All the subsequent regulations then worked by modification of an Act also applying to the Isle of Man, so if they had the power to apply there then they would have done so.)
I'm fairly sure from the IOM documents that Jersey and Guernsey followed the same date, but their archives have not got that far back yet. So do we have to duplicate the whole zone record each timezone to add this one item of data?
We have dates of legal adoption of GMT in Jersey (around 4pm on Saturday 11 June 1898) and Guernsey (18 June 1913). What we don't have is full details from WW1 or WW2, beyond the quote about time being put forward under occupation in WW2 (https://en.wikipedia.org/wiki/File:Orders_of_the_German_Commandant_Evening_P... gives one transition time as 11pm on 2 July 1940 in Jersey and we do have reasonably detailed information on German time from the PTB). Certainly if the cut-off is 1945 or before, you need different data for them than for Europe/London and we don't have the full information (whereas after 1945, the links to Europe/London are probably correct).
A first pass at adding this data is logged on my repo - http://lsces.org.uk/hg/tz/ I'm not sure that I have included the war time period correctly? This is the part I'm still getting my head around. Also the way locations used for calculating LMT are added may need to change? In my mind an additional table ( I'm normally working all this in databases ) would provide a list of locations and dates when the transition was made to a standard time, along with a link to the appropriate rule set. A time zone is simply a list of the appropriate rule sets + the locations linked to them. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
participants (10)
-
Alan Barrett -
Guy Harris -
Joseph S. Myers -
Lester Caine -
Paul Eggert -
Russ Allbery -
Stephen Colebourne -
Zefram -
Zoidsoft -
Ángel González