On 1/6/2024 7:06 PM, Guy Harris wrote:
On Jan 6, 2024, at 3:35 PM, Paul Eggert<eggert@cs.ucla.edu> wrote:
On 2024-01-06 14:51, Guy Harris via tz wrote:
POSIX's time API has no notion of "gmtoff" and "stdoff". Although that's true for current POSIX, POSIX 202x/D3 does have tm_gmtoff in struct tm, as a result of Austin Group Defect 1533<https://austingroupbugs.net/view.php?id=1533> which saw steffen and kre as contributors. Yes, I know, I signed up for the list so that I could read the draft.
But there's no tm_stdoff, so it wasn't obvious that "gmtoff" and "stdoff" were referring to values in struct tm, and Brooks Harris said
I just meant the values of gmtoff and stdoff are adjusted to satisfy Posix-time rather than reflect the values in the source files. but tm_gmtoff is currently obviously not "adjusted to satisfy Posix-time" as, when tm_gmtoff was introduced, there was no tm_gmtoff in POSIX or, for that matter, in most UN*Xes (was it in *any* UN*Xes before tzcode put it in?), and it's also not "adjusted", it's just the current offset as calculated from the values in the source files.
And, as there's no "tm_stdoff" value even now, there's nothing *to* adjust about stdoff. Right, but I think there should be. Posix cannot distinguish an stdoff shift independent from a gmtoff shift.
For example, there is a STDOFF shift at 1971 Oct 31 2:00u from 1:00 to 0:00 (a West shift) in London: # Zone NAME STDOFF RULES FORMAT [UNTIL] Zone Europe/London -0:01:15 - LMT 1847 Dec 1 0:00s 0:00 GB-Eire %s 1968 Oct 27 1:00 - BST 1971 Oct 31 2:00u 0:00 GB-Eire %s 1996 0:00 EU GMT/BST Zic and TzIf reflect this change as a shift in gmtoff, not stdoff: 57722399 1971-10-31 02:59:59 isdst 0 gmtoff 3600 stdoff 0 BST 57722400 1971-10-31 02:00:00 isdst 0 gmtoff 0 stdoff 0 GMT That's what I mean by "adjusted" for Posix sake. It gives the proper UTC offset, yes, but not for the right reason. The underlying reason was an STDOFF shift, presumably stated in the law behind it. TzDb source has been careful to try to honor the laws and customs behind the local time rules, and I strongly support that policy. I would presume this custom originates with the USA laws that specifically define the time zones as offsets from UTC and the optional one-hour DST shifts. These are the familiar behaviors of time zones where DST is "observed". But many time zones do not follow these familiar patterns and we have to be careful about that. Many zones have shifted their STDOFF, not always by an hour, and made multiple DST shifts (co-called "double summertime") and not always by an hour, and sometimes simultaneously. This can get tricky. --------------------- I'm involved with several standards projects at the Society of Motion Picture and Television Engineers (SMPTE). The television and broadcast industries have used "SMPTE Timecode" (defined primarily by a standard named ST12-1) for decades, since the mid 1970s. SMPTE Timecode is used in very many ways, from labeling of video frames to synchronization and essentially glues the timekeeping of the whole industry together. SMPTE Timecode was derived from IRIG and is quite similar to the formats used by WWV radio and others with addition of video-related metadata. Timecode is used all over the world, where various video rates are used. For example most of Europe uses 1/25 (25 frames-per-second). These systems have an exact relation to running time in seconds and are sometimes called "integral rate". But the USA, Canada, Japan and others use the 'strange' rate off 30000/1001, called a "non-integral rate". This has all sorts of algorithmic ramifications and implementations must be careful with their math. But this is well-know in the industry. Meantime there is the matter of audio synchronization of various frequencies with video. A typical audio frequency is 48000/1. If you're interested in more detail about video and audio rates and SMPTE Timecode you might read: Conversion of Audio Samples to Video Frames Brooks Harris, May 8, 1998 http://edlmax.com/AudioNTSC.htm Conversion between SMPTE hh:mm:ss:ff Time Code and Frames Brooks Harris, EdlMax, LLC. Version V4 2015-04-04 http://edlmax.com/SMPTETimeCodeConversion.htm In broadcast television it's important to have the hh:mm:ss portion of the timecode run as close as possible to local time. But SMPTE timecode and equipment that depends on it cannot tolerate a discontinuity in the hh:mm:ss:video-frame sequence. To work around this limitation the industry has typically adopted a procedure called "Daily Jam". Here, DST shifts, STDOFF shifts and leap-second shifts are deferred to some local time and the hh:mm:ss:video-frame values are reset ("jammed") to match the local wall-clock time. This is an imperfect solution which can produce a "glitch" in the hh:mm:ss:video-frame sequence, especially for systems running at "non-integer" rates. Therefore the "Daily Jam" is typically instituted at some local time least likely to disrupt normal operations. In the USA this time is typically 02:00:00 (2am). Daily Jam is imperfect but an effective work-around that has been used for decades. In recent years SMPTE has developed several standards related to packet-based network "streaming" of video and audio. They've elected to use IEEE Precision Time Protocol as the primary synchronization reference. The standards called ST2059-1 and ST2059-2 in particular define the relation of timecode to PTP time. These also codify the use of Daily Jam procedures. User requirements call for use of local time together with video frame and audio-sample accuracy. And it is this set of challenges that have brought me (kicking and screaming :-) ) into the wider world of timekeeping and so to Tz Database. In 2019 I was invited to make a presentation to ION/PTTI: Accurate Local Timestamps, Brooks Harris https://www.ion.org/publications/abstract.cfm?articleID=16763 In this I pointed out that typical timestamps were insufficient to represent truly unambiguous local time. If we have two ISO 8601 timestamps with date, time and UTC offset: 2023-07-01 00:01:23 -7:00 2023-07-01 00:04:56 -7:00 We can normalize to UTC and determine that the first precedes the second; precedence is established. Great! This, of course, is the first of most important aspect of timekeeping. This was the whole point to Posix-time to begin with; maintain event precedence within the system. But this entirely misses the fact that the two timestamps may have come from two different time zones. If you add the time zone it becomes much clearer what the meaning of the two events are: 2023-07-01 00:01:23 -7:00 America/Los_Angeles 2023-07-01 00:04:56 -7:00 America/Phoenix But this does not signal that DST was in effect in America/Los_Angeles and not observed in America/Phoenix. You could add an "isdst" flag: 2023-07-01 00:01:23 -7:00 America/Los_Angeles DST 2023-07-01 00:04:56 -7:00 America/Phoenix STD That works (more-or-less) in these two typical and familiar cases. But when you get to other time zones that simple logic may not hold up. My favorite "kryptonite" example is Europe/Moscow: # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S Rule Russia 1917 only - Jul 1 23:00 1:00 MST # Moscow Summer Time Rule Russia 1917 only - Dec 28 0:00 0 MMT # Moscow Mean Time Rule Russia 1918 only - May 31 22:00 2:00 MDST # Moscow Double Summer Time Rule Russia 1918 only - Sep 16 1:00 1:00 MST Rule Russia 1919 only - May 31 23:00 2:00 MDST Rule Russia 1919 only - Jul 1 0:00u 1:00 MSD Rule Russia 1919 only - Aug 16 0:00 0 MSK Rule Russia 1921 only - Feb 14 23:00 1:00 MSD Rule Russia 1921 only - Mar 20 23:00 2:00 +05 Rule Russia 1921 only - Sep 1 0:00 1:00 MSD Rule Russia 1921 only - Oct 1 0:00 0 - Rule Russia 1981 1984 - Apr 1 0:00 1:00 S Rule Russia 1981 1983 - Oct 1 0:00 0 - Rule Russia 1984 1995 - Sep lastSun 2:00s 0 - Rule Russia 1985 2010 - Mar lastSun 2:00s 1:00 S Rule Russia 1996 2010 - Oct lastSun 2:00s 0 - # Zone NAME STDOFF RULES FORMAT [UNTIL] Zone Europe/Moscow 2:30:17 - LMT 1880 2:30:17 - MMT 1916 Jul 3 # Moscow Mean Time 2:31:19 Russia %s 1919 Jul 1 0:00u 3:00 Russia %s 1921 Oct 3:00 Russia MSK/MSD 1922 Oct 2:00 - EET 1930 Jun 21 3:00 Russia MSK/MSD 1991 Mar 31 2:00s 2:00 Russia EE%sT 1992 Jan 19 2:00s 3:00 Russia MSK/MSD 2011 Mar 27 2:00s 4:00 - MSK 2014 Oct 26 2:00s 3:00 - MSK Whoa man! Take the transition at 1991-03-31 02:00:00 3:00 Russia MSK/MSD 1991 Mar 31 2:00s 2:00 Russia EE%sT 1992 Jan 19 2:00s This is an STDOFF shift with a simultaneous offsetting DST shift. As Time And Date puts it: Mar 31 No change, Mar 31, 1991 - Daylight Saving Time Started DST started on Sunday, March 31, 1991, 2:00:00 am. However, clocks were not changed because Moscow switched time zones at the same time. https://www.timeanddate.com/time/change/russia/moscow?year=1991 This is a "no-op" transition (the YMDhms sequence is not interrupted) but several important facts changed; both STDOFF and DST shifted, and the Abbreviation changed. Zic also sets the isdst flag: 670373999 1991-03-31 01:59:59 isdst 0 gmtoff 10800 stdoff 10800 MSK 670374000 1991-03-31 02:00:00 isdst 1 gmtoff 10800 stdoff 10800 EEST There is another objective in my work. In video we have always been able to represent any time-point within the 24-hour range, important for editorial adjustments a duration calculation. To do this comprehensively with local time you need to know that this day includes a transition and what and when that transition occurs. Thus, my formats include information to signal "today is a transition day", "what the transition is", and "when it occurs". In this Moscow example both STDOFF and DST shift and these are independently indicated. D1991-03-30T23:59:59U+03Zeurope/moscowAmskV2021aL16MuX UTC 00670366815 D1991-03-31T00:00:00U+03w01+02Zeurope/moscowAmskV2021aL16S00t01a02cMuX UTC 00670366816 D1991-03-31T01:59:59U+03w01+02Zeurope/moscowAmskV2021aL16S00t01a02cMuX UTC 00670374015 D1991-03-31T02:00:00U+03w01+02Zeurope/moscowAeestV2021aL16S01t01a02cMuX UTC 00670374016 D1991-03-31T23:59:59U+03w01+02Zeurope/moscowAeestV2021aL16S01t01a02cMuX UTC 00670453215 D1991-04-01T00:00:00U+03Zeurope/moscowAeestV2021aL16S01cMuX UTC 00670453216 There are many details to the formats and implementation I'm working on but I hope this gives an idea why I believe it is useful to include the STDOFF data in any timestamp format and to include such transitions in the output of zic. I also hope Posix might consider adding "tm_stdoff" to struct tm. Thanks, -Brooks