The zoneinfo file system tree structure ?
[please cc me, I am not subscribed] Hello, I have been working on Util-linux' hwclock and other GNU/Linux date-time issues. I have a couple of questions that I hope someone here can answer. The tz database file system tree was changed on May 25 1998, moving 'right' and 'posix' from subdirectories of zoneinfo to sibling directories. However GNU libc continued to use the old structure until it stopped including the data files on Mar 7 2012. Now some Linux distributions are still using the old subdirectory tree structure. Is there a reason that the old tree structure is still being used? Does it break something to have 'right' and 'posix' as sibling directories? I ask because the problems that were the catalyst for the change in 1998 are still happening. It seems to me like it was a good solution, but nobody used it. If I may ask one more favor, could you please review the information below which I have added to the hwclock man-page? I still need to include information about using TZDIR. Thank you very much, William POSIX vs 'RIGHT' A discussion on date-time configuration would be incomplete without addressing timezones, this is mostly well covered by tzset(3). One area that seems to have no documentation is the 'right' directory of the IANA Time Zone Database, aka tz, aka zoneinfo. There are two separate databases in the zoneinfo system, posix and 'right'. 'Right' (now named leaps) includes leap seconds and posix does not. To use the 'right' database the System Clock must be kept in UTC + leap_seconds, i.e., TAI - 10. This allows calculating the exact number of seconds between two dates that cross a leap second epoch. The System Clock is then converted to the correct civil time, including UTC, by using the 'right' timezone files which subtract the leap sec- onds. Note: this configuration is considered experimental and is known to have issues. To configure a system to use a particular database all of the files located in its directory must be copied to the root of /usr/share/zoneinfo. Files are never used directly from the posix or 'right' subdirectories, e.g., TZ='right/Europe/Dublin'. This habit was becoming so common that the upstream zoneinfo project restructured the system's file tree by moving the posix and 'right' subdirectories out of the zoneinfo directory and into sibling directories: /usr/share/zoneinfo /usr/share/posix /usr/share/leaps Unfortunately, some Linux distributions are changing it back to the old tree structure in their packages. So the problem of system administra- tors reaching into the 'right' subdirectory persists. This causes the system timezone to be configured to include leap seconds while the zoneinfo database is still configured to exclude them. Then when an application such as a World Clock needs the South_Pole timezone file; or an email MTA, or hwclock needs the UTC timezone file; they fetch it from the root of /usr/share/zoneinfo , because that is what they are supposed to do. Those files exclude leap seconds, but the System Clock now includes them, causing an incorrect time conversion. Attempting to mix and match files from these separate databases will not work, because they each require the System Clock to use a different timescale. The zoneinfo database must be configured to use either posix or 'right', as described above.
J William Piggott wrote:
Is there a reason that the old tree structure is still being used?
Sure, inertia. Things move very slowly in the time business.
Does it break something to have 'right' and 'posix' as sibling directories?
Well, it does mean TZ='right/America/New_York' doesn't work any more, and one must use TZ='../zoneinfo-leaps/America/New_York' instead. So there is a compatibility issue there.
/usr/share/zoneinfo /usr/share/posix /usr/share/leaps
By default, the subdirectories are called 'zoneinfo' and 'zoneinfo-leaps', with 'zoneinfo-posix' being a symbolic link to 'zoneinfo'. Of course as you mention many distributions still do things the old way. E.g., Debian and Fedora both ship multiple copies of the 'posix' directory, which obviously takes far more disk space than the symbolic-link approach in the tz distribution. I don't think applications can rely on either directory structure per se. For example, Solaris puts the directory in a different place, under /usr/share/lib/zoneinfo, and does not ship a leap-second version at all.
<<On Fri, 13 Feb 2015 11:40:26 -0800, Paul Eggert <eggert@cs.ucla.edu> said:
I don't think applications can rely on either directory structure per se. For example, Solaris puts the directory in a different place, under /usr/share/lib/zoneinfo, and does not ship a leap-second version at all.
FreeBSD likewise. -GAWollman
[please cc me, I am not subscribed] On 02/13/2015 02:40 PM, Paul Eggert wrote:
E.g., Debian and Fedora both ship multiple copies of the 'posix' directory, which obviously takes far more disk space than the symbolic-link approach in the tz distribution.
As does Slackware; it also offers up every tzfile to its users during installation: root, posix, and right. Most users have no idea what the difference is. I believe that there is great deal of misunderstanding with regard to the tz database. That is what prompted me to add some information to the hwclock(8) man-page. Thank you for taking time to reply Professor Eggert. It is very much appreciated! William
On Thu, Feb 12, 2015, at 16:56, J William Piggott wrote:
Is there a reason that the old tree structure is still being used?
Does it break something to have 'right' and 'posix' as sibling directories?
Isn't it a bit obnoxious to create three directories in /usr/share? Also, to me this raises the question of why the files are installed at all, if they shouldn't be used without rearranging everything (you could just as easily rebuild from source). What distributions are doing this, and is there a possibility of an actual dialogue with their maintainers rather than just wondering what they're thinking? Do we know if other packages like ntp are actually able to handle the "right" situation correctly, even _with_ the appropriate UTC zone installed? Nevermind the fact that no-one actually uses time2posix/posix2time, and there's been no real effort to identify programs that need it (any tar archiver would be an example) and evangelize these functions to those projects. Maybe it's time to stop shipping these files entirely, and ask some serious questions about the future of leap second support. I think the core problem is that the use of a numeric time_t-like format for representing leap-aware timestamps is an attractive nuisance in a world where POSIX mandates a non-leap-aware version. If anything, I think the most likely conclusion is that these maintainers are likely trying to maintain backwards compatibility for people who do broken things like TZ=right/whatever not knowing any better.
On Fri, Feb 13, 2015 at 2:52 PM, <random832@fastmail.us> wrote:
Do we know if other packages like ntp are actually able to handle the "right" situation correctly, even _with_ the appropriate UTC zone installed? Nevermind the fact that no-one actually uses time2posix/posix2time, and there's been no real effort to identify programs that need it (any tar archiver would be an example) and evangelize these functions to those projects. Maybe it's time to stop shipping these files entirely, and ask some serious questions about the future of leap second support.
I think the core problem is that the use of a numeric time_t-like format for representing leap-aware timestamps is an attractive nuisance in a world where POSIX mandates a non-leap-aware version. If anything, I think the most likely conclusion is that these maintainers are likely trying to maintain backwards compatibility for people who do broken things like TZ=right/whatever not knowing any better.
Back in the late '80s, when this stuff was first written, I ran a system using a "right clock" and found that it required surprisingly few time2posix()/posix2time() calls to (appear to) work. One such call was certainly in NTP, and there were a few others in programs that attempted to do time_t arithmetic (like cron). Raw timestamps stored in file systems and sent in other network protocols did not appear to cause major issues, but this was before the days of widespread interconnection. I'm sure things are infinitely worse now. My intent was certainly to demonstrate an alternative to the POSIX definition, in which clocks did not need to be adjusted over leap seconds and time_t's where more opaque. With equal certainty I conclude that it didn't catch on.
On Thu 2015-02-12T16:56:55 -0500, J William Piggott hath writ:
There are two separate databases in the zoneinfo system, posix and 'right'. 'Right' (now named leaps) includes leap seconds and posix does not.
To use the 'right' database the System Clock must be kept in UTC + leap_seconds, i.e., TAI - 10.
I would clarify that to read To use the 'right' database the System Clock must count allseconds of UTC (and TAI), including leap seconds. From the beginning of 1972 onward this is equivalent to setting the System Clock to (TAI - 10 s).
This allows calculating the exact number of seconds between two dates that cross a leap second epoch.
I would clarify that to read This means that the difference between two dates that span a leap second counts the number of seconds of TAI (and this is not congruent with POSIX).
The System Clock is then converted to the correct civil time, including UTC, by using the 'right' timezone files which subtract the leap sec- onds. Note: this configuration is considered experimental and is known to have issues.
-- Steve Allen <sla@ucolick.org> WGS-84 (GPS) UCO/Lick Observatory--ISB Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m
On 2015-02-13 12:54, Steve Allen wrote:
On Thu 2015-02-12T16:56:55 -0500, J William Piggott hath writ:
There are two separate databases in the zoneinfo system, posix and 'right'. 'Right' (now named leaps) includes leap seconds and posix does not.
To use the 'right' database the System Clock must be kept in UTC + leap_seconds, i.e., TAI - 10.
I would clarify that to read
To use the 'right' database the System Clock must count allseconds of UTC (and TAI), including leap seconds. From the beginning of 1972 onward this is equivalent to setting the System Clock to (TAI - 10 s).
This allows calculating the exact number of seconds between two dates that cross a leap second epoch.
I would clarify that to read
This means that the difference between two dates that span a leap second counts the number of seconds of TAI (and this is not congruent with POSIX).
The System Clock is then converted to the correct civil time, including UTC, by using the 'right' timezone files which subtract the leap sec- onds. Note: this configuration is considered experimental and is known to have issues.
Do astronomers or physicists keep their system clocks on TAI and use these right/leaps files? -- Take care. Thanks, Brian Inglis
participants (7)
-
Bradley White -
Brian Inglis -
Garrett Wollman -
J William Piggott -
Paul Eggert -
random832＠fastmail.us -
Steve Allen