Question about bug seen in OpenBSD and FreeBSD related to tzname
Hello! Earlier this month, I noticed bugs in OpenBSD and FreeBSD related to tzname's values. https://marc.info/?l=openbsd-bugs&m=157379849727998&w=2 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241978 I noticed that both those OSes share relevant code with tzcode. How would I best test this stuff out with a recent release of tzcode? (i.e. how to make sure I'm including headers only from the 2019c tarball and not from my OS, when possible)
On 11/20/19 12:17 AM, Andras Farkas wrote:
Hello! Earlier this month, I noticed bugs in OpenBSD and FreeBSD related to tzname's values. https://marc.info/?l=openbsd-bugs&m=157379849727998&w=2 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241978 I noticed that both those OSes share relevant code with tzcode. How would I best test this stuff out with a recent release of tzcode? (i.e. how to make sure I'm including headers only from the 2019c tarball and not from my OS, when possible)
The short answer is to run 'make date' in the tzdb distribution, to see how tzdb creates the 'date' program with its own code instead of the operating system's. The longer answer is that tzcode doesn't provide its own <time.h> headers; it provides only alternative implementations, which you need to link to. tzcode does provide a tzfile.h header, which you can include by using the -I option (but you probably don't want to be including it).
On Wed, Nov 20, 2019 at 2:49 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
The short answer is to run 'make date' in the tzdb distribution, to see how tzdb creates the 'date' program with its own code instead of the operating system's.
The longer answer is that tzcode doesn't provide its own <time.h> headers; it provides only alternative implementations, which you need to link to. tzcode does provide a tzfile.h header, which you can include by using the -I option (but you probably don't want to be including it).
I see. Thanks for this info! On Wed, Nov 20, 2019 at 5:08 PM Guy Harris <guy@alum.mit.edu> wrote:
Note that macOS 10.14 does:
$ ./a.out tzname[0]: CAT tzname[1]:
for TZ=CAT0 and does
$ ./a.out tzname[0]: CAT tzname[1]: DOG
for TZ=CAT0DOG.
I think the first of those two is at least as valid, if not more valid, than the
tzname[0]: CAT tzname[1]: CAT
claimed for TZ=CAT0 on Linux - the POSIX spec says that tzname[1] should be "dst", but "dst" isn't present.
I agree with you here. In behavior here, I'd rank: macOS > GNU > both tested BSDs
On Nov 20, 2019, at 12:17 AM, Andras Farkas <deepbluemistake@gmail.com> wrote:
Hello! Earlier this month, I noticed bugs in OpenBSD and FreeBSD related to tzname's values. https://marc.info/?l=openbsd-bugs&m=157379849727998&w=2 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241978
Note that macOS 10.14 does: $ ./a.out tzname[0]: CAT tzname[1]: for TZ=CAT0 and does $ ./a.out tzname[0]: CAT tzname[1]: DOG for TZ=CAT0DOG. I think the first of those two is at least as valid, if not more valid, than the tzname[0]: CAT tzname[1]: CAT claimed for TZ=CAT0 on Linux - the POSIX spec says that tzname[1] should be "dst", but "dst" isn't present. I think GNU libc, which most Linux distributions use, has its own independent implementation of time routines and tzdb reading, so it's probably not surprising that it doesn't behave the same way that macOS, which uses tzcode-derived code, does.
On 11/20/19 2:08 PM, Guy Harris wrote:
I think the first of those two is at least as valid, if not more valid, than the
tzname[0]: CAT tzname[1]: CAT
claimed for TZ=CAT0 on Linux - the POSIX spec says that tzname[1] should be "dst", but "dst" isn't present.
I doubt whether it's a POSIX-conformance issue, as POSIX doesn't say what should be in tzname[1] when "dst" is absent. (Perhaps POSIX should explicitly say that tzname[1]'s value is undefined in this case, though I wish POSIX would deprecate tzname entirely and so am not inclined to pursue this.) I can see arguments either way, for what should be in tzname[1] when TZ does not have a "dst" element. For developers, the tzcode/macOS behavior (where tzname[1] is " ") might be better because it provides an obvious clue that the program is buggy when it outputs three spaces instead of a time zone abbreviation. For end users, the glibc behavior (where tzname[1] equals tzname[0]) might be better because if a buggy program uses tzname[1] in an environment where "CAT" is always the right time zone abbreviation, the buggy program will output "CAT" which happens (luckily) to be the correct answer that the user wants.
I've tested things, using tzdb-2019c.tar.lz and rewriting the main() function of date.c in it to be the same as the daytime.c test program I wrote before. The output, both when compiling this on OpenBSD, and on FreeBSD, is the same as what Guy Harris reported for macOS 10.14: tzname[0]: CAT tzname[1]: (three spaces, for tzname[1]'s value) This behavior seems good enough to me, and thus shows me that the bug is not in upstream (here) but in OpenBSD and FreeBSD. Thanks for helping me out, guys! :D Unrelatedly, and interestingly enough, I couldn't get things to compile while using -DUSG_COMPAT=1 in CFLAGS, on FreeBSD. However, I was able to get things to compile like that on OpenBSD. (Notably, FreeBSD doesn't support daylight or timezone, itself, while OpenBSD does. This seems to indicate USG_COMPAT'd tzcode can only be compiled on OSes already supporting those features)
On 11/20/19 11:19 PM, Andras Farkas wrote:
I couldn't get things to compile while using -DUSG_COMPAT=1 in CFLAGS, on FreeBSD. However, I was able to get things to compile like that on OpenBSD. (Notably, FreeBSD doesn't support daylight or timezone, itself, while OpenBSD does. This seems to indicate USG_COMPAT'd tzcode can only be compiled on OSes already supporting those features)
What happens if you compile with -DHAVE_POSIX_DECLS=0? Arguably it should be set that way if FreeBSD does not declare the POSIX-required 'daylight' and 'timezone'. I see now that POSIX marks 'daylight' and 'timezone' with "XSI", meaning it's optional and present only if the X/Open Systems Interfaces option is supported, so perhaps tzcode should have a different macro for identifiers that are XSI rather than core POSIX.
On November 22, 2019 9:30:52 PM EST, Paul Eggert <eggert@cs.ucla.edu> wrote:
I see now that POSIX marks 'daylight' and 'timezone' with "XSI", meaning it's optional and present only if the X/Open Systems Interfaces option is supported, so perhaps tzcode should have a different macro for identifiers that are XSI rather than core POSIX.
That is in fact the position FreeBSD has long taken. We implement the historic BSD timezone(3) function. -GAWollman
On Fri, Nov 22, 2019 at 10:48 PM Garrett Wollman <wollman@csail.mit.edu> wrote:
That is in fact the position FreeBSD has long taken. We implement the historic BSD timezone(3) function. Yep! timezone(3) is not only historic to the BSDs, but also existed in Unix v7: http://web.cuzuco.com/~cuzuco/v7/v7vol1.pdf (page 271 in the PDF) http://man.cat-v.org/unix_7th/3/ctime
On Fri, Nov 22, 2019 at 9:30 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
What happens if you compile with -DHAVE_POSIX_DECLS=0? Arguably it should be set that way if FreeBSD does not declare the POSIX-required 'daylight' and 'timezone'. With CFLAGS set to -DHAVE_POSIX_DECLS=0 alone, things compile and date runs A-OK. With CFLAGS set to '-DHAVE_POSIX_DECLS=0 -DUSG_COMPAT=1' the compilation fails, albeit in a different way from with -DUSG_COMPAT=1 alone. Should I send both sorts of failures as text files? I see now that POSIX marks 'daylight' and 'timezone' with "XSI", meaning it's optional and present only if the X/Open Systems Interfaces option is supported, so perhaps tzcode should have a different macro for identifiers that are XSI rather than core POSIX. Correct. The standard is separated into core POSIX, and POSIX+XSI (known combined as SUS, UNIX, X/Open, and a bunch of other names) The read-only (or intended to be read-only in your C program, and set by the implementation, i.e. the OS) C macros are: _POSIX_VERSION _POSIX2_VERSION _XOPEN_VERSION The macros intended to be set within a C program's source are: _POSIX_C_SOURCE _XOPEN_SOURCE In all these cases, if a XOPEN macro is set to something, you can already assume a POSIX value too, since POSIX is a subset of XOPEN, and XOPEN a superset of POSIX. Hence, daylight and timezone might be considered XSI-required rather than POSIX-required, when using those words specifically. A quick grep shows tzcode does already uses _POSIX_VERSION and _XOPEN_VERSION (I wonder if USG_COMPAT could be replaced entirely by POSIX and XOPEN macros, but I'll have to look more deeply into that tonight before saying what I think about that)
As a historical note, I think more XSI extensions to POSIX are SystemV-biased than BSD-biased, with a lot of them coming from the SVID. (though there are a couple BSD-biased XSI extensions, too) That's just a note and not a judgment or anything. uwu
I noticed something in the NEWS file and in the code after sending this email. I see that in tzcode, along with USG_COMPAT for the XSI-required (optional in POSIX) daylight and timezone, there is HAVE_TZNAME for tzname, which is required by POSIX (and hence also in POSIX+XSI) Thus, the macros already exist for this separation of standards and optional/required stuff. On Sat, Nov 23, 2019 at 1:27 AM Andras Farkas <deepbluemistake@gmail.com> wrote:
On Fri, Nov 22, 2019 at 9:30 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
What happens if you compile with -DHAVE_POSIX_DECLS=0? Arguably it should be set that way if FreeBSD does not declare the POSIX-required 'daylight' and 'timezone'. With CFLAGS set to -DHAVE_POSIX_DECLS=0 alone, things compile and date runs A-OK. With CFLAGS set to '-DHAVE_POSIX_DECLS=0 -DUSG_COMPAT=1' the compilation fails, albeit in a different way from with -DUSG_COMPAT=1 alone. Should I send both sorts of failures as text files? I see now that POSIX marks 'daylight' and 'timezone' with "XSI", meaning it's optional and present only if the X/Open Systems Interfaces option is supported, so perhaps tzcode should have a different macro for identifiers that are XSI rather than core POSIX. Correct. The standard is separated into core POSIX, and POSIX+XSI (known combined as SUS, UNIX, X/Open, and a bunch of other names) The read-only (or intended to be read-only in your C program, and set by the implementation, i.e. the OS) C macros are: _POSIX_VERSION _POSIX2_VERSION _XOPEN_VERSION The macros intended to be set within a C program's source are: _POSIX_C_SOURCE _XOPEN_SOURCE In all these cases, if a XOPEN macro is set to something, you can already assume a POSIX value too, since POSIX is a subset of XOPEN, and XOPEN a superset of POSIX. Hence, daylight and timezone might be considered XSI-required rather than POSIX-required, when using those words specifically. A quick grep shows tzcode does already uses _POSIX_VERSION and _XOPEN_VERSION (I wonder if USG_COMPAT could be replaced entirely by POSIX and XOPEN macros, but I'll have to look more deeply into that tonight before saying what I think about that)
As a historical note, I think more XSI extensions to POSIX are SystemV-biased than BSD-biased, with a lot of them coming from the SVID. (though there are a couple BSD-biased XSI extensions, too) That's just a note and not a judgment or anything. uwu
There's also HAVE_TZSET (my bad for sending three emails in a row like this!) On Sat, Nov 23, 2019 at 1:37 AM Andras Farkas <deepbluemistake@gmail.com> wrote:
I noticed something in the NEWS file and in the code after sending this email. I see that in tzcode, along with USG_COMPAT for the XSI-required (optional in POSIX) daylight and timezone, there is HAVE_TZNAME for tzname, which is required by POSIX (and hence also in POSIX+XSI) Thus, the macros already exist for this separation of standards and optional/required stuff.
On Sat, Nov 23, 2019 at 1:27 AM Andras Farkas <deepbluemistake@gmail.com> wrote:
On Fri, Nov 22, 2019 at 9:30 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
What happens if you compile with -DHAVE_POSIX_DECLS=0? Arguably it should be set that way if FreeBSD does not declare the POSIX-required 'daylight' and 'timezone'. With CFLAGS set to -DHAVE_POSIX_DECLS=0 alone, things compile and date runs A-OK. With CFLAGS set to '-DHAVE_POSIX_DECLS=0 -DUSG_COMPAT=1' the compilation fails, albeit in a different way from with -DUSG_COMPAT=1 alone. Should I send both sorts of failures as text files? I see now that POSIX marks 'daylight' and 'timezone' with "XSI", meaning it's optional and present only if the X/Open Systems Interfaces option is supported, so perhaps tzcode should have a different macro for identifiers that are XSI rather than core POSIX. Correct. The standard is separated into core POSIX, and POSIX+XSI (known combined as SUS, UNIX, X/Open, and a bunch of other names) The read-only (or intended to be read-only in your C program, and set by the implementation, i.e. the OS) C macros are: _POSIX_VERSION _POSIX2_VERSION _XOPEN_VERSION The macros intended to be set within a C program's source are: _POSIX_C_SOURCE _XOPEN_SOURCE In all these cases, if a XOPEN macro is set to something, you can already assume a POSIX value too, since POSIX is a subset of XOPEN, and XOPEN a superset of POSIX. Hence, daylight and timezone might be considered XSI-required rather than POSIX-required, when using those words specifically. A quick grep shows tzcode does already uses _POSIX_VERSION and _XOPEN_VERSION (I wonder if USG_COMPAT could be replaced entirely by POSIX and XOPEN macros, but I'll have to look more deeply into that tonight before saying what I think about that)
As a historical note, I think more XSI extensions to POSIX are SystemV-biased than BSD-biased, with a lot of them coming from the SVID. (though there are a couple BSD-biased XSI extensions, too) That's just a note and not a judgment or anything. uwu
On Sat, Nov 23, 2019 at 1:27 AM Andras Farkas <deepbluemistake@gmail.com> wrote:
The read-only (or intended to be read-only in your C program, and set by the implementation, i.e. the OS) C macros are: _POSIX_VERSION _POSIX2_VERSION _XOPEN_VERSION The macros intended to be set within a C program's source are: _POSIX_C_SOURCE _XOPEN_SOURCE I'm gonna correct myself. When I said "your C program" and "a C program's source" I meant an application/util/etc. tzcode probably shouldn't be setting any of those 5 macros (because it IS the implementation, yet it's also not the OS) but can absolutely read all 5 of those in useful ways.
I'll probably start a new thread/subjectline for the topics of compiling with USG_COMPAT on FreeBSD, and tzcode preprocessor macros, soon, rather than continue to use this thread. Just to categorise things better.
Date: Sat, 23 Nov 2019 02:52:43 -0500 From: Andras Farkas <deepbluemistake@gmail.com> Message-ID: <CAA0nTRsAVKLycy=kxt3BMHDv9zmSUEZTMdAbRfO3P+V22aS3vA@mail.gmail.com> | (because it IS the implementation, yet it's also not the OS) That distinction, which is made by many standards in various fields is totally worthless - whether any particular piece of code (or hardware or whatever) is implementation or application (or however the relevant standard attempts to divide the world into two groups) depends entirely upon where you are looking at it from. tzcode as a separate software chunk is defiitely application when viewed from the system provider's point of view, but typically becomes implementation (and is viewed as that) when integrated into libc and viewed from a normal user's point of view. But even there, from the compiler writer's point of view, the compiler (and base system it runs on) is the implementation, and everything that the compiler compiles is application - including libc. And where you'd think that code that calls tzset()/localtime()/... must be application, you find a system provided library (like say syslog(3)) which does that, and is really implementation. The whole distinction is nonsense. kre ps: you might thing that the chips (processor, memory, etc) are definitely part of the implementation, but from the fabricator's point of view, they're implementations making use of the basic functionality provided by the underlying silicon...
On 2019-11-23 03:20, Robert Elz wrote:
Date: Sat, 23 Nov 2019 02:52:43 -0500 From: Andras Farkas <deepbluemistake@gmail.com> Message-ID: <CAA0nTRsAVKLycy=kxt3BMHDv9zmSUEZTMdAbRfO3P+V22aS3vA@mail.gmail.com>
| (because it IS the implementation, yet it's also not the OS)
That distinction, which is made by many standards in various fields is totally worthless - whether any particular piece of code (or hardware or whatever) is implementation or application (or however the relevant standard attempts to divide the world into two groups) depends entirely upon where you are looking at it from.
Programmers must clearly distinguish between standard or documented system independent feature test macros intended to be set by the programmer in feature_test_macros(7) and the documentation for the system headers, library interfaces, and standards e.g #define _POSIX_C_SOURCE 200809L #define _XOPEN_SOURCE 700 #define _DEFAULT_SOURCE 1 #define _GNU_SOURCE 1 Those feature test macros enable certain features of implementations in conformance with standards or documentation. Those standard or documented system independent feature test macros should be distinguished from implementation specific feature macros often set in /usr/include/features.h, /usr/include/sys/features.h, and their inclusions, included from feature test dependent library headers, which may set feature test macros, and implementation specific feature macros. Those implementation specific feature macros are tested in library headers in lieu of various feature test macros e.g. Linux __USE_POSIX, __USE_XOPEN, and should never be used or set by the programmer as they vary between implementations and may be changed by any implementation release, as they are non-standard and undocumented. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised.
Brian Inglis wrote in <f5634d94-4b76-5852-316d-a634d7191ae8@SystematicSw\ .ab.ca>: |On 2019-11-23 03:20, Robert Elz wrote: |> Date: Sat, 23 Nov 2019 02:52:43 -0500 |> From: Andras Farkas <deepbluemistake@gmail.com> |> Message-ID: <CAA0nTRsAVKLycy=kxt3BMHDv9zmSUEZTMdAbRfO3P+V22aS3vA@mai\ |> l.gmail.com> |> |>| (because it IS the implementation, yet it's also not the OS) |> |> That distinction, which is made by many standards in various fields |> is totally worthless - whether any particular piece of code (or |> hardware or whatever) is implementation or application (or however |> the relevant standard attempts to divide the world into two groups) |> depends entirely upon where you are looking at it from. | |Programmers must clearly distinguish between standard or documented system |independent feature test macros intended to be set by the programmer in |feature_test_macros(7) and the documentation for the system headers, \ |library |interfaces, and standards e.g | #define _POSIX_C_SOURCE 200809L | #define _XOPEN_SOURCE 700 | #define _DEFAULT_SOURCE 1 | #define _GNU_SOURCE 1 |Those feature test macros enable certain features of implementations in |conformance with standards or documentation. Since you refer to feature_test_macros(7) i want to add that i find headers of especially elder systems notoriously buggy regarding the above. One should especially not trust statements like "defining x produces the same effects as defining y". I am in the lucky position to be able to test on several operating systems, and thanks to OpenCSW.org this also includes SunOS 5.9 - 5.11. My findings are that you are best served with a sledgehammer by defining all of the above (i do except _DEFAULT_SOURCE), and i also still have a _DARWIN_C_SOURCE lingering even though the last i tested was Mountain Lion. (But it still works like this out of the box as of today via Homebrew and MacPorts.) |Those standard or documented system independent feature test macros \ |should be |distinguished from implementation specific feature macros often set in |/usr/include/features.h, /usr/include/sys/features.h, and their inclusions, |included from feature test dependent library headers, which may set \ |feature test |macros, and implementation specific feature macros. |Those implementation specific feature macros are tested in library \ |headers in |lieu of various feature test macros e.g. Linux __USE_POSIX, __USE_XOPEN, \ |and |should never be used or set by the programmer as they vary between |implementations and may be changed by any implementation release, as \ |they are |non-standard and undocumented. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
participants (7)
-
Andras Farkas -
Brian Inglis -
Garrett Wollman -
Guy Harris -
Paul Eggert -
Robert Elz -
Steffen Nurpmeso