Paul Eggert via tz <tz@iana.org> writes:

I didn't know this this code was used in freebsd-src/contrib/tzcode. I now see that there are some minor differences between the FreeBSD copy; would it make sense to merge these into tzcode, perhaps with some "#ifdef __FreeBSD__"s thrown in? I suspect that would save time overall in the long run.

I wouldn't call these differences minor, the diff in localtime.c alone is 450+ lines and includes changes which you've rejected in the past. I've attached a cleaned-up diff. Almost everything in it is already #ifdef'ed. DES -- Dag-Erling Smørgrav - des@des.no

Paul Eggert via tz <tz@iana.org> writes:

On 2025-09-23 04:16, Dag-Erling Smørgrav wrote:

...

I wouldn't call these differences minor, the diff in localtime.c alone is 450+ lines and includes changes which you've rejected in the past. Sorry about that; to be honest I'd forgotten. Let's revisit it.

I don't see much point in upstreaming FreeBSD's changes en masse. The one change I'd like to see upstreamed (because it would reduce future merge conflicts) is the offtime_r patch which I submitted and you rejected two years ago. The DETECT_TZ_CHANGES patch may be of interest to others, but it should be attributed to NetApp, Inc. rather than to FreeBSD. DES -- Dag-Erling Smørgrav - des@des.no

On Sep 23, 2025, at 11:51 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:

On 2025-09-23 10:05, Dag-Erling Smørgrav wrote:

...

The DETECT_TZ_CHANGES patch may be of interest to others, but it should be attributed to NetApp, Inc. rather than to FreeBSD.

What form should the attribution take? I don't see attribution in the FreeBSD time source code. I assume Guy Harris wrote that

I don't remember doing that. The "Timezone change detection" message to the tz list said

Hello. I'm working on upstreaming a patch, which originally came from NetApp, to modify localtime.c to automatically detect changes to the time zone. The use case is to make the applications handle moving between timezones, or changing the default system timezone.

Moving between timezones is far less common for NetApp machines than for, say, laptops running Linux or some flavor of BSD (including CupertinoBSD, which runs on several popular laptops with names containing the string "MacBook"), as most NetApp servers are hard to carry onto any form of transport. :-) Given that, I'm not sure when that entered the code base. I don't *remember* it being added, but then I left NetApp in 2003/2004, so that's 20 years of either neuron death or repurposing. FreeBSD on notebooks would probably be a more common user of that capability, although they might want to take a look at CupertinoBSD's way of handling this, which involves using its notification mechanism, as documented in notify(3). That avoids having to wake up and manually check every minute. (I don't know whether updates to tz files provoke such a notification; they *should*, especially given that I think there's a mechanism to update those in the background, but I don't think they *do*.)

so I'll cc him.

(I'm on the tz list, so no need.)

On 2025-09-23 12:19, Dag-Erling Smørgrav wrote:

Paul Eggert via tz <tz@iana.org> writes:

...

(By the way, offtime_r is not documented in FreeBSD, so is it present only as a compatibility hack there?)

I held off on documenting it after you rejected the patch.

Might not hurt to keep holding off until we finish this chat....

FreeBSD's current behavior seems more useful to me than what the C standard mandates. I realize it's easier said than done, but I would prefer at least trying to get the standard changed instead.

Changed to what, though? FreeBSD gmtime and localtime return pointers to malloc'ed storage that might be freed before their callers use the pointers, leading to undefined behavior. (This cannot happen on platforms that conform to ISO C and POSIX.) If a change is proposed to ISO C and POSIX, this issue should be mentioned and taken into account in the wording. To see the issue I'm talking about, compile and run the attached stress test. Since it has undefined behavior on FreeBSD due to accessing freed memory, I compiled it on a CheriBSD (FreeBSD 15) platform with "cc -march=morello -mabi=purecap gmt3.c -lpthread". The stress test failed with "In-address space security exception (core dumped)"; gdb reports that the failure occurs in eqtm because its first pointer A is invalid. The stress test trivially succeeds on GNU/Linux, which conforms to ISO C and POSIX and which therefore does not attempt to free struct tm objects dynamically (indeed, eqtm's two arguments are always the same pointer). Regardless of whether ISO C and POSIX are changed, the current FreeBSD behavior (assuming it's still wanted) should be covered in its man pages so that FreeBSD's conflict with the current standards is documented.

An earlier version of this patch was posted here and rejected in 2021: https://mm.icann.org/pipermail/tz/2021-September/030335.html

Thanks, I'll take a further look at that.

Date: Tue, 23 Sep 2025 09:42:31 -0700 From: Paul Eggert via tz <tz@iana.org> Message-ID: <1a028a13-7d36-4233-8647-1f267b7d8047@cs.ucla.edu> | A first review found a problem: the FreeBSD implementation does not | conform to the C standard or to POSIX. These standards require that | successful calls to localtime and gmtime must always return the same | pointer; They do nothing of the kind. They allow implementations to return the same data structure (a pointer to ...) they certainly do not require it. | see, for example: | https://pubs.opengroup.org/onlinepubs/9799919799/functions/gmtime.html Yes, let's read that: | It says, "The asctime(), ctime(), gmtime(), and localtime() functions | shall return values in one of two static objects: a broken-down time | structure and an array of type char. That just requires that the implementation provide storage, and by being static, storage that does not need to be (and cannot be) freed by the application. The "one of two" is just that some return a pointer to a struct tm, and others to a char []. But this next sentence is where it matters: | Execution of any of the functions | that return a pointer to one of these object types may overwrite the | information in any object of the same type pointed to by the value | returned from any previous call to any of them." Notice "may overwrite" not "shall overwrite" which it would need to be to require that there only be one struct tm for all of the functions which return a pointer to that, and one char[] for all the functions which return char *. The implementation is permitted to have all the tm returning functions use the same one, or all in the same thread use the same one, or all calls of the same function name use the same one, but different functions use a different one, or that, with different copies in different threads, or just about anything else it desires - just as long as it doesn't malloc() storage for the results (which would require a free() in the application). | FreeBSD doesn't do that: it returns a pointer to thread-local storage. That's perfectly fine. | Is it intended that FreeBSD not conform to POSIX and C here? On that issue at least (I didn't look at the rest of the code), it does. | I suppose the idea was that poorly-written unportable programs that use | localtime in multiple threads are more likely to behave as their | misguided authors intended. Probably, and I agree, such an application would not be portable, perhaps not conforming either, but that's not on the implementation. | Still, it seems clear there is a conformance issue here. Not in the implementation. | I suspect that when the 2009 change was put in, its | reviewers didn't know about the conformance bug. Hardly surprising, as there isn't one. The supposed test program is nonsense, both the reported results from Fedora, and FreeBSD, are conforming. kre

Date: Tue, 23 Sep 2025 18:49:47 -0700 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <99d51a4c-e15a-4fcb-b239-007b3918dae1@cs.ucla.edu> | I don't see that as a plausible reading. There really is no other. | The text says "one of two static objects", Yes, the wording is bizarre, attempting to specify the return data from localtime() and ctime() all in one sentence, in order to save a paragraph, is idiotic. | not "a pointer of one of two types". That distinction is not material, the functions return pointers, the object the pointer points to is what matters, not the pointer itself. | There are two | static objects, one for localtime/gmtime and the other for | asctime/ctime, and that's pretty clear from the wording. If you concentrate only on that first sentence, I can see how you might come to that conclusion, but the second sentence makes it absolutely clear that cannot be what is required. Look at it again: Execution of any of the functions that return a pointer to one of these object types may overwrite First we have "may overwrite" not "shall overwrite" - "may" means the implementation is permitted to, but not required to. If there was only permitted to be one struct tm for all of the results, that "may" would make no sense at all - every call would (necessarily) overwrite that struct. But wait, there's more: | the information in any object of the same type "In any object of the same type" - how can that make any sense to say if there's only permitted to be one of them. What other objects could that "any" be referring to? For that to make any sense it must be that it is possible that there are more than one. | And I'm not the only person to read the standards this way. That I cannot help. Unfortunately, lots of people don't spend the effort required to really understand what any of these things actually say. | Although later C standards relax this (and I'll have more to say about | that in a later email), POSIX.1-2024 still has the C89 requirement. Yes, while I don't have a C standard that old, the oldest ones I have have identical wording to POSIX (hardly surprising, POSIX "borrows" the C standard language as much as it can). If what is there is the same as in C89, then it has clearly been misinterpreted, all of this time, as that has both the "may" and "any object of the same type" which make it clear that there is no such requirement. There never was. | As you probably know, the standards did not invent this requirement. No, they didn't, as it doesn't exist. | In 7th edition Unix, This predates that, localtime() ctime() (etc) were not new in 7th edition (though lots of the rest of libc as we know it now was) - what was new was "struct tm" - in earlier versions that was int tm[9]; (or something like that) - ie: the return value was a pointer to an array of 9 integers, (to be strictly correct, it was a pointer to the first integer in that array) which contained the secs/mins/hours/mday/mon/year/wday/yday/isdst values (which morphed into struct tm in 7th edition - not actually changing anything, but giving names to the fields ... if code wanted to treat it the old way, that still worked as well, type checking back then was kind of non-existant). | localtime and gmtime calls always returned the same pointer Yes, they did, and that's why the standard allows that behaviour. There is a big difference between allowing it, and requiring it, however. There's no question that it is permitted to act like that in the standard (even C23 with its additions), and application code should not expect anything different. That does not mean (and never did) that application code is entitled to assume that that relationship between gmtime() and localtime() existed, it just has to allow for the possibility that it might. An implementation if it wanted could (assuming that the _r functions exist) implement localtime() & gmtime() like static struct ltm[N]; static int nxtltm = 0; struct tm * localtime(time_t *t) { if (nxtltm >= N) nxtltm = 0; return localtime_r(t, <m[nxtltm++]); } static struct gtm[N]; static int nxtgtm = 0; struct tm * gmtime(time_t *t) { if (nxtgtm >= N) nxtgtm = 0; return gmtime_r(t, >m[nxtgtm++]); } for whatever non-negative value of N it likes (including 0), and it could skip gtm & nxtgtm and use ltm and nxtltm in gmtime() as well if it wanted, and doing that, with N==0, achieves what you believe is required, though any implementation like this (any value of N) is permitted. This implementation may overwrite a value in any of the previously returned structs of the same type. Exactly as the wording says may be done. The CSRG (BSD 4.4(ish)) man page for this stuff includes (from 1993) BUGS Except for difftime() and mktime(), these functions leaves their result in an internal static object and return a pointer to that object. Subsequent calls to these function [sic] will modify the same object. So even way back then (actually, well before then) this behaviour was recognised as being a bug, and of course, anything which is a bug is something that's liable to be fixed at any time - no-one should be relying upon it remaining (but of course, nor should anyone be assuming that any particular implementation will have fixed it). The 6th edition man page just says (of the results of localtime() and gmtime()): The value is a pointer to an array whose components are ... While those two functions certainly used the same array for the result, the manual page doesn't say that (doesn't even really imply it). It isn't even explicit that "the value" will be the same pointer from two calls to localtime() (or gmtime()) - though there's no doubt that's how the implementation worked. It has simply never been safe for an application to assume that the result from any of these calls *must* be the same storage space as any of the others. Similarly it has never been safe to assume that it won't be - it can happen either way. kre ps: not directly relevant, but slightly, as it touches on how sloppy all of this old wording is, the paragraph in question says: Execution of any of the functions that return a pointer to one of these object types may overwrite the information in any object of the same type pointed to by the value returned from any previous call to any of them. That means, for example, that the char [] array that ctime() returns can be overwritten by another call to ctime(), or a call to asctime() (as both of those return a char *), but nothing there permits ctime() to overwrite the information returned by localtime() (or gmtime()) as that is not "information in an object of the same type returned ...". Yet we all know (I hope) that (until time_t became 64 bits and localtime() could fail, and the difference that made is immaterial here) that the implementation of ctime() is (was) just: char *ctime(time_t *t) { return asctime(localtime(t)); } and as localtime() clobbers the static struct tm it returns, ctime() clobbers it as well - even though the standard does not say that is permitted, that's not an object of the same type as that returned.

Date: Wed, 24 Sep 2025 07:39:27 -0700 From: Paul Eggert <eggert@cs.ucla.edu> Message-ID: <b5bd3cd6-6662-4911-a647-7acd06302ad3@cs.ucla.edu> | On 2025-09-24 00:36, Robert Elz wrote: | > If what is there is the same as in C89 | | It's not. C89 �4.12.3 "Time conversion functions" says, "Except for the | strftime function, these functions return values in one of two static | objects: a broken-down time structure and an array of char. Execution of | any of the functions may overwrite the information returned in either of | these objects by any of the other functions." That is actually more reasonable (more accurate to what is permitted) in that it allows ctime() to alter the struct tm that localtime() returns, but apart from that it is almost just the same. | And �4.12.3.2 says that ctime(x) is equivalent to | asctime(localtime(timer)). Yes, that's always been there, and says what the effect is, but that kind of thing doesn't require the implementation to be that. Once again, allows it, does not require it. | This wording is | intended to require the 7th Edition Unix behavior, where calling any of | localtime, gmtime, and ctime overwrote the same struct tm object. No it isn't, it is meant to allow that behaviour - no-one with even an iota of sanity would require things to be implemented that way. | So in C89 it's crystal clear that successful calls to localtime and | gmtime must return the same pointer. It doesn't say that at all, it still says "may overwrite", you're much too hung up on the "two objects" part. | C23 makes it clear that the requirement is no longer present; What they did there (which is the obvious change to make) is to more reasonably define a sane lifetime for the static data that is being returned, to avoid the issue that your gmt3.c code encountered (not that I can imagine any real code ever acting like that). If they were making a fundamental change to the way that these functions were required to behave, the way you are imagining they did, there would have been much more noise about this - that would have been a seriously major change. What's more, if the requirement was as you imagine it to have been, the change that they made would have not been needed, as there could never have been other than one static struct tm, global, and its lifetime would never have been in doubt. | C99 through C17 (and therefore | POSIX.1-2001 through POSIX.1-2024) relax the C89 requirement to allow | localtime and gmtime to use different objects, Even C89 permitted that. | But even under your more relaxed interpretation, the FreeBSD behavior | does not conform to POSIX versions through POSIX.1-2024, or to C89 | through C17, because it sometimes frees storage addressed by the | returned values of localtime and gmtime, something these standards do | not allow. That I agree with, which is why I did not object to your gmt3.c test, which exposes that issue. When you asked: | Changed to what, though? in response to a suggestion that the standard should be altered, I was (before you pointed out the C23 change) considering suggesting something essentially equivalent to that change - it is the obvious, and sane, thing to do. | Any library that wants to support old binaries that assume | pre-POSIX.1-2001 behavior must have just one object for both gmtime and | localtime. Nonsense. | This 7th Edition Unix behavior conforms to all the standards, It does. | and portable programs should allow (though not insist on) this behavior. Exactly. The only reason for having the library act the way you say it is required to act, is to support imaginary applications which might assume this is required. I've never seen one, the typical application issue is in assuming that the results aren't modified, I've never seen one that insists that they must be. Never. So, requiring the library (the implementation) to act like that, so those imaginary applications keep on working (like your gmt2.c - the only code I have ever seen which tests that) is just too much. How much effort the implementation should take to allow broken code to work is up to the implementor - but the FreeBSD compromise seems about right to me. In yet another message (the one that attached gmt2.c) you noted: | I see that a single-threaded program does not have this issue: gmtime and | localtime return the same pointer in single-threaded programs. So if the | motivation is to cater to poorly-written unportable programs, it appears | that this motivation doesn't extend to the common mistake of thinking that | gmtime and localtime return different pointers. which is correct - the difference between the two is that the non handled issue is trivial for an application to fix struct tm tm1, tm2; tm1 = *localtime(&t); tm2 = *gmtime(&t); (ideally with a little error checking for a NULL return included). The lifetimes of tm1 and tm2 can be whatever the application needs, and nothing will just randomly alter those. But there's no trivial technique like that which will handle multiple threads doing calls like these, and apart from using different objects in different threads, there's nothing that the implementation can possibly do which can deal with this problem. An application which knows it is threaded can deal with this issue (simply call the _r functions instead), but should that application need to make use of a library which simply calls the original (old functions) when it needs to, there is no solution other than implementing locks around every call to any function in that library to prevent 2 threads from ever calling functions in that library concurrently - and that's even if the library has written the code as above, believing that would avoid issues with static state, and that it should work - but it just doesn't. kre

It is pointless discussing this further, what is in C23 is much better than it was before, and that will certainly be what is in POSIX after its next revision (whenever that happens). If you want to keep implementing things the way you believe the standard requires, that's fine it is certainly permitted, and applications should be able to cope with that. Or if you wanted to move to the C23 version now, that's also fine, it really isn't a departure from what the standards currently require, except as it applies to the lifetime of the returned object, and for that the new text is really the only rational way of handling that. If you ever come across a real application which breaks because the pointer returned from gmtime() and the one from localtime() aren't identical, or because calling one of those functions fails to overwrite the result from an earlier call, please let us know. The chances of finding one are about the same as the chances of finding any of the mythical creatures that are sometimes thought to exist. kre ps: implementation tests, and similar things, are not real applications.

Magnus Fromreide via tz wrote in <aNlfwKKePtMOEWFB@rickard.link.teligent.org>: |On Wed, Sep 24, 2025 at 12:57:49PM -0700, Paul Eggert via tz wrote: |> On 2025-09-24 10:14, Robert Elz wrote: |>> Date: Wed, 24 Sep 2025 07:39:27 -0700 |>> From: Paul Eggert <eggert@cs.ucla.edu> |>>| This wording is |>>| intended to require the 7th Edition Unix behavior, where calling any of |>>| localtime, gmtime, and ctime overwrote the same struct tm object. |>> |>> No it isn't, it is meant to allow that behaviour - no-one with even an |>> iota of sanity would require things to be implemented that way. |> |> If only you could go back to the 1980s and whisper that into Douglas \ |> Gwyn's |> ears! | |If I could go back and whisper in his ears I would whisper that the output |buffer should be passed as an argument, that way the _r versions wouldn't \ |be |needed and this whole issue would evaporate. I would ask for objects, as via tzalloc(), which strives me as the only truly sane interface: you can implement all others with it. Paul Eggert wants to port them for about ten years also to their GNUlib and GNU's libc, but these heavyweights seem to miss them still. It would surely aid in standard adoption if they came. --End of <aNlfwKKePtMOEWFB@rickard.link.teligent.org> --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)

Testtt