Compiler warning in tzcode2016g about SIZE_MAX being wider than int

When trying to build 2016g using macOS's current compiler (or, probably, any reasonably late-model version of clang), I get cc -DTZDIR=\"/usr/local/etc/zoneinfo\" -c -o zic.o zic.c zic.c:434:51: warning: implicit conversion from 'unsigned long long' to 'int' changes value from 18446744073709551615 to -1 [-Wconstant-conversion] int amax = nitems_max < SIZE_MAX ? nitems_max : SIZE_MAX; ~~~~ ^~~~~~~~ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include/stdint.h:153:20: note: expanded from macro 'SIZE_MAX' #define SIZE_MAX UINT64_MAX ^~~~~~~~~~ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include/stdint.h:87:27: note: expanded from macro 'UINT64_MAX' #define UINT64_MAX 18446744073709551615ULL ^~~~~~~~~~~~~~~~~~~~~~~ 1 warning generated. Or in other words, clang is pointing out, quite correctly, that SIZE_MAX won't fit in an int. This doesn't represent a runtime bug because the condition "nitems_max < SIZE_MAX" can never fail, but that just begs the question why have it. I am not aware of any C implementation that has ever had INT_MAX > SIZE_MAX, and surely there are none in use today. I suggest, therefore, a fix along the lines of $ diff -u zic.c~ zic.c --- zic.c~ 2016-09-06 00:39:50.000000000 -0400 +++ zic.c 2016-10-20 13:06:39.000000000 -0400 @@ -431,8 +431,7 @@ return ptr; else { int nitems_max = INT_MAX - WORK_AROUND_QTBUG_53071; - int amax = nitems_max < SIZE_MAX ? nitems_max : SIZE_MAX; - if ((amax - 1) / 3 * 2 < *nitems_alloc) + if ((nitems_max - 1) / 3 * 2 < *nitems_alloc) memory_exhausted(_("int overflow")); *nitems_alloc = *nitems_alloc + (*nitems_alloc >> 1) + 1; return erealloc(ptr, size_product(*nitems_alloc, itemsize)); Alternatively one might think of changing the types of nitems_max and amax to size_t --- but that would just make it even more crystal clear that the "nitems_max < SIZE_MAX" conditional is useless. And if, somewhere, one did find a C implementation in which size_t is narrower than int, that coding wouldn't work at all. Basically I don't see a way of dealing with this that is superior to just assuming that INT_MAX is less than or equal to SIZE_MAX. regards, tom lane

On 10/20/2016 10:15 AM, Tom Lane wrote:
Or in other words, clang is pointing out, quite correctly, that SIZE_MAX won't fit in an int. This doesn't represent a runtime bug
Yes, it's a false alarm. I often get false alarms like that when using Clang. I've been too lazy to report them to the Clang maintainers, but perhaps you can find the energy....
I am not aware of any C implementation that has ever had INT_MAX > SIZE_MAX, and surely there are none in use today.
The test is there partly because the code attempts to be as portable as possible, and partly because the code shouldn't limit the number of zones etc. to INT_MAX and this test is a placeholder for fixing that problem later. Generally speaking in C, object sizes should be limited only by min(PTRDIFF_MAX, SIZE_MAX). I'll see if I can rewrite the code to remove the INT_MAX restriction, which is arbitrary here. Not sure whether this will pacify Clang, but it's really a Clang bug so this is low priority.

On 10/20/2016 04:05 PM, Paul Eggert wrote:
I'll see if I can rewrite the code to remove the INT_MAX restriction, which is arbitrary here.
Done, by installing the attached patch into the development repository. I don't know whether this will pacify Clang, but at least it fixes some (mostly theoretical) bugs in zic. By the way, people may have some trouble accessing GitHub now, due to the ongoing denial-of-service attack on Dyn. I had to bypass DNS to access GitHub myself. See: Leyden J. Dyn dinged by DDoS: US DNS firm gives web a bad hair day. The Register 2016-10-21. http://www.theregister.co.uk/2016/10/21/dns_dyn_ddos/

Paul Eggert <eggert@cs.ucla.edu> writes:
On 10/20/2016 04:05 PM, Paul Eggert wrote:
I'll see if I can rewrite the code to remove the INT_MAX restriction, which is arbitrary here.
Done, by installing the attached patch into the development repository. I don't know whether this will pacify Clang, but at least it fixes some (mostly theoretical) bugs in zic.
Thanks, I can confirm that this silences the warning with macOS's version of clang. I'm a little confused as to why, since on this machine ptrdiff_t is "long int" and size_t is "long unsigned int", so that SIZE_MAX still doesn't fit into amax if one is being pedantic. I suppose that clang's check must be calibrated to allow stuffing an unsigned value into a signed value of the same width, just not narrower widths. regards, tom lane

Tom Lane said:
I suppose that clang's check must be calibrated to allow stuffing an unsigned value into a signed value of the same width, just not narrower widths.
Three things to note about this sort of stuff in C: (1) The range of values of an unsigned type has to include all the non-negative values of the corresponding signed type, but doesn't have to be any bigger. In other words, INT_MAX == UINT_MAX is allowed. (2) Just because two types with the same signedness have the same size as given by sizeof, they don't have to have the same range of values. So even if sizeof(int) == sizeof(long), INT_MAX != LONG_MAX is still allowed. (3) Just because two types have the same size, signedness, and range of values, doesn't mean they have the same representation. For example, int could be big-endian and long little-endian. -- Clive D.W. Feather | If you lie to the compiler, Email: clive@davros.org | it will get its revenge. Web: http://www.davros.org | - Henry Spencer Mobile: +44 7973 377646

I wrote:
Paul Eggert <eggert at cs.ucla.edu> writes:
On 10/20/2016 04:05 PM, Paul Eggert wrote:
I'll see if I can rewrite the code to remove the INT_MAX restriction, which is arbitrary here.
Done, by installing the attached patch into the development repository. I don't know whether this will pacify Clang, but at least it fixes some (mostly theoretical) bugs in zic.
Thanks, I can confirm that this silences the warning with macOS's version of clang. I'm a little confused as to why, since on this machine ptrdiff_t is "long int" and size_t is "long unsigned int", so that SIZE_MAX still doesn't fit into amax if one is being pedantic.
I installed this update into Postgres, and now our weekly run of Coverity is unhappy: *** CID 1373152: Control flow issues (DEADCODE) /srv/coverity/git/pgsql-git/92stable/src/timezone/zic.c: 452 in growalloc() 446 { 447 if (nitems < *nitems_alloc) 448 return ptr; 449 else 450 { 451 ptrdiff_t nitems_max = PTRDIFF_MAX - WORK_AROUND_QTBUG_53071;
CID 1373152: Control flow issues (DEADCODE) Execution cannot reach the expression "18446744073709551615UL" inside this statement: "amax = ((nitems_max < 18446...".
452 ptrdiff_t amax = nitems_max < SIZE_MAX ? nitems_max : SIZE_MAX; 453 454 if ((amax - 1) / 3 * 2 < *nitems_alloc) 455 memory_exhausted(_("integer overflow")); 456 *nitems_alloc += (*nitems_alloc >> 1) + 1; 457 return erealloc(ptr, size_product(*nitems_alloc, itemsize));
This is sort of the same thing as the original clang complaint, although expressed differently: the test against SIZE_MAX is still useless because it doesn't fit in ptrdiff_t any more than it did in int. If I were you, I'd change growalloc() so that its nitems-related arguments are defined as size_t not ptrdiff_t, and forget about this idea that ptrdiff_t has anything to do with the limit of what can be requested from malloc. None of this hoop-jumping has anything to do with any data set zic will ever see, anyway. regards, tom lane

Date: Sun, 06 Nov 2016 11:00:02 -0500 From: Tom Lane <tgl@sss.pgh.pa.us> Message-ID: <22781.1478448002@sss.pgh.pa.us> | I installed this update into Postgres, and now our weekly run of Coverity | is unhappy: After tzcode2016i was added to NetBSD this broke the NetBSD builds, because NetBSD (for whatever reason) uses almost every gcc (or clang) warning that exists (just a couple that are almost always bogus are excluded) and also has -Werror - so any warnings are fatal they all need to be cleaned up. | *** CID 1373152: Control flow issues (DEADCODE) That's really to be expected, the code is attempting to make a run time decision about which of ptrdiff_t and size_t has more bits - for any particular system, the answer will be fixed, and hence the run time testing must have dead code (we just don't know before compiling which branch it will be.) Best would be just to disable that warning for this line. | 451 ptrdiff_t nitems_max = PTRDIFF_MAX - WORK_AROUND_QTBUG_53071; | 452 ptrdiff_t amax = nitems_max < SIZE_MAX ? nitems_max : SIZE_MAX; The problem NetBSD experienced (initially) is because of the comparison itself, nitems_max is signed, and SIZE_MAX is unsigned, and gcc warns about comparing signed values to unsigned (as the comparison promotes the signed value to unsigned first, which makes small negative values seem big, and often uncovers code bugs). Here however, even though nitems_max is a signed type, it has a known positive value, so the bug really can't happen, but gcc (at least) is just too stupid to work that out. Currently that's "fixed" (in NetBSD) by casting nitems_max to a size_t explicitly for the comparison, but that actually breaks the algorithm (it is fine for NetBSD as size_t and ptrdiff_t contain the same number of bits, but if a ptrdiff_t was a wider type than size_t, which some arguments would suggest it should be, then the cast reduces the magnitide of nitems_max and would likely (well, possibly) lead to incorrect results. | This is sort of the same thing as the original clang complaint, although | expressed differently: the test against SIZE_MAX is still useless because | it doesn't fit in ptrdiff_t any more than it did in int. That depends upon the number of bits in a ptrdiff_t - consider a 64 bit ptrdiff_t with a 32 bit size_t for example. The code is actually very clever, and if it weren't for all these (not nearly so clever) analysers attempting to second guess what is happening and getting it wrong, it would be fine as it is. I had been wondering about the need for nitems_max as a variable though and wondered if perhaps #define NITEMS_MAX (PTRDIFF_MAX - WORK_AROUND_QTBUG_53071) ptrdiff_t amax = NITEMS_MAX < SIZE_MAX ? NITEMS_MAX : SIZE_MAX; might not work just as well, and avoid (at least) the gcc warning (though I am yet to actually test this hypothesis). It will probably fall foul of the "constant comparison" warning though. Ugh!!! kre

Robert Elz <kre@munnari.OZ.AU> writes:
I had been wondering about the need for nitems_max as a variable though and wondered if perhaps
#define NITEMS_MAX (PTRDIFF_MAX - WORK_AROUND_QTBUG_53071) ptrdiff_t amax = NITEMS_MAX < SIZE_MAX ? NITEMS_MAX : SIZE_MAX;
might not work just as well, and avoid (at least) the gcc warning (though I am yet to actually test this hypothesis). It will probably fall foul of the "constant comparison" warning though. Ugh!!!
Yeah, I'd been wondering about that too, but it seems unlikely to avoid warnings. What about doing the comparison in an #if? #define NITEMS_MAX (PTRDIFF_MAX - WORK_AROUND_QTBUG_53071) #if NITEMS_MAX < SIZE_MAX #define AMAX ((ptrdiff_t) NITEMS_MAX) #else #define AMAX ((ptrdiff_t) SIZE_MAX) #endif ... then use AMAX in place of the variable ... regards, tom lane

Tom Lane wrote:
I'd change growalloc() so that its nitems-related arguments are defined as size_t not ptrdiff_t, and forget about this idea that ptrdiff_t has anything to do with the limit of what can be requested from malloc.
ptrdiff_t is relevant since zic subtracts pointers, and subtraction has undefined behavior if the result doesn't fit into ptrdiff_t. All other things being equal I prefer signed integer arithmetic to unsigned, because on some platforms overflow checking works with signed arithmetic and this can help find programming errors. This is why growalloc uses ptrdiff_t rather than size_t even though either would do.
What about doing the comparison in an #if?
#if is best avoided when possible (it can't always be). Robert Elz wrote:
if it weren't for all these (not nearly so clever) analysers attempting to second guess what is happening and getting it wrong, it would be fine as it is.
Yes, attempting to pacify all these analyzers can contort the code, which I'd rather avoid. I don't even like the INITIALIZE macro, and considered adding -Wno-maybe-uninitialized to GCC_DEBUG_FLAGS so that INITIALIZE can be removed. I kept INITIALIZE only because I use GCC and -Wmaybe-uninitialized is so useful in finding real bugs elsewhere. As far as Coverity etc. go, I would rather that people filed bug reports to get these other static analyzers fixed, as I try to do with GCC. I just now checked tzcode with Clang and found one recently-added bit of unnecessary code, which I fixed with the attached.
participants (4)
-
Clive D.W. Feather
-
Paul Eggert
-
Robert Elz
-
Tom Lane