Hacker News Reader

> Previously, in C89, left shift on signed integers was completely undefined...

No, it was implementation defined, which is very different. Every implementation you were ever likely to encounter defined it reasonably. Making some of it undefined now has created a bunch of code that uses macros to cast back and forth between unsigned to get the expected behavior (or, for cases where it is provable that the result is representable, to replace the left shift with a multiplication and hope the optimizing compiler translates it back, because compilers now warn on any left shift of a negative value, despite the fact that the semantics are supposed to be the same as the corresponding multiplication now).

In other words, it created a mess for basically no reason.

This is the complete definition of the behavior of left shift operators in C89:

> The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are filled with zeros. If E1 has an unsigned type, the value of the result is E1 multiplied by the quantity, 2 raised to the power E2, reduced modulo ULONG_MAX+l if E1 has type unsigned long, UINT_MAX+1 otherwise. (The constants ULONG_MAX and UINT_MAX are defined in the header <limits.h>.)

You notice what's not mentioned there? What happens if E1 is signed. The standard says nothing. And when the standard says nothing, it's undefined behavior.

The behavior at the bit level seems very well defined based on that excerpt. You would need to know how signed integers are defined to know what the numeric value of the result would be. That sounds like it would be implementation dependent, but also completely predictable. Am I missing something?

From the beginning of Section 3.3 (several pages before the text you quoted in Section 3.3.7):

      Some operators (the unary operator ~ , and the binary operators << ,
  >> , & , ^ , and | , collectively described as bitwise operators)
  shall have operands that have integral type.  These operators return
  values that depend on the internal representations of integers, and
  thus have implementation-defined aspects for signed types.

this only warn that there is some implementation-defined stuff coming next, not that all these operator are free of undefined behaviour

I would be very curious to know why such shifts would be UB as opposed to merely implementation specific.

I guess I can understand that signed shifts bump up against how the hardware represents signed numbers at the bit level, but it’s not like the assembly is unclear on any given platform.

Some ancient machines trapped on signed overflow, including the ones caused by arithmetic shift left (as opposed to logical shift left), and apparently you can't define your implementation-defined behaviour as "oh, it traps in this case" — although still I don't believe the last part.

But if it is true, then yeah, that'd make (some) sense. I say "some" because surely in this case the implementations could simply emit the "logical shift left" instruction for the shift operation both for ints and unsigned ints? I don't think there ever was a machine that actually tracked the signedness/unsignedness of registers.

Yeah that’s what I was thinking. I didn’t think of the traps until I read the sibling threads, but even that doesn’t seem like an impediment.

> My guess is that it has to do with signed-magnitude representation causing left shifts to be absolute garbage with negative numbers.

First of all, why not make it implementation-defined? That seems rather more useful for the programmer. Second, I'm personally aware of only a single machine with signed-magnitude representation, IBM 7090, and its ALS (Accumulator Left Shift) instruction treated the sign bit as sticky; amusingly, LGL (Logical Shift Left) did as well, so you had to put the value being shifted into the MQ register, not the accumulator, when you wanted an unsigned shift. So... no, that can't be it?

One of the tricky things is that undefined behavior is commonly understood as having an unlimited "blast radius" and implementation-defined behavior is commonly understood as not permitting that kind of "blast radius." So any extension point that needed that unlimited "blast radius" had to be undefined behavior. (This has changed somewhat in modern times, but it does hold a little bit today--I'd point you to the discussion of N3863 at the most recent meeting, but I don't have any minutes that I feel really captured that discussion well enough.)

And one of those kinds of "blast radius" is traps, and I suspect the original motivation for making signed integer overflow undefined behavior was precisely to preserve the ability to trap on signed integer overflow, which couldn't be implementation-defined. But the records of the discussions leading up to C89 and C99 are just too fragmentary for me to be certain.

The problem with UB, as evidenced over the years, is that it causes compiler writers to drift much too far from reality, despite the standard itself suggesting "behaving [...] in a documented manner characteristic of the environment" as one of the (better) choices.

> And one of those kinds of "blast radius" is traps,

Wait. An implementation-defined behaviour isn't allowed to be trap/termination of execution? Okay, make it unspecified then, with the options of trapping or calculating some unspecified (not indeterminate) integer value.

> N3863

Hm?

    Because implementation-defined behavior relies on the definition of unspecified behavior, it
    means that implementation-defined behavior requires two or more possibilities to be defined in
    the standard. However, a brief look at its uses in the standard clearly shows that there isn’t
    always a list of potential behaviors to pick from.

Eh, I'd argue that omission of such a list is merely an implicit infinite ("two or more", you know) list of possibilities, its construction being self-evident.

    N3783 5.2.1.2p1.1: Physical source file multibyte characters are mapped, in an implementation-
    defined manner, to the source character set (introducing new-line characters for end-of-line
    indicators) if necessary.

There is an infinite list of possible ways to map multibyte characters to the source character set — an implementation has to pick one and document it.

    N3783 5.2.1.3p1: A conforming implementation shall produce at least one diagnostic message
    (identified in an implementation-defined manner)

There is an infinite list of possible diagnostic messages...

    N3783 5.2.2.2p1-2: In a freestanding environment (in which C program execution can take
    place without any benefit of an operating system), the name and type of the function called at
    program startup are implementation-defined.

There is an infinite list of possible C identifiers that an implementation may chose as the name of the function to call at program startup.

N3863 is one of those papers where what the paper proposed and what was actually discussed in the meeting ended up being rather different things, and it's the meeting discussion that I'm pointing to, not the paper itself.

> Wait. An implementation-defined behaviour isn't allowed to be trap/termination of execution? Okay, make it unspecified then, with the options of trapping or calculating some unspecified (not indeterminate) integer value.

Unspecified behavior also isn't allowed to be a trap. The difference between unspecified and implementation-defined is whether or not the compiler has to document its choice; the difference between unspecified and undefined is whether or not the choice is limited.

There is already a precedent for a signed integer operation to produce an implementation-defined value or to trap. It's for integer conversion:

https://open-std.org/JTC1/SC22/WG14/www/docs/n3220.pdf#subsu...

> Otherwise, the new type is signed and the value cannot be represented in it; either the result is implementation-defined or an implementation-defined signal is raised.

There is no reason that this wording strategy could not be reused for bit shifts.

In general there is no constraint that unspecified or implementation-defined behavior can't be a trap. Without further qualification "unspecified behavior" or "implementation-defined behavior" is still open-ended, and it's up to the surrounding standard text to restrict the behavior further.

https://open-std.org/JTC1/SC22/WG14/www/docs/n3220.pdf#subse...

> behavior, that results from the use of an unspecified value, or other behavior upon which this document provides two or more possibilities and imposes no further requirements on which is chosen in any instance

"Possibilities" include trap

Well, you could start to help? (instead of just being there to make sure special interests are preserved in voting)

I also do not see where the quality suffered. There were some rather minor issues we created and need to cleaned up.

> (instead of just being there to make sure special interests are preserved in voting)

This kind of attitude tends to make me suspect that you are disinclined to actually listen to my feedback.

Prove it. Aaron told me explicitly that he was inviting people because he was not happy that there are too many with a user view point. And then a couple people show up that never contribute papers but consistently vote for some special interest.

Just as a recent example. There is a case where the standard obviously is broken . With a lot of support by most of the committee and actual work by a couple of people there is a paper that aims to fix it. The only people who never constructively helped but still vote against this important repair is your block of other non-contributing people you are part of: https://github.com/c-lang-org/polls/blob/main/n3909.md

(Also a counter proposal would be constructive.)

But I apologize, I should not be so negative. But I hope you see where the frustration comes from. The cleanup efforts are driven by a lot of volunteer work. We could use any help and instead of people just voting against such works.

> 404

Very informative, thanks

yes, internal link. It is about discarded: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3909.htm

The complaint that has been brought up there--repeatedly--is that the term 'discarded' creates confusion with the similar 'discarded-value expression' in C++, which is completely different semantically despite the similar name.

I joined WG14 and WG21 because I was worried that there were not enough people on either committees who had the perspective of someone who works in middle-end optimization rather than the frontend. I haven't written any papers for WG14 because so far all of the feedback I need to give can easily be given merely by discussion on the reflector or during the paper discussions (this is not the case for WG21, and I have consequently written papers for them), and--as a middle-end person--there is actually relatively little feature development for me to specifically propose. Most of the big features I'd want to see included already have active proposals (e.g., _Vec) and there's little reason for me to write a different proposal. Maybe you'll eventually see a paper from me if I take up some of the proposal work for CFP, or if I get around to writing a version of restrict that's based off of the Tree Borrows model. Or maybe if I turn some of my thoughts on multiple return values into a paper, though I don't actually have a proper frontend proposal for that that I like, so it's more likely to remain a reflector message.

Yes, thanks. Sorry I am not trying to be hostile, I am just a little bit annoyed about the lack of progress we have in some areas.