flatfinger

In many cases, the semantics programmers would require are much looser than anything provided for by the Standard. For example, if a programmer requires an expression that computes (x \* y / z) when there is no overflow, and computes an arbitrary value with no side effects when there is an overflow, a programmer could write the expression with unsigned and signed casting operators, but that would force a compiler generate machine code to actually perform the multiplication and division even in cases where it knows that y will always be twice z. Under "yield any value with no side effects" semantics, a compiler could replace the expression with (x \* 2), which would be much faster to compute.

There are many cases where accommodating weird corner cases would be expensive, and would only be useful for some kinds of program. Requiring that all implementations intended for all kinds of task handle corner cases that won't be relevant for most kinds of tasks would needlessly degrade efficiency. The problem is that there's no way for programs to specify which corner cases they do or don't need.

> Yes, clearly.

Unfortunately, the C Standard doesn't specify an abstraction model that is amenable to the optimization of usable programs.

> In that case, I'm not sure what we're even arguing about; the language standard might or might not standardize a way of specifying said waiver, but as long as it's not lumped in with -On or -std=blah that are necessary to get a proper compiler, it has no bearing on real-world programmers that're just trying get working code. Hell, I'd welcome a -Ounsafe or whatever, just to see what sort of horrible mess it makes, as long -Ono-unsafe exists and is the default.

The only reason for contention between compiler writers and programmers is a desire to allow compilers to optimized based upon the assumption that a program won't do certain things. The solution to that contention would be to have a means of inviting optimizations in cases where they would be safe and useful, analogous to what `restrict` would be if the definition of "based upon" wasn't so heinously broken.

> With, I presume, a eye toward further producing: x.a[dat[i]] = i; y.a[dat[i]] = i;

Bingo.

> I assume you mean zero-extending; otherwise `x=255` would result in `result=-1`, which is clearly wrong.

Naturally.

> None of this involves that the compiler infering objective (and frequently false) properties of the input program (such as "this loop will terminate" or "p != NULL"), though.

Thus the need to use an abstraction model which allows optimizations to alter observable aspects of a program whose behavior is, generally, defined. I wouldn't describe such things as "behavior characteristic of the environment", though the environment would affect the ways in which the effects of optimizations might be likely to manifest themselves.

Note that programs intended for different tasks on different platforms will benefit from slightly--but critically--different abstraction models, and there needs to be a way for programs to specify when deviations from the "load/store machine model" which would normally be acceptable, aren't. For example, there should be a way of indicating that a program requires that automatic objects always behave as though initialized with Unspecified rather than Indeterminate Value.

A good general-purpose abstraction model, however, should allow a compiler to make certain assumptions about the behaviors of constructs, or substitute alternative constructs whose behaviors would be allowed to differ, but would not allow a compiler to make assumptions about the behaviors of constructs it has changed to violate them.

Consider, for example:

    typedef void proc(int);  // Ever seen this shorthand for prototypes?
    proc do_something1, do_something2, do_something3;

    void test2(int z)
    {
      if (z < 60000) do_something3(z);
    }

    int q;
    void test1(int x)
    {
      q = x*60000/60000;
      if (q < 60000) do_something1(q);
      int y = x*60000/60000;
      if (y < 60000) do_something2(y);
      test2(y);
    }

Under a good general-purpose model, a compiler could generate code that could never set q to a value greater than INT_MAX/60000, and a 32-bit compiler that did so could assume that q's value would always be in range and thus omit the comparison. A compiler could also generate code that would simply set q to x, but would forfeit the right to assume that it couldn't be greater than INT_MAX/60000.

There could be optimization value in allowing a compiler to treat automatic objects "symbolically", allowing the second assignment/test combination to become:

      if (x*60000/60000 < 60000) 
        do_something2(x*60000/60000);

even though the effect of the substituted expression might not be consistent. I wouldn't favor allowing inconsistent substitutions by default, but would favor having a means of waiving normal behavioral guarantees against them for local automatic objects whose address is not taken. On the other hand, there would need to be an operator which, when given an operand with a non-determinisitic value, would choose in Unspecified fashion from among the possibilities; to minimize security risks that could be posed by such values, I would say that function arguments should by default behave as though passed through that operator.

The guiding principle I would use in deciding that the value substitution would be reasonable when applied to y but not q or z would be that a programmer would be able to see how y's value is assigned, and see that it could produce something whose behavior would be "unusual", but a programmer looking at test2() would have no reason to believe such a thing about z.

Replying to the code [discussed deeper in this sub-thread]:

    struct blob { uint16_t a[100]; } x,y,z;
  
    void test2(void)
    {
      int indices[] = {1,0};
      {
        int* dat = indices;
        int n = 2;
        {
          struct blob temp;
          for(int i=0; i<n; i++) 
            temp.a[dat[i]] = i; // This is what I'd meant
          x=temp;
          y=temp;
        }
        z=x;
      }

The rewrite sequence I would envision would be:

    struct blob { uint16_t a[100]; } x,y,z;
  
    void test2(void)
    {
      int indices[] = {1,0};
      {
        int* dat = indices;
        int n = 2;
        {
          struct blob temp1 = x; // Allowed initial value
          struct blob temp2 = y; // Allowed initial value
          for(int i=0; i<n; i++)
          {
            temp1.a[dat[i]] = i;
            temp2.a[dat[i]] = i;
          }
          x=temp1;
          y=temp2;
        }
        z=x;
      }

Compilers may replace an automatic object whose address is not observable with two objects, provided that anything that is written to one will be written to the other before the latter is examined (if it ever is). Such a possibility is the reason why automatic objects which are written between "setjmp" and "longjmp" must be declared "volatile".

If one allows a compiler to split "temp" into two objects without having to pre-initialize the parts that hold Indeterminate Value, that may allow more efficient code generation than would be possible if either "temp" was regarded as holding Unspecified Value, or if copying a partially-initialized object as classified as "modern-style Undefined Behavior", making it necessary for programmers to manually initialize entire structures, including parts whose values would otherwise not observably affect program execution.

The optimization benefits of attaching loose semantics to objects of automatic duration whose address is not observable are generally greater than the marginal benefits of attaching those semantics to all objects. The risks, however, are relatively small since everything that could affect the objects would be confined to a single function (it an object's address is passed into another function, its address would be observable during the execution of that function).

BTW, automatic objects whose address isn't taken have behaved somewhat more loosely than static objects even in compilers that didn't optimized aggressively. Consider, for example:

    volatile unsigned char x,y;
    int test(int dummy, int mode)
    {
      register unsigned char result;
      if (mode & 1) result = x;
      if (mode & 2) result = y;
      return result;
    }

On many machines, if an attempt to read an uninitialized automatic object whose address isn't taken is allowed to behave weirdly, the most efficient possible code for this function would allocate an "int"-sized register for "result", even though it's only an 8-bit type, do a sign-extending load from `x` and/or `y` if needed, and return whatever happens to be in that register. That would not be a complicated optimization; in fact, it's a simple enough optimization that even a single-shot compiler might be able to do it. It would, however, have the weird effect of allowing the uninitialized "result" object of type "unsigned char" to hold a value outside the result 0..255.

Should a compiler be required to initialize "result" in that situation, or should programmers be required to allow for the possibility that if they don't initialize an automatic object it might behave somewhat strangely?

The indent is getting rather crazy on this thread; I'll reply further up-thread so as to make the indent less crazy.

Yeah, I meant `slow_function_no_side_effects(x)`. My point is that there's a huge difference between saying that a compiler need not treat a loop as sequenced with regard to outside code if none of the operations therein are likewise sequenced, versus saying that if a loop without side effects fails to terminate, compiler writers should regard all imaginable actions the program could perform as equally acceptable.

In a broader sense, I think the problem is that the authors of the Standard have latched onto the idea that optimizations must not be observable unless a program invokes Undefined Behavior, and consequently any action that would make the effects of an optimization visible must be characterized as UB.

I think it would be far more useful to recognize that optimizations may, on an opt-in or opt-out basis, be allowed to do various things whose effects would be observable, and correct programs that would allow such optimizations must work correctly for any possible combination of effects. Consider the function:

    struct blob { uint16_t a[100]; } x,y,z;

    void test1(int *dat, int n)
    {
      struct blob temp;
      for (int i=0; i<n; i++)
        temp.a[i] = i;
      x=temp;
      y=temp;

    }
    void test2(void)
    {
      int indices[] = {1,0};
      test1(indices, 2);
      z=x;
    }

Should the behavior of test2() be defined despite the fact that `temp` is not fully written before it is copied to `x` and `y`? What if anything should be guaranteed about the values of `x.a[2..99]`, `y.a[2..99]`, and `z.a[2..99]`?

While I would allow programmer to include directives mandating more precise behavior or allowing less precise behavior, I think the most useful set of behavioral guarantees would allow those elements of `x` and `y` to hold arbitrarily different values, but that `x` and `z` would match. My rationale would be that a programmer who sees `x` and `y` assigned from `temp` would be able to see where `temp` was created, and would be able to see that some parts of it might not have been written. If the programmer cared about ensuring that the parts of `x` and `y` corresponding to the unwritten parts matched, there would be many ways of doing that. If the programmer fails to do any of those things, it's likely because the programmer doesn't care about those values.

The programmer of function `test2()`, however, would generally have no way of knowing whether any part of `x` might hold something that won't behave as some possibly-meaningless number. Further, there's no practical way that the author of `test2` could ensure anything about the parts of `x` corresponding to parts of `temp` that don't be written. Thus, a compiler should not make any assumptions about whether a programmer cares about whether `z.a[2..99]` match `x.a[2..99]`.

A compiler's decision to optimize out assignments to `x[2..99]` and `y[2..99]` may be observable, but if code would not, in fact, care about whether `x[2..99]` and `y[2..99]` match, the fact that the optimization may cause the arrays to hold different Unspecified values should not affect any other aspect of program execution.

What should be relevant is not programmer "intent", but rather whether the behavior would likely match the that of an implementation which give that describe behavior of actions priority over parts of the Standard that would characterize them as "Undefined Behavior".

Has there ever been any consensus as to what that "...or is copied as an array of character type..." text is supposed to mean, or what sort of hoops must be jumped through for a strictly conforming program to generate an object whose bit pattern matches another without copying the effective type thereof?

The ability to use pointers to structures with a Common initial Sequence goes back at least to 1974--before unions were invented. When C89 was written, it would have been plausible that an implementation could uphold the Common Initial Sequence guarantees for pointers without upholding them for unions, but rather less plausible that implementations could do the reverse. Thus, the Standard explicitly specified that the guarantee is usable for unions, but saw no need to redundantly specify that it also worked for pointers.

If compilers would recognize that operation involving a pointer/lvalue that is freshly visibly based on another is an action that at least potentially involves the latter, that would be sufficient to make code that relies upon the CIS work. Unfortunately, some compilers are willfully blind to such things.

How about allowing `return` to be used as a qualifier within a function or prototype's argument which, if present, would adjust the qualifiers of the function's return value to match those of the argument, e.g. adding "const" or removing "volatile" as appropriate?

This, if one did:

    char *strchr2(char return *restrict src, char target)
    {
      while(*src)
      {
        if (*src == target) return src;
        src++;
      }
    }

then one version of the function could support both const and non-const usage. BTW, I'd also like to see "register" and "return register" be usable as a qualifiers for pointer-type function parameters which would promise that the passed in pointer wouldn't "escape", or else that it could only escape through the return value (so a compiler that could see everything done with the return value wouldn't have to worry about the argument escaping).

Functions like malloc are only required for hosted implementations. Many operating systems are built using freestanding implementations.

Further, on many platforms, one should avoid using malloc() unless portability is more important than performance or safety. Some operating systems support useful features like the ability to allocate objects with different expected lifetimes in different heaps, so as to help avoid fragmentation, or arrange to have allocations a program can survive without fail while there is still enough memory to handle critical allocations. Any library that insists upon using "malloc()" will be less than ideal for use with any such operating system.

> This won't work because defective implementations will just claim that their intended purpose is to do [whatever emergent behaviour that implementation produces], or to generate the fastest code possible regardless of whether that code bears any relation to what the programmer asked for.

I would have no qualm with the way clang and gcc process various constructs if they were to explicitly state that its maintainers make no effort to make their optimizer suitable for any tasks involving the receipt of untrustworthy input. Instead, however, they claim that their optimizers are suitable for general-purpose use, despite the fact that their behavior isn't reliably suitable for many common purposes.

> This is actually completely unneeded, even for optimisation.

Consider the following function:

    unsigned long long test(unsigned long long x, int mode)
    {
      do
        x = slow_function_no_side_effects();
      while(x > 1);
      if (mode)
        return 1;
      else
        return x;
    }

Suppose the function is passed a value of "x" which would get caught in a cycle that never hits zero or one, but "mode" is 1. If the code is processed by performing every individual step in order, the function would never return. The rule in C11 is designed to avoid requiring that generated code compute the value of x when its only possible effect on the program's execution would be to prevent the execution of code that doesn't depend on its value.

Suppose the most important requirement that function test() must meet is that it must never return 1 unless mode is 1, or the iteration on x would yield 1 before it yields zero; returning 1 in any other cases would cause the computer's speaker to start playing Barney's "I love you" song, and while looping endlessly would be irksome, it would be less bad than Barney's singing. If a compiler determines that slow_function_no_side_effects() will never return an even number, should it be entitled to generate code that will return 1 when mode is zero, without regard for whether the loop actually completes?

I would think it reasonable for a compiler to defer/skip the computation of x in cases where mode is 1, or for a compiler that can tell that "x" will never be an even number to generate code that, after ensuring that the loop will actually terminate, would unconditionally return 1. Requiring that the programmer write extra code to ensure that the function not return 1 in cases where mode is zero but the loop doesn't terminate would defeat the purpose of "optimization".

> If I understand correctly, that would actually be "implementation-defined", not "undefined".

That is an extremely common myth. From the point of view of the Standard, the difference between Implementation Defined behavior and Undefined Behavior is that implementations are supposed to document some kind of behavioral guarantee with regard to the former, even in cases where it would be impractical for a particular implementation to guarantee anything at all, and nothing that implementation could guarantee in those cases would be useful.

The published Rationale makes explicit an intention that Undefined Behavior, among other things, "identifies areas of conforming language extension".

> There are size-specific integer types specified in the C99 standard (e.g. `uint32_t`). I use those, except in the most trivial cases (e.g. `int main()`), and limit myself to those size-specific integer types that are "guaranteed" by the standard.

A major problem with the fixed-sized types is that their semantics are required to vary among implementations. For example, given

    int test(uint16_t a, uint16_t b, uint16_t c) { return a-b > c }

some implementations would be required to process test(1,2,3); so as to return 1, and some would be required to process it so as to return 0.

Further, if one has a piece of code which is written for a machine with particular integer types, and a compiler which targets a newer architecture but can be configured to support the old set of types, all one would need to do to port the code to the new platform would be to add a directive specifying the required integer types, with no need to rework the code to use the "fixed-sized" types whose semantics vary among implementations anyway.

See post above. There is no good way for compilers to handle that case, but gcc gets "creative" even in cases where the authors of C89 made their intentions clear.

I can't really blame gcc for that one, since the most straightforward way of using signed integer arithmetic would yield a negative value if the result is bigger than INT_MAX, but it would be very weird that programs would expect and rely upon that behavior.

On the other hand, even the function "unsigned mul_mod_65536(unsigned short x, unsigned short y) { return (x * y) & 0xFFFF; }" which the authors of the Standard would have expected commonplace implementations to process in consistent fashion for all possible values of "x" and "y" [the Rationale describes their expectations] will sometimes cause gcc to jump the rails if the arithmetical value of the product exceeds INT_MAX, despite the fact that the sign bit of the computation is ignored. If, for example, the product would exceed INT_MAX on the second iteration of a loop that should run a variable number of iterations, gcc will replace the loop for code that just handles the first iteration.

It's too bad Unicode wasn't designed around the concept of easily-recognizable grapheme clusters and "write-only" [non-round-trip] forms that are normalized in various ways. A text layout engine shouldn't have to have detailed knowledge of rules that are constantly subject to change, but if there were a standard representation for a Unicode string where all grapheme clusters are marked and everything is listed in left-to-right order, and an OS function was available to convert a Unicode string into such a form, a text-layout using that OS routine would be able to accommodate future additions to the character set and and glyph-joining rules without having to know anything about them.

A conforming implementation could extend the language with an 8-bit type __nonaliasingbyte which has no special aliasing privileges, and define uint8_t as being synonymous with that type.

On the other hand, the Standard should never have given character types special aliasing rules to begin with. Such rules would have been unnecessary if the Standard had noted that an access to an lvalue which is freshly visibly derived from another is an access to the lvalue from which it is derived. The question of whether a compiler recognizes a particular lvalue as "freshly visibly derived" from another is a Quality of Implementation issue outside the Standard's jurisdiction.

If the Standard were to make recursion an optional feature, many programs' stack usage could be statically verified. Indeed, there are some not-quite-conforming compilers which can statically verify stack usage--a feature which for many purposes would be far more useful than support for recursion.

If a compiler decides that it's going to process:

    unsigned mul(unsigned short x, unsigned short y)
    { return x*y; }

in a way that causes calling code to behave in meaningless fashion if x would exceed INT_MAX/y [something gcc will sometimes actually do, by the way, with that exact function!], the hardware isn't going to have any say in that.

If one wants a function that will compute xy/z when xy doesn't overflow, and yield some arbitrary value (but without other side-effects) when it does, wrapping functions will often be much slower than would be code that doesn't have to guarantee any particular value in case of overflow. If e.g. y is known to be 30 and z equal to 15, code using a wrapping multiply would need to be processed by multiplying the value by 30, computing a truncated the result, and dividing that by 15. If the program could use loosely-defined multiplication and division operators, however, the expression could be simplified to x+x.

72 static in-line functions. If a compiler does a good job of handling such things efficiently, most of them could be accommodated by chaining to another function once or twice (e.g. to read a 64-bit value that's known to be at least 16-bit aligned, on a platform that doesn't support unaligned reads, read and combine two 32-bit values that are known to be 16-bit likewise).

Far less bloat than would be needed for a compiler to recognize and optimize any meaningful fraction of the ways people might write code to work around the lack of portably-specified library functions.

What is meant by "portable code"? Should it refer only to code that should theoretically be usable on all imaginable implementations, or should it be expanded to include code which may not be accepted by all implementations, but which would have an unambiguous meaning on all implementations that accept it?

Historically, if there was some action or construct that different implementations would process in different ways that were well suited to their target platforms and purposes, but were incompatible with each other, the Standard would simply regard such an action as invoking Undefined Behavior, so as to avoid requiring that any implementations change in a way that would break existing code. This worked fine in an era where people were used to examining upon precedent to know how implementations intended for certain kinds of platforms and purposes should be expected to process certain constructs. Such an approach is becoming increasingly untenable, however.

If instead the Standard were to specify directives and say that if a program starts with directive X, implementations may either process integer overflow with precise wrapping semantics or refuse to process it altogether, if it starts with directive Y, implementations may either process it treating "long" as a 32-bit type or refuse to process it altogether, etc. this would make it much more practical to write portable programs. Not all programs would run on all implementations, but if many users of an implementation that targets a 64-bit platform need to use code that was designed around traditional microcomputer integer types, a directive demanding that "long" be 32 bits would provide a clear path for the implementation to meet its customers' needs.

BTW, what do you think of the suggested text I offered near the top of this thread, that UB represents a waiver of the Standard's jurisdiction for the purpose of allowing implementations to best serve their intended purposes? It's too late to go back in time and add that to C89 or C99, but a lot of insanity could have been avoided had such text been present.

Further, instead of characterizing as UB all situations where a useful optimization might affect the behavior of a program, it would be far much safer and more useful to allow particular optimizations in cases where their effects might be observable, but where all allowable resulting behaviors would meet application requirements.

As a simple example, instead of saying "a compiler may assume that all loops with non-constant conditions will terminate", I would say that if the exit of a loop is reachable, and no individual action within the loop would be observably sequenced with regard to some particular succeeding operation, a compiler may at its leisure reorder the succeeding operation ahead of the loop. Additionally, if code gets stuck in a loop with no side effects that will never terminate, an implementation may provide an option to raise a signal to indicate that.

If a function is supposed to return a value meeting some criterion, and it would find such a value in all cases where a program could execute usefully, but the program would do something much worse than useless if the function were to return a value not meeting the criteria, a program execution where the function loops forever may be useless, and may be inferior to one that gets abnormally terminated by the aforementioned signal, but may be infinitely preferable to one where the function, as a result of "optimization", returns a bogus value. Allowing a programmer to safely write a loop which might end up not terminating would make it possible to yield more efficient machine code than would be needed if the only way to prevent the function from returning a bogus value would be to include optimizer-proof code to guard against the endless-loop case.

> C's charter is to standardize existing practice (as opposed to invent new features), and no such feature has emerged in practice. Same for modules. (C++ takes a very different approach.)

One thing that I'd really like to see would be some new categories of compliance. At present, the definition of "conforming C program" makes it possible to accomplish any task that could be done in any language with a "conforming C program", since the only thing necessary for something to be a conforming C program would be for there to exist some conforming implementation in the universe that accepts it. Unfortunately, the Standard says absolutely nothing useful about the effect of attempting to use an arbitrary conforming C program with an arbitrary conforming C implementation. It also fails to define a set of programs where it even attempts to say much of anything useful about the behavior of a freestanding implementation (since the only possible observable behavior of a strictly conforming program on a freestanding implementation would be `while(1);`).

I would propose defining the terms "Safely Conforming Implementation" and "Selectively Conforming Program" such that feeding any SCP to any SCI, in circumstances where the translation and execution environments satisfy all requirements documented for the program and implementation, would be required not to do anything other than behave as specified, or indicate in documented fashion a refusal to do so. An implementation that does anything else when given a Selectively-Conforming Program would not be Safely Conforming, and a program which a Safely Conforming Implementation could accept without its behavior being defined thereon would not be a Selectively Conforming Program.

While it might seem awkward to have many implementations support different sets of features, determining whether a Safely Conforming Implementation supports all the features needed for a Selectively Conforming Program would be trivially easy: feed the program to the implementation and see if it accepts it.

I think there's a lot of opposition to "optional" features because of a perception that features that are only narrowly supported are failures. I would argue the opposite. If 20% of compilers are used by people who would find a feature useful, having the feature supported by that 20% of compilers, while the maintainers of the other 80% direct their effort toward things other than support for the feature, should be seen as a superior outcome to mandating that compiler writers waste time on features that won't benefit their customers.

Realistically speaking, it would be impossible to define a non-trivial set of programs that all implementations must process in useful fashion. Instead of doing that, I'd say that the question of whether an implementation can usefully process any program is a Quality of Implementation issue, provided that implementations reject all programs that they can't otherwise process in any other conforming fashion.

I don't know if you saw my question right below, since I guess I replied to the wrong post, but I'd be really interested to know how you view the purpose of the Standard. For years, the language has been caught in a catch-22 where the authors of the Standard have seen no need to have it recognize constructs that compilers have, almost unanimously, processed usefully for years without being required to do so, but some compiler maintainers interpret the failure to mandate such constructs as deprecation.

I would like to see the Standard either rewritten in such a way as to actually define (sometimes as optional features) everything necessary to make an implementation suitable for a wide range of tasks, or else expressly state that, e.g. "There are some circumstances where the behavior of some action would documented by parts of the Standard, the documentation of the implementation and execution environment, or other materials, but some other portions of the Standard would characterize those actions as invoking Undefined Behavior. This Standard expressly waives jurisdiction in such cases so as to allow implementations designed for a variety of purposes to process them in whatever fashion would best suit those purposes."

What would you think about including something like those last two sentences in the Standard, so as to help clarify its intention?

Such an interpretation would be the only way to justify the way the maintainers of clang and gcc actually behave in response to complaints about their compilers' "optimizations".

Rather than trying to come up with "compromise aliasing rules", the Standard needs to recognize that different tasks require different features, and allowing all possible optimization opportunities that would be useful for some tasks would make an implementation totally unsuitable for others.

I would suggest that the Standard define directives to demand three modes, with the proviso that a compiler may reject code which demands a mode it cannot accommodate:

1. clang/gcc mode, which would be adjusted to match the way clang and gcc actually behave, as well as anything they want to do but their interpretation of the Standard woudln't allow.

2. precise mode, which behaves as though all loads and stores of objects whose address are taken behave according to a precise memory-based abstraction

3. sequence-based mode, which would allow compilers to hoist, defer, consolidate, and eliminate loads and stores in cases where they honor data dependencies that are visible in the code sequence, but would require that compilers recognize visible dependencies which clang and gcc presently ignore, and would also require that the definition of "based on" used by "restrict" recognize that any pointer formed by adding or subtracting an integer from another pointer by recognized as "at least potentially based on" the former, even in corner cases where clang and gcc would ignore that.

Recognizing mode #1 would avoid allow clang and gcc to keep using their aliasing logic with programs that can tolerate it. Mode #2 would ensure that all programs that have trouble with that logic could have defined behavior by adding a directive demanding it. Mode #3 would allow most of the same useful optimizations as mode #1, but work with a wide range of programs that would presently require `-fno-strict-aliasing`.

If one recognizes the need for different modes, the effort required to describe all three modes would be tractable, compared to the obviously-intractable problem of reaching consensus about how one mode that would need to serve all purposes.

What makes pointer provenance really great is that clang and gcc will treat that pointers that are observed to have the same address as freely interchangeable, even if their provenance is different. Clang sometimes even goes so far with that concept that even uintptr_t comparisons won't help.

    extern int x[],y[];
    int test(int i)
    {
        y[0] = 1;
        if ((uintptr_t)(x+5) == (uintptr_t)(y+i))
            y[i] = 2;
        return y[0];
    }

If this function is invoked with i==0, it should be possible for y[0] and the return value to both be 1, or both be 2. If x has five elements, however, and y immediately follows it, clang's generated code will set y[0] to 2 and yet return 1. Cool, eh?

> When you handle unsanitized input in C on a security boundary, you must literally treat the compiler as a "lawful evil" accomplice to the attackers: you must assume that the compiler will follow the spec to the letter, but will look for any excuse to open up a gaping security hole. It's incredibly stressful if you know that fact, and incredibly dangerous if you don't.

I'd say more chaotic evil, since the Standard has many goofy and unworkable corner cases, and no compiler tries to handle them all except, sometimes, by needlessly curtailing optimizations. Consider, for example:

    int x[2];
    int test(int *restrict a, int *b)
    {
      *a = 1;
      int *p = x+(a!=b);
      *p = 2;
      return *a;
    }

The way the Standard defines "based upon", if a and b are both equal to x, then p would be based upon a (since replacing a with a pointer to a copy of x would change the value of p). Some compilers that ignore "restrict" might generate code that accommodates the possibility that a and b might both equal x, but I doubt there are any that would generally try to optimize based on the restrict qualifier, but would hold off in this case.