This was an eye opener for me. Bell Labs tech is as usual a mountain of hidden complexity hiding under a "simple" facade.
No more excuses for me. Time to learn Rust.
No more excuses for me. Time to learn Rust.
The tension between weakly typed pointers and the desire to generate efficient code is where there is a problem. More or less anywhere you violate the aliasing rules you are doing something that Fortran doesn't allow at all, and by disallowing it semantically the hope was that C programmers could have their cake and eat it too. The reality is that all systems code should probably be compiled with alias analysis disabled.
A spec of a quarter page can already be incredibly convoluted and with very hard to anticipate consequences -- even more so when crazy people are interpreting it without caring about the consequence of their acts in the real world. And C is not just about aliasing rules; the current situation is that ANY undefined behavior is a landmine waiting to kill you, regardless of whether is seems to makes sens for your target architecture. And that is mostly because compiler writer have an insane interpretation of the standard: the definition of "undefined behavior" is "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"
A key word here is "nonportable" and it is clearly not considered often enough by some compiler writers, who generally prefer to see all undefined behaviors as a licence to "optimize" your code without carrying too much about warning you about potential bad side effects because it's "hard" according to them.
This does not make even the beginning of any sense. If it is not what many programmers are expecting (a majority ? -- most coworkers I know including my direct boss are not even aware of all that mess), is costly during the translation phase, has unclear/unquantified runtime performance benefits, is dangerous in the real world and is hard to detect when bugs are activated by those "optimizations", then WHY they are doing them in the first place? From an engineering point of view this is just plain insane. Correct executions and in depth safety are extremely valuable, and only becoming more so year after year, and when they pretend that it's not their fault that programs are breaking they are being even more ridiculous; a compiler does not exist in a vacuum, and neither just to reproduce itself and satisfy the curiosity of geeks for mathematical logic.
Obviously some amount of alias analysis can be useful, and this is clearly one of the topic really intended from scratch in the standard to address some performance issues, but maybe it would be enough to explicitly identify what you want to not alias. Seeing the bug when they discuss about allowing uint8_t to not behave in general as an unsigned char in regard with aliasing is just plain disgusting and makes me lose yet again a portion of the tiny remaining trust I had in them.
It will soon get to the point that C/C++ will not be realistic languages to consider if you want any kind of reliability. Maybe I'm even deluded in thinking this is not already the case.
Somehow, the developers of gcc, clang, and various commercial compilers are all crazy, while people who work on any other project in C are sane? Why haven't the sane software developers forked an open-source compiler and implemented sane semantics?
Blaming the state of compilers on compiler developers without understanding their motivations for making engineering choices is intellectually lazy, particularly for other software developers.
I know already too much for my taste about the reasons they gave to come with such horribly risky designs. Example: "it's difficult to properly warn where we are doing dangerous optims". I'm not buying it. Don't do them in the first place or use more heuristics. Publish your data about your risk / benefit analysis. I think they made strategic mistakes. "Everybody" in the academia and security / safety related business is saying that. And no, that won't be solved with dynamic checkers. They should concentrate on optimizations minimizing the risk of new behaviors concretely appearing, and heuristics maximizing the detection of the intent of the programmer. That sounds informal as hell, because this is, but in the real world solving a problem does not always mean finding the perfect solution to an equation, and good enough approx are often the best thing to seek. Compiler writers know that. They are just not seeking the good thing. The "smartness" they think they are creating though their imaginative use of undefined behavior is as smart as my ass, because on real projects those kind of sufficiently smart compilers are indistinguishable from an adversary.
I have recently implemented some kind of compiler, though not C to machine code. In the real world it's not easy. It's messy. It's made of blood and tears. You have to care about all kind of little details for your users. You have to make the junction between two domains that sometimes have quite different semantics -- there is never a perfect solution to that. And all of this, with the minimal risk to be misused. If you don't care for your users, their is no point. I've the greatest respect for the authors of CLang, which I used for the front end. It's not perfect, but it is quite good and it does the work. But sometimes, when you feel that somebody effort really is misplaced, you better tell him (or the community, my voice is not particularly original), and explain your reasoning. Otherwise, you effectively won't be entitled to complain latter, if you wait too long.
So now what do we have: two domains, C and various target architectures, that used to be with quite some low impedance mismatch, by design, are considered radically different. The expectations of most users have nothing to do anymore with the expectation of how the compiler writers thinks the users should write their code (or have written it in the past, for less maintained software). That won't ends well. Actually, we are already in the mess, while we don't especially needed more than we had already.
So go on and please explain your POV. But anyway, no, I won't fork and write a "friendly C" compiler overnight. Neither will maintainers of cryptography softwares, probably. I'm just yet another datapoint that will loose some time in adding all the -fdisable-insanity of the day anywhere he passes. Because somebody else somewhere else in the world thought that would be a good idea to infer proofs using rules written for the least common denominator of all computers to detect if you read the n1570 over and over again enough times to obtain the privilege to get a sane translation of your code that actually will anyway only ever run on x64_86, thank you very much.
The same debacle somehow happened about memory models (see how they have been received for kernel work). If you only ever care about the abstract, concrete real prospective users won't greet you as you might have expected. Rightly so. Especially when your formalism has been proven unimplementable and unsound.
Secondly the way people shop for compilers is completely broken. It's often nearly entirely done by running the candidate compilers against a suite of benchmarks and picking the one with the best numbers. Usually only a single -O option is passed. Disabling any of these "optimizations" by default means you lose.
Thirdly I think you are overestimating the number of people that care about this problem. John Regehr ran a bunch of crypto code through clang's undefined behavior detection tools and reported bugs. Many of those bugs were closed with a WONTFIX because the code generated today is correct!
Lastly, the ISO spec is what it is, and has what it has. To the extent that compilers are in competition with each other, a single vendor working on a separate dialect of C is spending effort that helps it not at all in the benchmark game, and which over 90% of their users will never use. GCC being the 800lb. gorilla it is, occasionally users of other compilers request language extensions from GCC, but most often their checkboxes are "Complies with the ISO standard" and "generates good code" I would love for the next version to have a more friendly C dialect as an annex, and then everyone who cares could just pass --std=c17friendly and it would work across all compilers.
[edit]
As far as my original post being very meta, that's mainly because none of what I'm saying above has anything to do with my primary complaint: people who aren't writing compilers are complaining about decisions made by those who are, and attributing it to something fundamentally wrong with those people. Compiler developers aren't insane or stupid. Since most C compilers are self-hosting, compiler developers use their product more than most other developers. Saying they have tunnel-vision is perhaps a bit more close to the truth, but it is worth examining the environment they are in, as it's not like there is a "will be a compiler developer" gene that selects for compiler writing skill along with "doesn't give a shit about users"
And most of this isn't directly pointed at you, but rather the fact that there's been an unusual amount of compiler-writer hate on HN recently. I thought I could let it go by, but I hit my limit.
[edit2]
To sum up my first edit; all of my "meta" post I feel like I could have said even if I didn't know a lot of compiler developers. I don't really know any web developers well, but I don't assume that the 3MB of JS that is loaded for tracking and advertisement on most large sites is because web developers are insane or stupid. I assume rather that at least some of them are trying to deliver the best product within the constraints of their customers (a page that can be monetized, and completed in under a certain number of billable hours).
The root cause is the ISO C standard itself, so if you want any change in that direction, the best approach would be to join the ISO C working group and make proposals to replace various undesirable undefined behaviors with, at the minimum, implementation-defined behaviors (starting with signed integer overflow, perhaps). Probably there aren't any vendors of one's complement machines and the like left to veto your proposals.
The C11/C++11 memory models for the most part don't solve a problem that kernel developers have, because all kernels already contain tested solutions to the same problem, so one would not expect kernels to quickly exchange all their tested and highly performant concurrency primitives. The only benefit for them is that the memory model effectively prohibits implementations from doing some optimizations that would be valid only under the assumption that there is only one thread of execution, and this benefit is implicit, i.e. you don't need to use any of the new language features or standard library headers to benefit.
The standard library does assume this, but it's not an inherent language issue. Most people who are in resource constrained environments won't be using the standard library anyway.
The reason Rust's standard library isn't safe against malloc failure is that propagating Error everywhere would be cumbersome; I distinctly recall rust mailing list discussions on this point.
The only reason handling allocation failure has to be difficult is that the Rust people put themselves into giving into the anti-exception people. Once you allow exceptions, allocation failure becomes as easy to address as any other kind of resource exhaustion.
The Rust designers made a serious error eschewing the only fully general and fully ergonomic error handling strategy we've found.
> the Rust people put themselves into giving into the
> anti-exception people
> That the documentation suggests not using it as "catch"
> is merely a political statement