Breaking all the Eggs in C++
scottmeyers.blogspot.com
scottmeyers.blogspot.com
EDIT:
For examples, I'd love to have -Wold-style-cast on all of my compilers. The nullptr example in the article is good too, maybe call it -Wold-literal-null? It would be nice if my compilers could give me hints when std::move might be appropriate, if I might be missing a copy or move ctor, or when I could consider using initialization instead of assignment for members. Maybe this is more of a job for the static analyzer and not the compiler--dunno.
Like this?
"The rules are designed to be supported by an analysis tool. Violations of rules will be flagged with references (or links) to the relevant rule. We do not expect you to memorize all the rules before trying to write code."
Introduced in this year's cppcon https://channel9.msdn.com/Events/CPP/CppCon-2015/Writing-Goo...
Another great, modern C++ feature: `constexpr` functions, methods and compile-time constants have the potential to put most, but not all, macros out-of-business... Macros are powerful for DRYing C/C++ similar code sections which cannot be expressed with templates, lambdas or function pointers, for macro pasting identifiers together and for project configuration management for (ie, kernel hardware support). Defines also don't occupy space but also lose semantic value in debugging because they're computed at compile time and spread out wherever they're used... debuggers encountering `constexpr`s should offer both the computed value and a way to debug or debug (not program code at all) how that value was constructed: gdb and lldb should be able to handle that with plugins.
Here's also how we compile most our code, using both gcc and clang, on both stable and head:
clang++ -std=c++1[14z] -Weverything -Werror -Wno-c++98-compat -Wno-weak-vtables ...
g++ -std=c++1[14z] -Wpedantic -Wextra -Werror -Wformat=2 ...
I would also really like to see C++1[7z] concepts implemented by major compilers, because this will make our code a whole lot simpler by reduce the hacks for runtime and compile-time type checking, and make generic programming much cleaner by being able to constrain requirements via arbitrary bool constexprs.const isn't the same as immutable. 'const' implies a (mostly) read-only view, not read-only data. That is, C++ does not guarantee that the memory does not change, just that some parts of the code will not change it (and even then, there are ways to get around 'const').
It would be marginally better to be const by default, because there would be less code to consider when analyzing changes to data, but the cost of this feature is much larger than the benefits we'd see from it.
And, to top it all off, dealing with a lot of immutables really requires different syntax and different style. Read through the rust book and then try to do the examples in C++. You can do pattern matching and variant types in C++, but they are ugly and probably slower than what we do now.
Clang-tidy: http://clang.llvm.org/extra/clang-tidy/
Cppcheck: http://cppcheck.sourceforge.net/
Flint++: https://github.com/L2Program/FlintPlusPlus
Clazy: http://www.kdab.com/use-static-analysis-improve-performance/
Ideally, you'd add these as feature flags to the compiler. So me and my team can decide "0 is not a pointer in our code", so we compile with -f0-is-not-a-ptr and with -fdefault-initialiser-0. You can already choose if you want exceptions or not, if you want RTTI or not, if you want floating-point math to be optimised as if you're using real numbers (which can lead to subtle numeric instability) - or if you want the compiler to emit exactly what you've written, and so on and so forth.
Adding more safety options which we can turn on for our code would help us write safer code. We can measure the impact of initialise-with-0 for our case and decide whether to use it or not.
Talking about a magic wand that makes source-to-source changes on a 100MLOC codebase trivial is pure fantasy - requiring you to ignore that even in the best version control systems, the resulting ~10MLOC merge is going to cause a lot of merge conflicts (some of which will be incorrectly resolved) and history clutter. To say nothing of all the poor bastards who are satisfied with the occasional tarball per release. Who are still doing better than the people who can't be bothered to backup the code they wrote directly on the server it will run on. And nevermind the fact that at least some of your downstream customers are still using some old compiler that they're stuck with - the only option for some legacy embedded platform they can't eliminate yet, made by that company that went out of business.
And maybe that kind of cost would be worth it - for the right reasons, given the right motivations. But I feel like I'm looking at a laundry list of mostly minor, tiny issues, that are a "solved problem" in the form of existing warnings. I can crank them up to errors as I modernize a codebase. You can enable them by default in your build systems and IDEs. You can mandate them across your organization, and in your contracts. You can incrementally apply them per library as you get around to doing more maintenance on them.
C++ has atrocious defaults, where you have to add a bunch of keywords and do all these things just to make the simplest classes do the "right" thing. It's long overdue for some new behaviors.
Adding things like "override" to existing code has been an absolute joy. It has already prevented errors during maintenance in projects at work. It is well worth a few days of ugly revision control logs.
There are lots of ways to smooth a backward-incompatible transition for C++ on a larger scale. For instance, they could enable it on a file-by-file basis by having some new declaration like "using c++17;" (or better still, require only the old, out-of-date files to add this declaration in order to continue compiling).
https://www.python.org/dev/peps/pep-0214/
Where he basically said "Maybe print should be a function, but... nah".
Two years later, he says he regrets it:
http://legacy.python.org/doc/essays/ppt/regrets/PythonRegret...
And the benefits are explained here:
(Sorry, trolling. ML-style function call syntax would clash a lot with other parts of Python.)
def g(): return 5
assert g() == 5
assert g != 5
Without parenthesis, how can the language distinguish the last two lines? How does ML do it? Would the same trick work in Python? I know Ruby has optional parenthesis, but functions aren't quite as first class there as they are in Python, which is why the have Proc and whatnot.Look here: http://try.ocamlpro.com/
Try this code:
let g() = 5 g() == 5 g != 5
First line defines the function. Second line should evaluate to true. Third line doesn't type check because a function is not the same type as a literal.
Haskell does differentiate between IO-actions and functions. Here is an example:
main = do
let printFive = print 5
print 1
printFive
print 6
printFive
Will print 1
5
6
5
The type systems helps enormously keeping things straight here, but the basic concept would work in a non-statically typed language, too.How? How do I compare. say, two functions for equality, without comparing the return value instead?
The decoupling of function calling and IO actions can work in non-statically typed languages, too.
Your question concerns a different topic. To give something like an answer: that problem can only occur with `functions' of zero arguments. Haskell sidesteps the issue by requiring all functions to have exactly one argument.
Second of all it's such a silly syntactic difference that if people are honestly bothered enough by it to stay on python 2 i just lost all hope for that community. Besides, the reason for changing it was not vanity with syntax, it was to move it from a language keyword into a standard function so it could be used in all the ways described above, marvys links describe this better.
Since pretty much everything has at least one print statement, its removal ensured that everything had to be converted.
If Guido&co had tried to maintain backwards compatibility, they could have more easily made the important semantic changes (like unicode strings). Users would have been enthusiastic to upgrade, instead of angry that their time and money was being wasted on irrelevant syntax details.
People know that global variables are a bad thing but fail to realize that print "foo" is essentially equivalent to global_variable_output_buffer+="foo". It simply shouldn't be scattered around everywhere.
This is not just academic nonsense, I was once given the task to parallelize the code in a large python code base, and to send the output in an email instead of printing to console. It would have been an easy job but a lot of the code was using print just the way you describe it causing intermingled output in the console when run in parallel.
I'm not aware of warnings for uninitialized variables in constructors. Can you clue me in how to get these on in Visual Studio? I'm using a third party tool that can detect uninitialized scalars only and would love to have the extra scanning.
There are third party tools - such as cppcheck - which should catch uninitialized members however.
I generally also compile with clang, which catches a lot of additional stuff for me.
And here's a random John Carmack link noting some of the static analysis tools out there:
http://www.gamasutra.com/view/news/128836/InDepth_Static_Cod...
I've been playing around with clang's Address Sanitizer all day as well, catching buffer overflows and underflows left and right in a janky old codebase I was suddenly thrust into recently...
Thanks for /SDL! This looks like it may be very promising: http://blogs.microsoft.com/cybertrust/2012/06/06/warnings-sd...
At the very least that would help remove random behavior at the cost of speed (if I understand correctly). I'll have to sleep on /SDL and do some perf testing come Monday. I had really wanted to use /analyze (https://msdn.microsoft.com/en-us/library/d3bbz7tz%28v=VS.100...) but it crashes! D:
Unfortunately, I'm rooted in Windows SDK and I don't think clang supports that. The Microsoft Application Verifier has some run-time memory checks, so I'm not totally unarmed.
I looked at PC-lint doesn't have complete support for C++11 yet and I'm worried about the rate of false positives. I had not heard of cppcheck, so I'll have to give that go since it looks free.
Thanks for the info!
Actually, I have a slightly unrelated question: what are the issues that affect 100MLOC codebases that aren't so important for smaller projects? This is something people sometimes bring up, but I'm not too sure what exactly they mean by it.
Unless there is a very good reason that convinces the business sponsors that it's OK to spend a few man years on it. Switching to a shinier version of the programming language usually isn't.
Getting ~100KLOC of C++ compiling on a "new" compiler has been something that's taken me a few days before. Not too unreasonable.
100MLOC is literally 1000x the size - which extrapolating from "a few days" would mean it'd take me 8-14 years to fix up on my own were I to do it by hand. Or more realistically, the pace of change/growth on a codebase of that size probably outstrips my ability to convert it.
So now even a simple straightforward task such as fixing up technically nonstandard C++ syntax to standard C++ that multiple compilers will happily accept, is a task requiring either automated code modifications (scary!) or a large coordinated team effort. If 100 people working together are, optimistically, 100x as efficient as me alone - i.e. there's no overhead in terms of coordination and planning - it will still take them around a month to do the job via brute force.
;)
Yeah, easy to say in hindsight, but we won't figure out how to budget better or call out bad business plans if we don't start recognizing it when it happens.
So the perspective you might be missing is that there absolutely are broken or useless parts of the C++ standard. There has to be a way to fix those if we want:
1. Continued improvements to the C++ language
2. Compilers that aren't needlessly complex to implement and support
3. Fewer weird edge cases and therefore fewer bugs when people miss them.
4. A language that can be taught to you instead of drilled into you.
A case in point is trigraphs. They are an almost entirely useless feature that requires an extra compiler pass. They cause real bugs that confuse and surprise all but the most grizzled C++ devs (myMap["crazyUserName!!!??!?!"]). It was like moving heaven and Earth to get them slated for removal in C++17.
It is not acceptable that things like this are this hard to get deprecated and removed. There needs to be an accepted, uncontroversial way to identify cruft, mark it deprecated, and eventually remove it from the language.
So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones.
Not only is it acceptable, it's preferable. What's not acceptable is when it's IMPOSSIBLE to get things deprecated and removed.
I'm not missing it. I feel C++ is a broken enough language that I generally prefer to use others. That said, C++ is my day job, so I get to feel the burn and wish it weren't so broken.
As a result, I'm all for getting rid of useless parts of the C++ language. But backwards compatibility is not useless! I'm willing to sacrifice minor, "theoretical" backwards compatibility for relatively minor reasons - I don't think I've seen any code broken by >> being interpreted differently in C++11 in templates, for example. I only know of one case where trigraphs were used intentionally - and compilers have been disabling them by default and warning about them for quite some time now (to the point where I don't really care if the C++ standard is fixed, because C++ in the wild is fixed.)
But getting rid of NULL and 0 as pointers? Breaking overrides missing the override keyword? Every C++ codebase I know will break massively. It'll make C++ification of a C codebase even more of a massive pain in the ass. Pass. Make them enabled warnings by default instead. Enable warnings as errors when you can. People are already doing this for you: http://codepad.org/R5XyUby2.
--------------------------------
RE 1: Continued improvements to the C++ language are occuring. C++11, C++14, and C++17 are all introducing new stuff, and providing better options for securing your code. This does not require breaking backwards compatability. Unfortunately this goes against points 2, 3, and 4, for reasons that have nothing to do with what's been listed here.
RE 2: I don't see anything in this list that would help significantly there. If anything, mandating more diagnostics about override keywords will complicate the compiler further.
RE 3: So lets add some sequence points that make parameter evaluation order defined. Lets get rid of those type aliasing rules that every large codebase runs afoul of. Lets do some high impact stuff that will catch some really nasty edge cases that, realistically, will fix more codebases than it breaks.
...but none of that seem to be on this list.
RE 4: A lost cause that's getting worse. Adding a bunch of caveats that "oh, except in C++2x you can't do this thing that's in decades worth of C++ tutorials, examples, and codebases - including most of what you've seen online" isn't going to make the teaching problem any easier. It's going to make it worse.
----------------------------------
"So... why the minor, tiny issues? Because we're not ready for the big ones yet. Let's get some minor issues under our belts and then we'll figure out how to deal with bigger ones."
We've already got lots of experience under our belts with fixing minor issues. auto, >>, template exports, throw() statements - all the way back to the very first standard, which eliminated implicit conversions from int -> enum, void* -> t* , and for loop scoping - with all the breaking of C or prestandard C++ that this entailed.
What will it take for us to be "ready" to deal with things like parameter evaluation order being unspecified - and thus at the mercy of your compiler version, flags, and phase of the moon? Fixing that would break no code that wasn't already broken, and eliminate a massive source of unspecified weirdness that I still struggle to explain to other C++ developers who have been coding for years.
I still have to carefully explain the difference between operator precedence (how a() - b() - c() is parsed), and sequence points (which define the evaluation order of a(), b(), and c() - or more accurately don't). They point to && and ||, and I explain how they - along with "," and "?:" - are special cased by the C++ standard to define additional sequence points that just happen to match the operator precedence rules - unless you overload them, so never do that - and by the end of it, I've either blown their minds or they still don't quite believe me.
Showing them how std::cout << i++ << i++ << i++ << "\n"; changes behavior depending on compiler flags tends to help - although that's cheating as there's additional unspecified (or undefined?) behavior for modifying the same integral multiple times without an intervening sequence point...
[1] https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...
https://channel9.msdn.com/Events/CPP/CppCon-2015/Writing-Goo...
static checking of the core guidelines is the entire reason they are writing them. This isn't just a nice literary exercise
The static checker is found here (obviously still under development) https://github.com/Microsoft/GSL
In the beginning of stackoverflow all of its content was up-to-date but now that it is running for a few years a lot of stuff is out of date.
If you google for "c++ pointer 0 or NULL" you get this page from 2008 as the top result: http://stackoverflow.com/questions/176989/do-you-use-null-or...
With C++11/17 these answers are now out of date but this pagr will continue to be at the top of google for the forseeable future.
And exactly the same happens for all other technical stuff as well... I learn Javascript at the moment and at least 50% of all solutions I find on stackoverflow and other pages are heavily out of date.
And this problem will get worse in the next 10 years. Perhaps it is time to go back to curated link directories?
(I hope it is clear what I want to say, I am on mobile and busy with other stuff)
The solution that feels right is adding a new answer rather than stomping on somebody else's, but it seems unlikely that the new answer will get a lot of upvotes and even less likely that the asker will come back and reassign the checkmark. So a lot of more current answers are probably abandoned before they're even posted, and those that are posted end up buried (like the correct answer to that question, which is posted there but is currently languishing at 4 upvotes).
Personally, I say that if you feel like an answer is outdated, you should either comment with a polite correction if you'd like the author to fix it or just edit in the updated information yourself. (This nets less karma than answering new questions, but I can say from personal experience that 150k karma and a dollar will buy you a pack of gum.)
I had the impression that more often than not such questions also contain a recently added or edited answer, upvoted and all, pointing to the C++11 of C++14 answer. Also note that the accepted&highest vote and second highest vote answer to the question in your link both mention C++11 nullptr.
The relevance of a page at StackOverflow is less relevant if it is 5years old than if it is 1 year old, even if the page rank based on links is higher. I'm not sure if/how Google implements this, but a good heuristic would be to classify content as modern tech and have a page rank decay based on age.
For an even more advanced implementation, very "new topics" should decay even faster. Articles about Windows 10 or the Rust programming language should have a half life of 6 months now, while older topics can have longer tails.
When I google for a rust term now, I'd rather hit a 6 month old blogpost than a 2 year old SO question, ranks being equal. With C# it's the other way around. The only difference between the two is that the search trend history shows that we are on year 15 of C# searches so the difference is small compared to the age of the technology.
1. What about templates? If there's a "template<T> void foo(x &T)", and it does "x++" inside, and I sometimes use it on int, sometimes on bool, how would the magic wand upgrade only the bool cases to "x = true"? (And please don't tell me "then you have borked codebase, u go fix it, man!" In C++, you have codebase you have, sometimes decades of legacy code you hope to upgrade bit by bit "when time, boss and priorities permit".)
Similarly, if the template does "x = 0;", how could the magic wand upgrade only the cases where T is a pointer, to "x = nullptr;"?
2. One more important, yet not mentioned feature of the magic wand tool, would be that it'd have to be flawless! If the tool has bugs, it could break my codebase without me knowing, till it's too late. Especially tricky here is how would the tool behave on code with undefined behavior; after years of working with C++, I have a nagging suspicion that virtually every sizable C++ codebase does actually have some undefined behavior lurking somewhere.
You are being too optimistic. Any C++ code base over at most a couple hundred very carefully crafted lines will have undefined behaviour and implementation specified behaviour.
Generate a new template specialization or overload? I realize this could be rather questionable if the offending operator use is somewhere in the middle of other code.
Break all the eggs! C++ could be so much more effective if the backwards compatibility requirement could be overridden on the most basic of stuff that's still part of the language even though consensus, i.e., best practice, was to drop it out years ago.
That said, it's not an insurmountable problem. Usually I'm looking at a particular line, and if history says it was touched in a compiler cleanup then I just have to click a few more times to get 'svn blame' of the version preceding the cleanup.
For NULL, 0, and nullptr, gcc and clang already treat NULL as a magic __null value and -Wconversion-null (enabled by default) can warn about using NULL as a non-pointer value. gcc also can break 0-as-a-null-pointer with the -Wzero-as-null-pointer-constant flag. I had hoped to fix these warnings in Firefox (https://bugzilla.mozilla.org/show_bug.cgi?id=777515) bug there were thousands of warnings because gcc warns about using NULL, not just 0.
For override, gcc has -Wsuggest-override and -Woverloaded-virtual warnings that may help here. clang has -Winconsistent-missing-override.
For uninitialized variables, gcc and clang have -Wuninitialized. However, gcc -Wuninitialized has historically reported a lot of false positives. Fortunately, recent gcc version moved the flakier uninitialized warnings to a new -Wmaybe-uninitialized flag. In my experience with Firefox, -Wmaybe-uninitialized warnings are false positives about 90% of the time. On the other hand, clang has a -Wsometimes-uninitialized which is reliable.
For bool++, clang has -Wdeprecated-increment-bool.
btw, clang does not document its warning flags, but some resources I've used are the http://fuckingclangwarnings.com/ reference, searching through clang's source code at https://github.com/llvm-mirror/clang/blob/master/include/cla..., and clang's -Weverything flag (which truly is everything, including contradictory warnings).
I cannot for the life of me help but think that this is simply a distinction without a difference.
I work in the embedded space, and in each case when I've used C++ instead of 'C', me and at least one other senior ( and I mean age senior - +40 ) guy end up bemoaning that if we'd just had a 2-hour meeting on which subset of 'C' we'd have used, it would have taken half the time.
Oh so obviously, everybody has a different experience, and celebrate the diversity and all that, but I am really happy that I can always fall back on good old 'C' and get something done.
EDIT: s/invert/invest
But in general "big C++ in embedded" kind of went out with the dotcom crash.
Very few new projects are written in C++. It's the old products with 1M+ LOC codebases that most C++ devs live in.
So code that relies on a lot of existing software then (ie the game engine and other in-house libraries that any company would have). You have proved his point instead of countering it.
I don't know if the JVM is better or worse for this than some other runtime like Go's or the CLR or whatever, I just wrote about the JVM because it is the one that I have used in this context.
1. There is a lot of expert tuning to get that sort of performance with respect to latency. That tuning would need to be reproduced for every target system.
2. Those trading systems can have ridiculous amounts of RAM to give the JVM GC lots of room to work with.
3. Reading packets from a wire and writing other packets to a wire is a different problem than reading controller inputs and modelling complex 3D worlds.
...that's not to say you can't have AAA Java games. To some degree that already happens with Android games, at least if you're willing to stretch the definition of AAA. (If you define AAA as "uses C++", the nomenclature becomes fairly moot).
My response is that we use C++ for the language itself, existing engine or not. :)
There are toolchain requirements that push in this direction, but honestly I doubt they make any significant difference in the matter.
Not even fully on C++11 yet!
Vil: How's it going?
We'll hopefully be moving to compilers capable of supporting c++11 at work soon - maybe even in 2016 if we're lucky - but it'll probably be an even longer wait before we can start using c++11 features in our code. Still, a guy can dream...
Yeah, there's a bit of c++11 going on here - but until all the DCC apps move next year, as you say, it's going to be a while.
There's a huge world outside of those two categories, and there's tons of new C++ projects being written.
.NET provides all the tools to write cache friendly programs.
That many devs use it like Java is another matter.
If we include high integrity systems there is Ada as well, even though it is a niche.
For .NET, almost with a few additional tricks.
- Value types
- Struct layout attributes
- Field offsets.
- Arrays with value types
- Unsafe
- Interop with native buffers
- Buffer manipulation with custom serialisers
Your challenge can be fulfilled with getting a native buffer, with a custom serialiser and a few unsafe tricks.
As a matter of fact, you just gave me a mini project idea. Lets see if I am up for the challenge or need to eat my own words.
Meanwhile here is an example what C# devs are able to achieve, when they know how to explore the tooling properly.
http://www.infoq.com/articles/Big-Memory-Part-2
I am an old C++ dog since the C++ ARM days, meanwhile busy with other languages.
I am not being snarky, such libraries would help me a great deal.
http://www.roguewave.com/products-services/imsl-numerical-li...
http://quantalea.com/highlights/
I do concede that my remark was a bit flame-bait, but as I replied on a sibling thread, there are mechanisms available that allow for cache friendlier code, while staying inside .NET.
It is going to outperform or be shoulder-to-shoulders to an optimizing C++ compiler?
Most likely not (lets see how .NET Native evolves), but in certain use cases will be already fast enough.
The problem is that many see .NET as Java clone, as I once did as well, Microsoft and miss to explore the mechanisms .NET has available for a bit closer to the metal coding.
Many times 90% there is already a solution costumers will appreciate.
Plus there are plans to bring more of their learnings from System C# (aka M#) and project Midori into .NET.
In theory .NET can do a whole lot of things, but I have been waiting for this hole to be filled for a long time. So far there is nothing even remotely useful as Eigen, Armadillo and their ilk.
Heck, there isn't even a standard sin, cos exp log etc. for floats. Not a place where a number-cruncher will feel at home.
So the idea of:
int x = void;
is inherently wrong to me. Why not just create a new keyword with a clear and explicit meaning?
That said, breaking changes are simply wrong. C/C++ is what it is, stop trying to make it into Java, Go or whatever. Trying to change it into a language that it's not is simply going to cause confusion and chaos, and kill the language. The things Meyer points out are all correct, but just because it's right doesn't mean it's right for the C++ community.
Every language has it's own pluses and minuses and thinking you can squash all problems in all languages and make it "perfect" is a losing battle.
Ummm... What is your suggestion for a new keyword that would not be a breaking change?
Plus, I think the idea of putting each keyword behind an include gets annoying if you do this for a significant fraction of your keywords. (It's neat that you can, but... should you?) For instance, imagine if throw, class, and protected were all behind separate includes?
int x = zeroinitializer;
void *ptr = zeroinitializer;
int y = undefined; // explicitly undefined
int z; // compile error
int w = poison; // compiler error if w or &w is read from
Additionally, I am not fond of C and C++ casts and their semantics which vary with the types being used (and can get really hairy when templates are added to the mix). I'd much rather see "casts" that are named after the semantics they use. E.g. uint16_t u16 = 1234;
uint32_t u32 = zero_extend(u16);
int16_t s16 = -342;
int32_t s32 = sign_extend(s16);
uint16_t bits = bitcast(s16); // must be same size
float fp = 1.234;
int32_t fpint = floattoint(fp);
uint32_t fpbits = bitcast(fp);
In the two examples above, all the semantics are explicit, there are no surprises at runtime and the compiler/static analyzer has much more power in catching problems.This, of course, isn't going to happen in C++ in any kind of backwards compatible way. I'm not sure these things can or should be fixed in C++, because of the amount of existing code and the fact that most projects would never upgrade because of the manual effort required in updating the code and merging it.