I've honestly never heard that before. If you'd said "a good C compiler will generate code faster than someone writing assembly," then I'd agree... ;)
I've honestly never heard that before. If you'd said "a good C compiler will generate code faster than someone writing assembly," then I'd agree... ;)
No idea. But I've done some assembly myself (as a console game programmer) and generally speaking, you don't have to think about any of the issues you raised to get faster code than what the compiler creates. There are some corner cases occasionally, but not very often. Does any of this mean I tend to choose assembly? No way.
> the compiler does these really quickly for you
It sounds like you generally agree with what I said? ;)
That said, there are reasons that compilers often make slower code than humans, reasons that may always exist. The primary reason to use a compiler isn't to get faster code, it's to use a programming language abstraction and get "fast enough" code. The abstractions are (generally speaking) what add layers that are hard for a compiler to optimize against a person who's not using abstractions. There's a long list of commonly used features that compilers have a hard time optimizing: virtual functions, the threat of aliasing, floating point pipelining, etc. etc.. Compilers are required to make difference choices than humans writing assembly would, because of the abstractions that we choose to use in higher level languages.
Lots of people agree that it's not hard to make assembly faster than an optimizing C compiler.
http://stackoverflow.com/questions/577554/when-is-assembler-...
The generated code? "mov ax, 255; ret"
That was pretty good optimization, seeing completely through the abstraction and flattening it to the simplest possible implementation. I was impressed.
Anyway there're lots of further abstractions these days, harder to see through. I imagine a clever human, with domain knowledge, can write a loop to do what a high-level library used to do.
Inlining code, especially with multiple passes, is something humans don't do. I still modularize in assembly and write functions, factor code used in multiple places into a single spot. The compiler is free to expand the same function inline a thousand places, and then use the expanded code to collapse to fewer instructions.
The flattening can definitely be cool to see. So in your example, what would you have done if you'd written the assembly yourself? Do you feel like the compiler did something you wouldn't have?
But if you're not aware of what the compiler is emitting, it's often not hard to pull an order of magnitude improvement out of the code.
Instruction scheduling is not a problem on modern cpu's. Not that it's hard either way, just reorder things around memory access and heavy instructions like division. Agner Fog made a nice table of instruction latencies and throughputs [1].
Instruction encoding length... do you use 16bit integers or do you just int everything ? The compiler has to do what you tell it to do. Sure there are plenty of tricks around that but those are not the ones that give the compiler a remarkable advantage over humans. Tricks like using bitwise operations instead of multiplication and such and shortening equations is the advantage that the compiler has. But even that is not that big of a deal. (again, do you use the restrict keyword ? or static/define for constants ? do you know that C treats every "compile unit"(.c file) as a separate program ?)
[0] https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=g...
- Most performance-critical code is regular enough to stay far away from the NP-hard corners of register allocation. In the rare instances when I need to do careful hand-allocation to get an inner loop to fit perfectly, the compiler usually gives up and spills several variables.
- Performance-critical code is rarely latency bound, and modern processors are wildly out-of-order, so scheduling based on latency is usually a waste of time (and sometimes a pessimization). Minimizing the critical path length of loop-carried dependencies and balancing port utilization are far more important factors in most code sequences.
When code is latency-bound, it's often due to bad design decisions further up the stack; refactoring those to shift the concern to throughput is vastly more profitable than anything you can do to reduce latency locally, and is the sort of optimization that a compiler simply can't do (and usually can't even tell you you should do). I can tell the guy calling my BLAS library that he should try to batch up many vectors and call GEMM instead of GEMV. The compiler usually doesn't do that, and no-matter how hard it works at optimizing GEMV, it's going to be bound by memory access.
- Encoding length is a factor for whole-program optimization (reducing i-cache pressure), but critical sections often fit into the decoded µop cache / loop buffer / inner cache / etc making this a non-issue. Sometimes you want to choose a particular encoding to optimize for e.g. fetch group alignment or to separate branches to aid the predictor, but there you're usually not trying to minimize the length but to get alignment without no-ops instead (and compilers generally don't do this optimization, because they don't have alignment/instruction size information until really late in the process).
- Avoiding pipeline hazards and getting inlining right are are both worthwhile, but really, really straightforward for the most part. (Inlining is hard to get right if you're trying to avoid wasted work in the name of compiling as fast as you can, but humans don't have the restriction).
The compiler works a lot faster than I do, but I'm a lot better at balancing these and other considerations and methodically optimizing a non-trivial code sequence until it's nearly optimal. I might take a week to optimize the same function that the compiler optimizes in a millisecond, but my code is faster. For most code, the compiler version is fine. For heavily-used system library functions, you definitely want my version instead.
How can we improve compilers so that they're as good as I am? Compilers today mainly work in terms of local transforms on a computation graph, which necessarily cannot capture long-range optimizations that are available to me; most expert assembly programmers I know work mainly in terms of a global resource utilization minimization approach. There's no real reason why we can't make compilers that work that way, but it will almost surely make them significantly slower, so it would be more of a super-optimizer mode that you would only want to run once.
I can also make inferences about what the programmer was "really" trying to do, and go down the hall to ask him. The compiler doesn't (usually) do that; there's a lot of opportunity for compilers to say "hey, I noticed that if x always has this common property, I can make your code much faster -- is that a valid assumption here?"
Compilers aren't bad, but there's a lot of room for improvement, still.
Essentially you also lose they advantage of runtime optimization Compare plain java compiler with a java JIT compiler.
It's a good fit for low-power single-issue architectures and DSPs, because ... well, there's only trivial scheduling to be done because the machine is so dead simple. It's a good fit for GPUs because they are primarily throughput oriented, and can make scheduling issues "go away" by running lots of threads / warps / whatever marketing wants to call them to hide latency.
Old Intel Atoms also used to lack OoO execution.
The second biggest problem is that cache residency is often not statically predictable, which means that you generally have no idea in what order the various memory dependencies will actually be satisfied, which makes all your effort on scheduling approximately useless.
Those older Atoms were pretty universally terrible, by the way.
Agreed.