Test results for Knights Landing
agner.org
agner.org
Those two, I think, go hand in hand: the point is not using unused execution resources with the additional hyperthreads (thus the decode limitation), but using the other threads to fill the pipeline bubbles created by the higher latency instructions, a-la barrel processor.
edit: this of course works as KNL is aimed at throughput jobs, not anything latency sensitive.
The advantage of its microarchitecture is that if you use the hyperthreads correctly, it will run a broad range of code at close to the theoretical IPC of the silicon without too much code effort (if you understand the model). This is in contrast to CPUs, which rarely get close to their theoretical IPC no matter what you do, or GPUs, which can only run a very narrow range of codes at close to theoretical IPC. In principle it is extremely efficient in terms of operation throughput, and it isn't sensitive to what those operations are, but it requires a large number of independent operations to be in flight to do its magic hence all the hypertheads.
While the architecture explicitly uses latency-hiding to get throughput, it is mostly about latency-hiding at the sub-microsecond level. As a practical matter, it shouldn't affect the perceived latency of most real-world software.
The design is supposed to be "balanced", and it does appear to do a reasonable job of that for the sort of code that uses the bulk of the time on our system. In some cases, Broadwell will do substantially better, of course. I don't have URLs for performance examples to hand.
People mostly seem to be ignoring the potentially important additions in KNL -- the memory system and the built-in interconnect (though I don't know if the latter is available yet). Also the large core count should help by keeping more MPI communication local. I don't know of any relevant results for multi-node jobs, but 64 cores covers a fair number of the HPC jobs I see.
Although he dismisses it, he actually mentions the reason: 'perhaps [it is useful] for code that is limited by memory access, branch mispredictions, or long dependency chains': especially the first and the last are going to be common on code running on KNL.
It strikes me as insane that you are not making it easier for Agner to get early access to your hardware. He's writing fabulous manuals for free that fill the gaps in the official sources. His work helps more programmers to get the most out of your hardware than just about anything else available. You should be throwing money at him to keep him doing this, or at least showering him offers of free pre-release hardware. But instead you are depending on his generosity and leaving it to chance whether he eventually finds a machine to test on.
Why?
a lot of companies tend towards that behavior. a lot of single developers tend towards that behavior. I don't like it because it represents a kind of destructive egotism.
hopefully that's not what it is and it's just Intel not thinking too hard.
You get what you measure for. Intel measures a lot. Intel fails to measure some important things.
The same is probably true of any company with hyper-focused management. Maybe even your startup.
At this point Intel is betting that anyone who's capable of doing large scale low level optimization will be designing their own hardware including the CPU so they'll better focus on high performance computing for the masses.
Interoperability with x86 big cores is also what intel wants because it means that the software can run on anything, even GPU based HPC efforts want X86 compatibility this is why AMD and Intel dragged NVIDIA to court a couple years back.
Abstracting this away to the compiler is a safe bet, and developers will soon learn ways to at least generally optimise the code for the compiler, much like people have already learned to optimise code for javac, v8, etc.
It doesn't really require any new compiler breakthrough, as it is pretty much what all the GPU compilers (i.e. cuda) have been doing for a while already.
The central lesson of RISC is not "fewer, simpler instructions is good", it is "let the compiler do what it does well, and let the hardware do what it does well." The increase in available silicon has been moving that boundary for many years. Ever since we finally had enough silicon to implement out-of-order execution with synchronous exceptions, for example, register allocation is no longer a problem to be solved in the compiler.
Classic RISC only made sense when available bandwidth to main memory was at rough parity with on-die memory bandwidth. Those days were brief. But the idea of intelligently dividing optimization tasks between the CPU and the compiler is a timeless idea. The optimal implementation, however, is a function of available technology on both sides.
When you go hand-optimizing, you want to squeeze the last bit of performance. And that (currently) is very CPU-specific. Because all CPUs have different cache/memory system, instruction latency, and pipeline structure. When I was doing low-level work (for video processing), a lot of time I has to strike a balance across various microarchicture. Some more extreme people (including the code by Intel ICC) target each CPU model specifically.
Because the MIC has vastly different architecture, the resulting hand-optimized code will be vastly different. You still need to learn both of them anyway.