Should CPUs include more masking instructions for regular, non-vectorized code as well?
Should CPUs include more masking instructions for regular, non-vectorized code as well?
Also, you can get something very similar even for scalar code as long as your machine has a conditional move operation, which they all have:
/* true path */
a = ...
b = ...
true_result = ...
/* false path */
x = ...
y = ...
false_result = ...
/* final result */
result = (condition ? true_result : false_result)
This works if the two "branches" have no side effects (memory writes, function calls) and cannot raise exceptions. Again, it's very difficult to estimate for compilers (and humans) if it will pay off to run both computations rather than branch.The trade-offs for vectorization are different from scalar code since vectorizing a loop by a factor N gives a huge win, and even if you waste some time doing some redundant computations, you still have a reasonable chance of being faster than scalar.
Gcc will not under any circumstances produce two cmov instructions in a basic block, so that is your only alternative without dropping to asm. Clang is happy to produce two adjacent cmov instructions. Usually your ALUs are not otherwise so engaged as to make the number of operations involved costly.
int select(int x, int y, bool condition) {
...
}https://news.ycombinator.com/item?id=23237663
The bitwise thing is correctly -c&x|(c-1)&y.
*edit: wording
The problem in question is due to predictive execution in which the CPU has to flush the pipeline on misprediction. This is dominant in desktop PC CPUs. I have no knowledge on what is prevalent in e.g. server or mobile CPUs.
Both are (according to https://en.wikipedia.org/wiki/Speculative_execution) forms of speculative execution.
I'm not sure if eager evaluation is related to the topic at hand and was speaking specifically about speculative execution. Eager evaluation is just what pretty much every language except Haskell (or Haskell-like) does.
The point is that there are at least two ways CPUs can speculatively execute a conditional: Either predict which branch is taken and flush the pipeline if the prediction was wrong, or execute both arms of the conditional and discard the one that was not actually taken. The top-level SIMD comment is about the latter, but most optimization around speculative execution is for the former.
Yes, CPUs do speculative execution. The "flush pipeline on mispredict" kind ("predictive execution"). That is not the same as the "execute both and discard the untaken one" kind ("eager execution"). Your first reply suggests you consider them equivalent when they are not. I just wanted to clear that up.