GPUs do not actually have control flow.
They are SIMD vector processors (Same Instructions Multiple Data).
CUDA and OpenCL make this a bit more explicit.
They are SIMD vector processors (Same Instructions Multiple Data).
CUDA and OpenCL make this a bit more explicit.
Wrong. Let's look at the instriction set of the AMD GCN3 ISA:
> http://gpuopen.com/compute-product/amd-gcn3-isa-architecture...
which links to
> http://32ipi028l5q82yhj72224m8j.wpengine.netdna-cdn.com/wp-c...
which even has a whole chapter about control flow: "Chapter 4 Program Control Flow".
I'd say it's more like "not quite". ~10 years ago, the statement was pretty much correct. GPUs were entirely SIMD, so they couldn't truly branch, but could fake it with predication. Longer branches would involve executing both branches on every thread and only committing the results of the active branch.
Modern GPUs can do much better, since individual warps/wavefronts can truly diverge. Within warps, however, it's still a bit of a mess.
If one looks at the capabilities of CPUs from 10 years ago, one can easily justify similar completely outdated statements for CPUs.
GPUs can implement control flow by calculating both results and then throwing away one of them based on a predicate. Also, GPUs typically do have a scalar unit which can perform branching but this is heavily discourage by CUDA and OpenCL due to the performance implications.