GPU Architecture Types Explained

The biggest barrier for anyone making a purchasable GPU is the current patent minefield dominated by Nvidia, Imagination, and others.

Not that I know of, I don't think they are specifically unviable, but even RiscV is relatively new and in it's infancy, and CPU's in a sense are a subset of GPUs (modern GPUs have pretty comprehensive computation ISAs) and additionally, there exists a much more viable market for low-end CPUs that doesn't exist for GPUs in microcontrollers.

> Why even bother implementing actual rendering in hardware when you can just implement fast general-purpose calculations and use them to accelerate software rendering?

The answer is sort of a mixture of A. That's what we have and B. because custom hardware is very hard to beat. Do read up on Larrabee[1]

So, GPU's have been converging on being more and more programmable, there are a few parts that are still much faster done in silicon, and very widely used, specifically rasterization (Though, software can be a win with e.g. micropolygons, See UE4's Nanite), Texture Sampling, and we're starting to see BVH traversal in the case of ray-tracing. And rasterization specifically is the one that pretty much dictates the structure of the render pipeline, with vertex/fragment shaders, etc and is broadly what makes it's hard to fit it into a "general-purpose" pipeline with specialized instructions.

And just keep in mind, Cuda is basically just a compute shader, and a compute shader itself is basically just a fragment shader without rasterization stuck on the front and without render target blending stuck on the end, and that's also specifically the history of how we got to general-purpose GPU computation ala Cuda.

[1]https://brightsideofnews.com/an-inconvenient-truth-intel-lar...

So at any point in time among the N threads they are M that are ready to execute (ie not waiting on memory) and those do execute. Anytime a thread need to lookup memory, like accessing a texture for instance, the memory lookup is schedule and the thread is put to sleep but its states stays in the GPU. But overall they are always M threads not waiting. This is why if you do performance analysis on GPU in some basic cases the memory lookup operation looks like they are free (ie takes no time).

On the CPU when you context switch between thread the kernel save the CPU states (all register values and ancillary states) to main memory. Which means that switching thread on a CPU core means writing current thread context to memory and reading next thread context from memory. So it even worsen the memory latency issues.

If you design a CPU capable of holding many threads context within silicon (on die memory) you might get closer to a GPU. But you do not get much from doing so for CPU workload. Also at which point does your design is more a GPU then a CPU ?

-- risc-v was created to support data parallel / vector codes, even though the open thing is what took off, and all sorts of cool stuff being built (not an expert but def a fan!)

-- ... but the reason nvidia >> amd, intel, google tpu in practice is the compounding ecosystem investment over the years: compilers, libs, and now, even rest of the chip/network . Ex: Intel & AMD & IBM tried to push OpenCL for example soon after CUDA, yet ~no one runs AI stuff that way, the ecosystem just isn't there in practice.

They are optimizing for different things.

CPUs are optimized for single-threaded performance, i.e. lower latency. They only have a few cores, but these cores are spending billions of transistors implementing cache hierarchy, reordering instructions, predicting branches with neural networks, predicting indirect branches, prefetching data from RAM, executing instructions speculatively with the ability to rollback, etc. All these things are very complicated, but they do help with latency.

GPUs don't care about latency. They only care about throughput on massively parallel workloads. For all these cases when a CPU needs to do something smart to minimize latency, GPU cores simply switch to another thread to do something else in the meantime. That's how they have count of cores measured in hundreds if not thousands.

One more thing, high-end CPUs run at ~4GHz to minimize latency, GPUs at 1-2GHz to maximize power efficiency.

The result is about an order of magnitude difference in throughput. Ryzen 9 5950X peaks at 1.8 TFlops FP32, yet similarly priced Radeon RX 6800 XT peaks at 17 TFlops.

There's a whole section on it near the end:

"Another class of hybrid architecture is one that is often referred to as tile-based immediate-mode rendering. As dissected in this article, this hybrid architecture is used since NVIDIA’s Maxwell GPUs."

>Notably missing any mention of TBDR which currently powers the fastest low-power mobile and desktop GPUs on the market.

Another section mentions:

"There’s a long-standing myth (that luckily slowly disappears) that deferred rendering techniques are not suitable for TBR GPUs. "

The micro-benchmark that suggested Maxwell (and later) was a tiled deferred gpu was actually measuring something else. Each GPC gets assigned different sceenspace areas, and concurrency rules between the areas is relaxed (unless explicitly required by shader atomics).

The result looks somewhat like tiled deferred rendering in that micro-benchmark. But it's still very much immediate mode.

A similar thing happened with Navi.

However, there are mobile GPUs (Qualcomm's Adreno) that dynamically switch between tiled deferred mode and immediate mode on a per renderpass basis, depending on what driver heuristics suggest will be faster.