Hacker News Reader

Comment by Scene_Cast2 | Hacker News Reader

I'm curious about the addition of Tensor Cores. It sounds like both the regular Shader Units and Tensor Cores can carry out GEMM, just that the latter are more efficient at it. I'm guessing that for ML applications, both the Tensor Cores and Shader Units are used.

Are tensor cores idle during 3D gaming? How do they differ from the regular Shader Units? Why are they more efficient?

View on HN

The tensor cores are basically more efficient because they don't implement a 'read from register file; execute one instruction; write to register file'-style architecture:

They are basically a big array of ALUs hard-wired in a grid to do a matrix multiply: The connectivity between ALUs is baked into silicon: no general purpose register file to hold the intermediate results, just wires directly from the output of one ALU into the input of the next (well, and registers for pipelining).

No overhead for register files, instruction decoding, instruction scheduling, load/store units, cache, etc. (load/store is handled by the usual SM instruction set).

This is basically the same concept you find in any "AI acceleration" hardware (Google's TPUs, various neural-thingys you find in Phones, etc).

As for using in games, this hardware is generally idle, except for a few cases (typically because Nvidia did the work to light them up):

- upscaling from lower resolution rendered products to higher resolution (Nvidia markets this under the name "Deep Learning Super Sampling"), which is basically taking a derivation Temporal Anti-aliasing and mapping it onto the tensor cores to exploit that hardware

- Doing CNN-like denoising and reconstruction of a noisy ray traced image.

> Are tensor cores idle during 3D gaming?

Nv is making a use for them via https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling

> How do they differ from the regular Shader Units? Why are they more efficient?

They basically implement a special-purpose, single-instruction 4x4 matrix multiply operation with 16-bit floats. https://www.anandtech.com/show/12673/titan-v-deep-learning-d...

> Are tensor cores idle during 3D gaming?

The tensor cores are used for real-time raytracing, ML-based anti-aliasing during gaming, and other features. The Turing whitepaper has a lot of information on that.

https://www.nvidia.com/content/dam/en-zz/Solutions/design-vi...

A more generic answer to this question is that everything NVIDIA puts in their GPUs is for the gaming market. Other use cases are incidental.

I think the ray tracing cores may be separate, they accelerate BVH traversal and intersection.

The tensor cores are used to denoise the raytraced result.

Between the 30% framerate hit and the (to me) barely-perceptible visual difference, real-time raytracing still feels like a gimmick.

The new Minecraft stuff is the very first time I've seen raytracing produce graphics that make my brain conceive this could be a "realistically lit" scene, and it doesn't even take advantage of things like significant amounts of reflection and re-reflection. HOWEVER, I see zero benefit to the actual game from that.

In the case of Minecraft no new graphics feature is really going to benefit the "actual game", but players who obsess over their created spaces might really appreciate having better lighting that they can control through things like reflectivity. It's something you'd have to poll the average teen-who-still-plays-Minecraft about though.

Have you tried the many advanced shader packs that've been available for years and don't need raytracing?

https://youtu.be/JJeb1eEZYhI

If you haven't seen any screenshots of it at work in Remedy's Control, it's pretty nice there. Nothing groundbreaking but it's the first time I went 'oh!' and realized that I was seeing stuff that would've been invisible without raytracing. It's used for shadowing, reflections on glass, and a bunch of other stuff that ties the scene together and ensures things are where you expect them to be.

I expect we'll see some more compelling uses of it in the next few years, given that it took until the last year or so to see games start using it in a way that wasn't just tacked on. (It makes sense that Control was one of the first titles to use it extensively, since they were publishing talks about GPU raytracing well before the hardware was in anyone's hands)

We're in an awkward transition period where hardware raytracing is now available but not very useful for most people as the hardware is still niche and getting the most incredible results from it doesn't run at 60+ fps. But, if we ever want games/VR/AR/etc. to match the photorealism of modern visual effects, full-blown raytracing / path tracing will be required. It's going to take a while but it will eventually happen. The results will look far different than the before/after screenshots in the Turing paper.

Also, once the hardware is more ubiquitous (e.g. in consoles), some of the features provided by raytracing will make the game development process much simpler even if the results are only marginally better for some of the presented hybrid use cases.

Reply on news.ycombinator.com