Reverse engineering the rendering of The Witcher 3 (2018) | Hacker News Reader

Reverse engineering the rendering of The Witcher 3 (2018)

astralcode.blogspot.com

263 pointsadamnemecek33 comments

As a movie compositor, I'm surprised to see so much compositing being done in sRGB space ("gamma space"). I'm used to doing all operations in linear space and only converting to sRGB for display purposes. Here, they seem to employ tricks to work around the issues of non-linear compositing, like making an exclusion mask of the brighter pixels so vignetting doesn't look bad.

What kinds of tradeoffs go into deciding to composite in sRGB instead of linear in a game engine? I assume there are good reasons to do so.

Memory bandwidth.

Typically linear is using 32 bit float while srgb is using 8 bit integers. So going linear needs 4x more data transfer.

There exists 16 bit float, too, but since it became popular with AI, NVIDIA has limited it's performance on gaming cards.

Ah, dear old Nvidia, never fails to extort you.

The other way to look at that is those being affordable for actual gaming and not perpetually sold out.

The top of the line gaming cards went from 899 for a 1080ti to 1299 for a 2080ti

I hope AMD / RADEON and Intel put some competition in that market.

Funny you mention it, there's new rumors on Big Navi just recently:

https://www.tweaktown.com/news/70848/amd-big-navi-specs-5120...

NVidia's grasp of the market is stronger than people realize. AMD video cards are comparable on a technical level, but they struggle immensely on the software side. NVidia has built a competitive advantage over those many years of total domination, resulting in better support for most games, more advanced features, more stability, etc.

> those being affordable for actual gaming and not perpetually sold out

This is less about availability and more about market segmentation. Making sure the cheaper product line doesn't eat into a market that is served by a much more expensive product line. This happens everywhere in any industry. And you can see it best in software where a simple license file will make the difference between a fully featured suite and a severely cut down version.

Availability is controlled via another mechanism: setting the price according to market demand. If you're selling so many cards that you miss a percentage of potential sales then the price will increase. And prices did increase.

There was the problem of that SGI patent that prevented using floats for framebuffers ... floats make a lot more sense rather than using this gamma nonsense.

AFAIK in HDR photography they often use linear uint16. I understand that it could be not enough for strong highlight modeling, but for general purpose compositing that should be ok. With an additional layer with like +8 bits of highlight resolution.

32-bit float per color channel would be 128bpp, which is really expensive. You rarely have the bandwidth to afford that.

There is also packed float format, R11G11B10. (it is probably cast to 32bit for calculation?)

What do you mean? When the values are already in the GPU, there is no need to usa gamma anymore.

Does the game described in the article transfer back and forth from RAM in the middle of compositing for some reason? If not, why is it compositing in sRGB space then?

In linear space (or should I say coding), operations like +, -, * <num>, average, a * 0.1 + b * 0.9 .. have a consistent, reasonable meaning with direct physical relevance. It is simple and preferable to calculate with linear values, but they don't fit well in 8 bits like sRGB does.

In sRGB, usual operations give weird results instead. Imagine when doing addition (e.g. light a + light b) you get something like

  c = a + b          # linear result
  c = sqrt(a² + b²)  # sRGB result (very simplified)

Instead of average (downscale 2 pixels), you get

  c = a/2 + b/2              # linear result
  c = sqrt((a/2)² + (b/2)²)  # sRGB result (very simplified)

Maybe I messed up the formulas and used inverse transformations, but the point is, if you operate on sRGB values with the usual +,-,*.. operations, instead of adding/subtracting/averaging/.. the light/colors, you end up applying some functions, that are kind of similar, but not very close.

We are kind of used to it when dealing with computer graphics, but this is the reason why antialiased fonts look too thin (or thick, depending on whether it's light or dark), scaled images look darker (or is it lighter? and slightly off-color), and generally the graphics operations in sRGB give disappointing results.

Thanks a lot, I know about both spaces. I thought virtually all modern games only work in linear mode within the GPU, and avoid any compositing in non-linear. That was my first surprise when reading the first comment.

Then someone else replied explaining that such compositing is done due to bandwidth, but I do not understand: after textures are uploaded to VRAM, PCIe bandwidth should be irrelevant. If they are talking about VRAM bandwidth, I would understand if this was for a lot of big textures... but for doing vignette? Is better to be going back and forth to sRGB in each compositing step just to keep the intermediate results smaller due to VRAM bandwidth?

sRGB can be stored in 8bpp. Linear can not. ALU is cheaper than memory bandwidth, and the conversion is in hardware on modern GPUs. Typically, the conversion from linear to sRGB is done as part of the tonemapping which converts from linear quantities, usually from an 16-bit HDR framebuffer target to an 8-bit one.

Mixed in with this is MSAA where the resolve is typically done in non-linear space (this is a choice to artificially boost "smoothness", this is an artifact that happens IRL) [0] https://mynameismjp.wordpress.com/2012/10/24/msaa-overview/

Is the lighting model of vegetation and trees mentioned in any of the chapters? The sunset lighting up the edges of decals was really what sold me, that and just the stylized graphics - not trying to jump into the uncanny valley. To me it's the best looking environments in a game I have seen.

This is some crazy sht, I’m surprised the expertise people have and to be able to do this

That's impressive indeed. I'm even more impressed at developers of DXVK, who implemented the whole translation of D3D11 into Vulkan, including for shaders (enabling playing TW3 on Linux).

The dolphin (Nintendo GameCube emulator) developers have written[1] (nice read!) an interpreter of the GameCube shader code inside of the host graphics card shader.

[1] https://dolphin-emu.org/blog/2017/07/30/ubershaders/

That's a very cool idea indeed!

It is a big bucket of different techniques that have been developed by many different people over the years. Very much a standing on the shoulders of giants (and then also putting in a lot of regular sized human research and hard work too) kind of situation I think.

This is indeed very cool content! The type that keeps me on this site. I too am constantly shocked by how many reverse engineering articles. The patience and knowledge required is insane

If you like this, you'll also like the "Graphics Study" articles of https://www.adriancourreges.com/blog/

Thank you for sharing this :)

Neat writeup, I haven't played the game but was surprised to see so much aliasing around objects in his screenshots considering everything was on "max settings". Does TW3 predate ubiquitous use of MSAA or FXAA techniques?

IIRC it doesn't use MSAA (to do MSAA on a deferred renderer you need to take it into account in your pixel shaders, which in practice means that basically nobody does it) but it does have post-processing antialiasing (including, IIRC, FXAA and TAA).

However since these are post-processing solutions they can mess up with the frame reverse engineering going in the article and most likely the author simply disabled them.

> to do MSAA on a deferred renderer you need to take it into account in your pixel shaders, which in practice means that basically nobody does it

I think the bigger problem with doing MSAA with deferred rendering is that your G buffer gets even bigger since you now need to store information for each sample.

I think it's off by default, but it has a config option:

    AllowMSAA

Standard anti-aliasing enabled through the settings menu is FXAA-based, with some temporal processing. MSAA can be switched on by editing the file bin\config\base\rendering.ini in the game directory and setting AllowMSAA=true and MsaaLevel=2|4|8 but the consensus seems to be this doesn't improve things much, while the performance impact is significant.

Another setting in the same file, UberSampling=1, could offer a bigger improvement but with an even larger performance hit. The most practical solution, given sufficiently powerful hardware, is probably to enable Dynamic Super Resolution (DSR) via the nVidia Control Panel.

Witcher 3 graphics is quite amazing and can get even better with community-created mods. It might take some tweaking to get there but the result can be like this: https://www.youtube.com/watch?v=28lGfkdkJHI

NoClip has a nice documentary series on YT about making of The Witcher 3: https://www.youtube.com/watch?v=hWDaiox6QqE&list=PL-THgg8Qnv...

amazing work OP.

blogspot!

Reply on news.ycombinator.com