Hacker News Reader

Comment by sillysaurusx | Hacker News Reader

It’s even harder to convince other people. Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one will acknowledge it.

Graphics — even movie quality graphics — don’t look anything like what a camera produces. The sole exception is when you’re mixing in real video with CG elements. But try to synthesize an entire frame from scratch, then put it next to a video camera’s output, and people can tell the difference.

Also, screenshots are misleading. You have to test video, not screencaps. Video is processed differently by the brain.

10 out of 10 times, all the graphics engineers come out of the woodwork going “but actually we do know how! It’s a matter of using so and so calculations and measuring BDRF and” none of those equations work.

View on HN

> don’t look anything like what a camera produces

The reverse-problem is a pet-peeve of mine: It seems many people have been accidentally brainwashed by Hollywood into thinking that film-camera effects are signs of "realism."

So then the first-person game introduces something like lens-flares, and everyone goes: "OMG it's so realistic now", even though the truth is the exact opposite. If you were "really there" as Hero Protagonist, you wouldn't have camera-lenses for eyeballs except in a cyber-punk genre.

I think it depends on context. There are a lot of sim games where the environment is very controlled and forums are littered with people who can't tell whether the occasional picture is real or in-game. A forest is hard to render accurately but a plane in flight is pretty trivial

Forests have pretty much been solved. Look at games like Shadow of the Tomb Raider, forest looks amazing. The difficulty now is limited to finer details, like hair. Hair is still pretty much unsolved.

See? It’s almost deterministic. People really can’t accept that we don’t know how to do something.

Plop a nature video next to your forest rendering and it’ll become apparent just how unsolved trees are. And everything else, for that matter.

The precise claim is this: viewers should be able to identify a rendered video no better than random chance. If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor.

> If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor.

To be fair: have you run such an experiment yourself, or are you just assuming that this conclusion will always result?

The motion blur will probably give it away. Accurate video motion blur is computationally expensive but conceptually simple. Just render at about 100 times your target frame rate and average batches of frames together in linear colorspace. You can speed this up by rendering at a lower frame rate (e.g. 10 times your target frame rate), estimating motion, and blurring along the motion vectors before averaging the frames. You can further speed it up by using an adaptive frame rate depending on motion speed and contrast. But a lot of rendered video doesn't even try. Look at a fast-moving bright point of light and you'll easily see the difference.

(But note this is only replicating video, not reality. Truly realistic motion blur requires ultra-high displayed frame rates beyond the capabilities of current hardware.)

A lot of games fake the motion blur so badly. Racing games are often especially guilty.

If you're driving at 200 mph, and there's a car next to you also going 200 mph, it shouldn't be blurry.

Also, the length of the blur should not exceed the distance an object travels on your screen in 1 frame. In other words, if an object moves 30 pixels from one frame to the next, then the blurred image shouldn't be more than 30 pixels wide.

This feel like multiple logical fallacies. It seems funny to me to claim in advance that people are going to argue with something that was intentionally non-specific, vague, controversial, unproven, and at least partly wrong, and then when someone argues with you, claim you were proven right. It's easy to predict that someone will argue with something misleading or wrong, and that doesn't give your CG argument any credibility.

What CG videos are you considering, what specifically have you looked at? Can you show some good faith examples of the best CG forests ever made, compared to some specific nature videos? Are you talking about attempts to match a nature video, and saying it's not possible regardless of what's in the shot?

Are you looking at the best examples of CG forests lately? There are some CG full frame video examples of forests I don't believe people would reliably identify as CG, if they didn't know before hand and you left out the explosions & spaceships.

https://www.youtube.com/watch?v=xOpuDhWzV1I

We definitely know how to make graphics look real - look at the unbiased renderers like LuxRender [https://luxcorerender.org/gallery/].

The problem is that we can't do that in 1/60 second on consumer-priced hardware, and also both scanning real objects and manually modelling are expensive.

Which of those images look real to you? To me, pretty much all of them look strictly like rendered images or have some aspect of them which gives away the fact that they are rendered.

We need a blind test where you evaluate a set of images on whether you think each one is real or rendered.

Heh, I agree! I'd love to try something like that if it exists.

With a caveat: the real images should not be cherry picked to look as close to cgi as possible but the other way around. Real and render can look indistinguishable that way too, but we want real looking cgi, not carefully arranged cgi-looking reality. The biggest giveaway is the simplicity and pristine sterility of the rendered scenes, no mess, clutter, irregularity. Just look at photos that people haphazardly snap in the real world.pick those for a blind study and cgi is nowhere near. Pick carefully lit and arranged artificial scenes with heavy post-processing, like studio shoots or hotel brochures or car ads and the difference will be less obvious.

I love this idea. Quick question for you, is this scene real or rendered? https://imgur.com/a/MpKMo6j

If you think the answer is obvious, then CGI definitely has more work to do. If not ... ??

Looks real. The dirt on the computer, the curls of the books and magazines and many other small details look very real.

If this is CGI, then I'm impressed and want to see more from where this came from.

Alas, it's real.

Agreed with you that I'd be super impressed if anyone could render at that quality, but I haven't seen it yet.

In the version that is shrunk down to fit my browser, it kind of looks rendered, because you can't see the subtle details that another user mentioned. Dust, the curling of book pages, etc.

But in the full version[0], it's clearly a photo. But it has been HDR'd which can sometimes create shading that gives a somewhat rendered look.

[0]https://i.imgur.com/Wn2XgFg.jpg

The full version also has artifacts from the camera that ironically give it away as a real photo. At full resolution you see this characteristic noise plus noise-smoothing effect that cameras apply.

That effect would be much easier to fake with a digital filter than rendering the entire complicated scene from scratch, though. Adding noise and noise-smoothing is simple. So if that's your heuristic, then you're gonna be fooled by digital renders that just incorporate that simple to implement effect.

Yeah, absolutely. It also shows though how you can easily accidentally bias a blind test. If an artist did manage to create a perfectly realistic render but forgot to add in one of the subtle unrealistic effects that cameras cause, people would be able to tell which images were photos.

(Although, maybe you could argue that if the challenge is to make photorealistic renders rather than realistic renders, nailing the camera artifacts is part of the challenge.)

Yeah, the goal has to be photorealistic renders, because what would the control data for the test be if not photos? You can't test one image on a screen vs "what real life looks like". The one on the screen is obviously the rendered one then every time.

It's not real multiple exposure HDR, it's just whatever default faux single exposure digital HDR the Pixel phone does. If it were real HDR the photo wouldn't be so blown out and over-exposed from the sunlight on the left side, particularly on the leaf.

It seems like your point is not so much that we can't technically render photorealistic scenes, but that artists don't put in the work to recreate the incredible level of detail we have in reality.

Yes. But in principle, tech could be advanced enough that it doesn't require so much manual effort. Like, if someone takes the time an works long hours on modeling a worn book with all its wrinkles and curls and crookedness, you can probably ray trace it realistically. But you could also say that artists can digitally paint a whole scene without any 3d engine or rendering tech.

The point is, currently it only works for simple scenes and needs tons of manual work otherwise. Reality is realistic effortlessly.

This isn't a big distinction to me? Either way, it's still true that it's nigh impossible to generate a realistic rendered every day scene. Whether that's because the environment modeling isn't there yet or the ray tracing isn't there yet doesn't make a big difference to me. Maybe the super detailed modeling will be the "last stand" prior to achieving general photorealistic renders, and the final breakthrough will be some kind of improved ML procedural generation algorithm for everyday lifelike objects.

I think Instagram is actually helping this from the other direction. My brain processes the two Jeep pictures as real but with a heavy-handed Instagram filter applied.

Again, it's important to focus on video, not screenshots. Video is processed differently by the brain.

"The Dress" illustrates how easy it is to fool people with still images. Movement gives crucial context. Our visual system has evolved for millions of years specifically to exploit that context.

also, the style of photos most of us are used to seeing are so processed digitally before publication, there's a real question of what it means to have a 'real photo' to approach in the first place.

That were capable of getting pretty close isn't that surprising, because most photos have already had N layers of digital effects applied, moving them closer to renders, rather than the other way around.

I think that with a lot of work we can make a real scene that looks like those renders, but none of those look like a "normal" real photo.

Do any of those actually look real to you? I'm not saying that a convincingly realistic render is impossible, but those are all obviously synthetic to me. Maybe that's just a lifetime of photography at play.

Meh...all of those are obviously renders. There's not enough contrast in the lighting in most of the interior shots, and overall surfaces just look too smooth.

Your comment seems to be ignoring the fact that today’s graphics look better than yesterday’s, i.e. the main point of the comment you replied to. The monotonic trend toward realism has continued ever since computer graphics was invented 40ish / 50ish years ago. VFX and games from 10 and 20 years ago look much less realistic that today. In addition to stills, there is an increasing number of videos and film scenes that are getting harder to distinguish from shot on film. Are you saying the trend will stop? If so, why?

> we do not understand how to make graphics look real > none of those equations work

Why do you claim this, and what do you mean exactly? Surely you aren’t claiming we can’t make any graphics look real at all. Some people are getting pretty good at understanding how to make some graphics look real, even if we aren’t. The math for approximating materials and lighting is getting pretty good these days, and the equations have been changing over time. It’ll continue to improve, but it’s certainly not the main thing holding CG back today. IMO less realistic CG comes much more from low resolution or low fidelity sampling / geometry / textures / animation / input data in general. Most of that is due to lack of time & money.

I’d say we understand how to make graphics look real in many cases, but not in all cases. We understand how to make good looking graphics, but how to do it quickly and efficiently enough to be practical and done with limited time and effort is an open problem.

> Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one will acknowledge it.

Making graphics that look real is almost equivalent to the Turing Test, I think plenty of people are willing to acknowledge that it's unsolved.

> Turing Test

Amusingly relevant yet slightly off-topic: https://existentialcomics.com/comic/357

Unreal engine looks pretty convincing with proper camera setup: https://youtu.be/zKu1Y-LlfNQ?t=118 .

So I've dabbled a bit in graphics, and it feels like a content problem. Sure, you can have an artist pump out some models and textures. But for every level of detail increase, the artist must spend ~3x the time of the previous level. So for example, even making just a simple a dirt road look _really_ good and photo-realistic would involve much more time (a week? a month?) than one can reasonably spend on a commercial project (where you have bajillions of assets to worry about).

I didn't study this movie frame by frame, but I feel the new lion king movie was photo realistic. It seems like that given enough time, human and computer, you can achieve a great result.

Reply on news.ycombinator.com