Hacker News Reader

Comment by Bjorkbat | Hacker News Reader

Genuinely technically impressive, but I have a weird issue with calling these world simulator models. To me, they're video game simulator models.

I've only ever seen demos of these models where things happen from a first-person or 3rd-person perspective, often in the sort of context where you are controlling some sort of playable avatar. I've never seen a demo where they prompted a model to simulate a forest ecology and it simulated the complex interplay of life.

Hence, it feels like a video game simulator, or put another way, a simulator of a simulator of a world model.

View on HN

Also, to drive my point further home, in one of the demos they were operating a jetski during a festival. If the jetski bumps into a small Chinese lantern, it will move the lantern. Impressive. However, when the jetski bumped into some sort of floating structure the structure itself was completely unaffected while the jetski simply stopped moving.

This is a pretty clear example of video game physics at work. In the real world, both the jetski and floating structure would be much more affected by a collision, but in the context of video game physics such an interaction makes sense.

So yeah, it's a video game simulator, not a world simulator.

The goal is to eventually be able to model physics and all the various interactions accurately.

Sure, but if you're trying to get there by training a model on video games then you're likely going to wind up inadvertently creating a video game simulator rather than a physics simulator.

I don't doubt they're trying to create a world simulator model, I just think they're inadvertently creating a video game simulator model.

Are they training only on video game data though? I would be surprised when its so easy to generate proper training data for this.

It is interesting to think about. This kind of training and model will only capture macro effects. You cannot use this to simulate what happens in a biological cell or tweak a gravity parameter and see how plants grow etc. For a true world model, you'd need to train models that can simulate at microscopic scales as well and then have it all integrated into a bigger model or something.

As an aside, I would love to see something like this for the human body. My belief is that we will only be able to truly solve human health if we have a way of simulating the human body.

In the "first person standing in a room" demo, it's cool to see 100% optical (trained from recorded footage from cameras) graphics, including non-rectilinear distortion of parallel lines as you'd get from a wide-angle lens and not a high-FOV game engine. But still the motion of the human protagonist and the camera angle were 100% trained on how characters and controllers work in video games.

It doesn't feel incredibly far off from demoscene scripts that generate mountain ranges in 10k bytes or something. It is wildly impressive but may also be wildly limited in how it accomplishes it and not extensible in a way we would like.

Reply on news.ycombinator.com