Hacker News Reader

Comment by seanwilson | Hacker News Reader

I'm likely completely missing the point but how is this concept of imagination different from looking ahead in a search tree? Isn't exploring a search tree like in Chess or Go exploring future possibilities and their consequences before you decide on what to do next?

View on HN

A search tree in something like chess is quite small, and very discrete. You can enumerate every possible action, and exploring the tree to a useful depth is computationally tractable. By contrast, for an agent operating in a complex environment, like a robot in the real world, even if you somehow came up with a coherent process for listing every possible action the robot could take, you might not even be able to store them all, let alone compute their consequences. Think about the sheer amount of information you'd need to process. Moreover, the real world is (for practical purposes) continuous. The robot would have the option of engaging one of it's motor for one millisecond, or two milliseconds, or three milliseconds, etc.

This seems to be tackling the issue of what to do when there are just too many options, and the depth of exploration necessary to make useful predictions is too high, for you to just enumerate everything, heuristically prune, and pick the optimum.

> Moreover, the real world is (for practical purposes) continuous. The robot would have the option of engaging one of it's motor for one millisecond, or two milliseconds, or three milliseconds, etc.

Are there not similar techniques to search trees that are used here? Obviously you wouldn't enumerate all options but you'd think you could guess at some practical ones then guess options between the most promising. Either way, it just feels "imagination" is making it sound like an entirely new approach when heuristically pruned search trees could be described in the same way to me.

Ability to generate candidate paths in continuous search space is a bit more general than pruning prebuilt search tree.

Search tree is an approximation of continuous search problems and needs to be built by someone. This approach builds its own search tree.

It also paves the way to algorithms which can solve problems by repurposing available actions for achieving unintended goals, and by creating new high-level actions from low-level ones.

That is solving problems creatively.

The difference is that in chess or go, generating a search tree is trivial - predicting the world state after a couple turns of go or enumerating all the possible opponent moves in chess takes just a small bit of straightforward code encoding the (simple) rules of the game.

But how does an agent (not you) figure out a search tree of some nontrivial problem? How do you predict what the world state will be after taking some action if a programmer hasn't done that for you? Heck, even how do you predict what the world state might be after a second of doing absolutely nothing in a real-time environment? This is what this research is about.

I am thinking exactly the same thing. Maybe they are trying to get some hype from media?

Reply on news.ycombinator.com