Laser: Learning a Latent Action Space for Efficient Reinforcement Learning | Hacker News Reader

Laser: Learning a Latent Action Space for Efficient Reinforcement Learning

arxiv.org

65 pointstosh9 comments

Open articleView on HN

More info (including a video):

https://www.pair.toronto.edu/laser/

Department of Redundancy Department (please knock twice, please)

There was already a LASER method in reinforcement learning :-/

LArge Scale Experience Replay - https://arxiv.org/abs/1909.11583

I am missing a link between the two?

Teams who really stretched to come up with a cool acronym?

So they manually restricted the search space for possible answers and it went faster, like a LASER? Is this April Fools?

Looks like they used a variational encoder-decoder net to reduce the dimensionality of the action space.

Not manual and it sounds like a good idea.

Author here, was pretty surprised to see this on HN when browsing over my coffee this morning. Your interpretation is correct, you use an encoder-decoder model to figure out what the dimensions of the task best for learning are.

The drawback is you can only learn tasks which are relatively similar (any time you restrict what motions are possible to improve learning, you obviously restrict what tasks are possible). The benefit is that you can learn tasks which do fall within the learned motion ranges a lot more quickly.

The best analogy within 'classical' control is task space control, where you do control in cartesian dimensions rather than the joint positions. But this has its own drawbacks in that you have to define these controllers manually, and Cartesian space is not sufficiently expressive / appropriate for many tasks.

They used a variational autoencoder where the latent space representation is disentangled.

That approach is a promising way to make it easier to navigate the latent space as changes in one dimension will have a reduced or no influence on other aspects of the data encoded.

Here is a nice overview on disentanglement with further references: https://paperswithcode.com/method/beta-vae

Ludicrous acronyms shirk efficient reasoning.

Reply on news.ycombinator.com