Model-Based Reinforcement Learning with Neural Network Dynamics
bair.berkeley.edu
bair.berkeley.edu
- Use neural network to learn (state, action) -> (state, prev_state) separately.
- Use this neural network to predict state in 100-step horizon and use that prediction to chose action in RL setting.
This method reduces sample complexity of RL. The method still isn't beating model-free methods with lots of data. I think the interesting part was using micro insects which unfortunately are not available commercially and results may not be reproducible by others.
Ah, OK. It's a bit more than that. They had to upgrade the bot significantly with next-gen materials to achieve high speeds, https://spectrum.ieee.org/automaton/robotics/robotics-hardwa...
I'm not aware of any work doing this combination - but I don't follow the simulation space closely. My impression is that NNs haven't been used broadly successfully in simulation until a couple of years ago.
But I think the model-based part is still considered relatively hard in real-world environments due to the large search space of possible actions/consequences.
And note that here they didn't actually use the model at runtime, they just used it as an 'expert' (rather than getting a human to provide guidance) to give a model-free agent policy a head start (see DAGGER algorithm) and reduce the number of training samples.
Best of both worlds?