Hacker News Reader

Comment by ris | Hacker News Reader

So, while it might be the fashionable thing to do some kind of (machine/deep/?) learning approach where you allow it to run millions of simulations and figure out things itself, I can understand why they didn't.

Learning approaches which depend on mass-simulation are great when your problem only ever exists in a "virtual" context, but what happens when you want to take your trained neural network out into the real world? Clearly it's going to have to adapt to the differences between the real world and the virtual world - but how would you do that? You can't run millions of dogfights in the real world to adjust its training.

View on HN

This is called domain adaptation and transfer learning in the literature. There are ways to do that. It is an active area of research. Basically the idea is to run a few real world dogfights (you could conceivably collect a few hundreds), and use methods to adapt the simulation model to the new domain. Solutions involving unsupervised learning (e.g. no dogfight, just collect sensor data from fighters - you could collect thousands of hours this way) are also active areas of research.

It is not about fashion, it is about not being ad-hoc.

For small scale problem where most of the variables are well understood, this kind of approaches work beautifully. Big problem are better tackled by a more generic approach (maybe with some ad-hoc adaptations, such as mixed approach between expert systems and statistical algorithms, feature engineering, etc.) because these approaches will be more resilient to an exposure to the real world, and the manpower invested in them is useful in more than one problem domain.

To address your last point, there is an extensive body of work on data-scarce environments. I've even seen a talk about applying reinforcement learning to endangered species preservation, where you only get a single digit number of interaction with the system !

The solution would be to make the simulator so good that there is no practical difference.

Reply on news.ycombinator.com