Science is a combination of theory and experiment. Sometimes theory advances faster than experiment, sometimes vice versa. Right now in ML, experiment aka practice is advancing faster than theory. Theory will eventually catch up.
Maybe the only way to make steady progress here is to blaze ahead and rely on empirical evidence, whilst the theory is inevitably fleshed out. That's often the counter argument - that we do not know how the brain works, but rely on it nonetheless.
In medicine, machine learning systems will work alongside other brains.
What are the instances you’re imagining where a group of brains running an important system are replaced by a single machine learning algorithm running in isolation?
> that "we rely on the brain without understanding it and that works out fine" is not a good argument.
It wasn't my idea to bring cars into it particularly.
the depressing thing about machine learning to me is the many convincing explanations of phenomena, that then turn out not to explain things (all the explanations of why dropout is effective, or why there are adversarial examples, things like that.)
this is where the comparison to alchemy is the sharpest in my opinion. The alchemists had extremely sophisticated theories that they used to provide explanations for the phenomena they observed; just ultimately none of them made any sense.
All of these things help make your model more robust and give you greater confidence in it, which will be important if we want to put ML in, say, healthcare or defense. But you don’t get as much of this theory with more complex ML models, and certainly not with neural nets. Good luck trying to get a confidence interval for the optimal value of a weight in your net, much less interpreting it.