Posing your question differently, you get a completely different understanding of why the black box is much more alluring.
Which is better, something that can learn or something that does exactly what we told it to?
Which is better, something that can learn or something that does exactly what we told it to?
When you feed a bunch of input to an ANN, isn't it doing exactly what it's told? The ANN itself does not seek out knowledge, it only adjusts weights according to an algorithm depending on inputs.
The allure to having a fixed, visible algorithm is that you are aware of many of the limitations. Having a black box obscures those limitations, but I won't deny it's alluring in other ways.
Which is better, something which can learn exactly as we programmed it to, or something which learns by some unknown way in which the big 'ol bucket of bytes orders itself?
Or, if you had a magic learning machine, wouldn't you want to know exactly what its learning mechanisms are?