I agree with that, I think it's even necessarily true in natural intelligence which after all emerged by some means spontaneously. But scientifically I think it is a big problem because it does not supply us with a theory or systematized knowledge of the mind or intelligence. I think you could even imagine say, why not just make a primordial soup simulation, insert some DNA, and crank the speed up, intelligence is just a byproduct somewhere in there in a physics simulation. don't even bother with such details as neural nets.
Scientifically this is unsatisfying but also if for some reason this turns out to be an engineering dead-end we have a big hole where a concrete theory of intelligence should be, with its components, mechanisms and so forth. And sadly I think this is still the weakest link in AI.
To me it seems a little bit like if you trained architects instead of having a theoretical basis for architecture, you just showed them every building in existence and sent them to work. It may very well work, but if it didn't you have a problem. And even if it did, you'd still want to have an understanding of why it works.