The same is true for traditional computer programs. The standard argument against valuing understanding is "well, you don't understand the assembly, processor microcode and the electrical signaling when your program runs either, why do you care about understanding the high-level code then?"
I'd counter this by saying that high-level languages have (mostly!) self-contained abstractions that allow you to "understand" a program deep enough to make accurate predictions of its runtime behavior, answer "why" questions when errors happening and safely make changes to it - without having to understand all the stuff at lower levels.
In contrast, with AI, I see the risk that we lose understanding of both the low- and the high-level structure of things. Then we really end up in "unpredictable alien artifact" territory.