But in all seriousness, this seems the most logical next step. AIs are getting so complicated that you need an AI to understand it. We just need to make sure they don't get so black box that we trust the outputs blindly.
Now you have two problems.
Black box is probably the only way we'll ever get AGI.
Our human brains are so black box that we can only accept their outputs blindly. We try to poke and prod the human black box as best we can, with magnets, sound waves, and light waves.
But it still generates useful output all the time, so we assume it's pretty good and generally let it be.
Obviously we believe only those brains that were primed with enough training data and have a decent track record of giving useful answers and making accurate-enough predictions.
But we still rely on this "blindly" in the absence of their source code or even any understanding about how or why they think, or why some collections of cells reason and demonstrate intelligence, while most don't.
We currently hold machine brains to a much higher standard of transparency and comprehensibility than we do human brains.
Human brains can't even tell us much about themselves or show any logging, because they barely know anything about how they work or where ideas or connections come from either.
But on the flip side, even today, a large part of the population doesn't trust people whose black boxes work faster than theirs. Now imagine a new black box that's 1000 times faster.
AIs are already effectively black boxes to most people, and too many already trust AIs blindly.