We should probably require AI to always be able to explain it's conclusions.
That way we can quickly assimilate knowledge from the AI and theoretically always have at least as much knowledge as the AI.
I suppose it also means that we can verify that the AI is not lying to us.