That’s kinda the point isn’t it. Central management of access to everything.
So if AD were to be compromised, that would be significant impact.
There are of course advantages to such a "single point of failure" such as concerted effort in one place. But one way to mitigate the spof is transparency, and I'm reminded of LastPass versus Bitwarden.