I understand LDAP can be quite obtuse to configure (particularly after OpenLDAP dropped config files), but once you get it working it is rock solid.
LDAP is definitely proven (esp in a LAN/non-cloud environment), but isn't perfect: if LDAP is DoS'ed (even from the very servers that it's authenticating for), then you can't log into any servers (or if a maintenance window exceeds the cache window, or even if it's just taken offline via hostname change or something). It also doesn't help much with user-based UX (i.e., key rotation, granting someone additional rights usually has to go through a centralized IT dept, etc) and caching is pretty awful, and it's a pain to securely configure. Lots of risk of inadvertently opening a hole through an insecure PAM/NSS configuration or ACL's.
The worst part with LDAP is the long and sordid history of LDAP and KRB5 root-level vulnerabilities.
Even with all of these drawbacks, LDAP probably less risky than storing (even different) hashes of the same password across thousands of boxen!
Disclaimer: I'm the CTO @ https://Userify.com ; we build an SSH Key management tool that's specifically designed for cloud servers.
Also its "easy" to set up insecure ldap or more precisely its easy to not deny insecure ldap connections, but its hard to mess up something like Kerberos to be overly permissive... But it doesn't matter, it just changes the scenario to "kerberos is dead for whatever reason, need root login to fix".
But, I hope, I describe point about LDAP and stuff in replies to other discussion members comments.