Which the recent xz attack used to mess with `sshd`, even if it never actually used functions from the library.
dlopen loading only loads the library *when it's actually used*, and doesn't include everything and the kitchen sink on startup.
If the libraries are used seldomly enough, it might also allow to make more dependencies optional in package management.