Even putting aside the ugliness of errno, this is pretty bad. I'm at the point with libc where I try to use as little of it as I can, though I'm not writing a lot of C these days
1) functions that you cannot implement in portable C - syscalls, signals, ABI specific things like longjmp and setcontext
2) functions that you could implement by yourself in portable C, but they are expected to play nice with the rest of the system - a lot of POSIX user stuff, dns management, dlopen, all the nsswitch mess. Can't think of any standard C examples because of the extreme portability requirements of C. Maybe the locale stuff, although it's usefulness is dubious.
3) functions that are there for convenience and sometimes compiler special-casing, like math, str*, <db.h> stuff that was thankfully removed from glibc
Most of #3 should be ignored. #2 is optional, especially if your software is for end users and not packaged by a distro. Sometimes it's safer to ship your own MDNS implementation for example rather than hope that the distro has sane defaults.
So you could write your own atexit, but maybe someone's got a version of atexit that cancels callbacks from unloaded libraries, or one that keeps them loaded if they have callbacks, or one that calls the callbacks immediately during dlclose.