The OS used to do the networking for me. It exposes the BSD sockets API, which is pretty simple and nice but is unfortunately no longer safe to use. Yet, the kernel doesn't offer anything else.
So you're right, but you don't go far enough. It should all be in the kernel. Instead of tls_config_set_cert_file() we should be using setsockopt(2). Until we can, however, we'll have to rely on user-space libraries.