You would be surprised. But to say the truth, I trust the OpenSSH code a lot more than any VPN software that you can install to prevent direct access to it.
On top of which, OpenVPN has actually had fewer security vulnerabilities released than OpenSSH, and HMAC validation enormously restricts the surface area of exposed code as compared to OpenSSH.