It would dramatically limit your attack surface to those who could gain access to your VPN.

I prefer requiring TLS mutual authentication with a corporate PKI and issuing employees client certificates.

Doing both wouldn't be a bad idea either.