This is not a bad idea in general. In fact, teleport proxy implements this exact model you have just described, where only proxy is available and acts as a jump host to the set of machines available only on the private net. The only difference is that instead of open VPN gateway it uses SSH jumphost model.
Teleport proxy uses OpenSSH cert auth, in addition to that teleport node also does cert auth.
Not everyone needs to always set up VPN, sometimes jumphosts + cert auth are perfectly fine.