you can keep them on a VPN and connect to that VPN yourself from anywhere, without exposing them to the possibility of anyone connecting to them directly from the internet.
If the application is properly secured (strong encryption, strong authentication) there is no point for the VPN
and if the application isn't configured correctly, the vpn/internal network might not be configured correctly either.