I wouldn't do that because any of those servers could have a security vulnerability that we're not aware of, so I feel like this must protect against that somehow, but I'm just not fully understanding what it does.
That's not to say that the ALB can't have a bug or a misconfiguration that will render it wide open. But that's probably true for VPN as well.
The poor man's version of this is to put all your services behind an nginx reverse proxy with HTTP Basic auth (and TLS of course). For personal/small scale operations, this is a great way to almost completely eliminate your attack surface, if you have single-digit users and they can be trusted. Everyone running webapps personally should prefer this over, or in addition to, app-specific login systems.