I would recommend just using a VPN, like tailscale, for all non-public resources - rather than IP whitelisting.

Ed: including private web services like self-hosted gitlab not used for publishing public projects.