I'd say the difference is how essential your user account is to the service. You might say you're "leaving Youtube". You'd definitely say you're "leaving Gmail".

Which touches on the point that saying that you're "leaving Google" is unclear because there's a huge family of products and services that you might mean.

You can use Google's search anonymously.