Maybe; but on a Nexus at least, you can't delete a lot of the basic Google services. For example you can "disable" Google Wallet (or whatever it's called now), but you can't "delete it permanently". It'll always be in your settings page, waiting to be reactivated. It's the same deal with things like Google Music.
In the MS anti trust days, you couldn't delete IE either. But you could use Firefox instead, and just pretend like IE wasn't there. Android seems like that kind of situation to me--you can't delete the Google defaults, but in a deep, dark advanced menu you can disable them at best, after a scary warning. In other words, you can use alternatives to Google apps, but you have to pretend like the defaults aren't there--but they always are.