OS X without Xcode installed doesn't have a compiler - if you can't compile, you can't use macports or homebrew.
It would be "neat" as in clean. No drama on Apple's behalf. Just put a price tag on the only real compiler, and it's over. Unless someone solves that issue.
Xcode is an IDE, you can still get the compilers as both of them are open source.
It's packaging and bundling that's the problem. There is enough people out there that don't want to have to manually install compilers etc versus just installing Xcode.
Well then they have a choice. Pay $5 for the convenience of installing gcc/clang via a 4 GB download, or install gcc/clang manually for free.
Uh...OSX just uses gcc as a C compiler and all the Apple specific changes are open source as well (GPL license and all that), so what's stopping me from just distributing a binary (Apple patched) gcc?