I moved back to Linux after a number of years on a macbook mainly because of package management. As a developer, it's too useful to be able to do 'apt-get install foo' for almost every piece of software and have it just work.
Edit: Don't presume this for everyone.
For example, epoll is not supported (and I haven't seen a good epoll wrapper for kqueue)
That's really the core problem; on linux, all of the software on my machine is from the open source ecosystem and it tends to follow the unix patterns and work well together. On OS X, it was a mix of open source and what comes out of Jobs' Garden. It also meant that ports to OS X package systems were always behind linux.
I can understand your trepidation when using XCode to build things, but in reality I've never had any trouble using Homebrew. The packages are updated often, and if you want to contribute something, it's super easy (I've done it myself), since they're on Github.