Think about it: we also spent a lot of time configuring drivers, and I think we all agree that hardware autodetection is something to be desired in 2015, isn't it? That was my thought with compiling from source. Not that there is anything wrong with it, I just found it very weird because I didn't know that was the usual way to go on BSDs
NetBSD still uses pkgsrc which does support binary packages but the command line tools are a little less intuitive in my opinion. Incidentally, a few other UNIXes also support pkgsrc.
You may already be aware of this, but it's worth noting that the different BSD's are more like separate OS's, rather than distributions of the same OS like you see with Linux. eg DragonflyBSD, despite being a fork of FreeBSD, has some quite significant kernel changes (different schedulers, file systems, etc). And it's a similar story with NetBSD and OpenBSD too - though they're forked from other, much older, BSDs.
This is actually what got me to switch from linux to freebsd (though had I been around - systemd would have caused a ragequit), after I started running into problems related to how hairy my ~/bin and ~/lib directories were getting. Hopefully there is a nice linux solution that I just overlooked at the time, but I'd be surprised if the awesomeness of /usr/ports is approached.
Following -stable and keeping up with security errata is where you run into needing to compile from source. It would be nice if that were easier.