Instead of reinventing its own selfupdate functionality, Homebrew just uses git. Updating Homebrew is really just fast forwarding a git repository. Finally, the maintainers explicitly do not duplicate the libraries/tools that ship with OS X.
This is a big step up on OS X from Macports, which installs itself in /opt/local and rebuilds everything possible. Homebrew is instantly understandable. Packages are files, you just match them by name. apt is admittedly more complex, but at least you get something for that complexity.
[1] really what's slow is building from source. Homebrew itself is very fast.