I have everything installed in ~/Homebrew and have had no problems.
I really wanted to keep liking Macports, but I got sick of having to install a new perl just to install git, among many examples. The kicker for me was installing pandoc; Macports wanted to download and compile a new GHC, refusing to recognize the one on my system. I let it run for 3 hours and finally killed the install and looked for a better way to do things.