However, if everything you're installing that depends on Homebrew-provided software is also installed through Homebrew, then it should work regardless of location (I say "should" because I suspect that not every single Homebrew formula has actually been tested with a non-/usr/local install path).
Personally, I stick with /usr/local both because I've never had a problem with it, and because that makes it easier when working with software that uses a default PATH (since /usr/local is usually put in these default PATHs). But if you have any concerns whatsoever, then by all means use a different install prefix.