Personally I was burnt by Homebrew many years ago already. I lost trust in any such package managers. Since then I went back to just compile everything myself (I manage about 300 packages like that in my installation.) It's not that a big of problem. Only initially when you need to bootstrap everything to replace Homebrew it was quite the time investment, of course. But now you compile like a package here or there… just follow the script. It's very quickly done.
The best thing… no clutter. Every package has its strict directory. I only symlink from that what I need into /usr/local/bin.