I did something like this a long time ago, and installing clang, gcc or ghc took FOREVER. I think I spent a whole day trying to install qmk just because various things required compiling. Granted my computer is old (MBP 2012).

I'm not sure it is worth the trouble. Can someone explain to me why I shouldn't just "trust" that the homebrew developers and maintainers know what they are doing and just have it installed in the default location? I get the theoretically I'm more susceptible to accidentally running a script that requires sudo, but are they any other downsides?