So, your alternative to letting someone else make their own decision is you (or someone else) making a decision for them?
All of the examples you just listed cut against your argument as much as they cut for it.
Bosses, peers, communities, priests in catholic churches or penn state employees _all_ have a history of abusing power as much as anyone else.
So, since abuse of power exists, and people can make good and bad decisions, it seems reasonable to expect a certain amount of bad decisions.
it seems safer to let individuals make their own bad (and good) decisions, rather than centralizing decision-making in a group of people that we hope will make good ones, but we must expect that a centralized group will make a certain number of bad ones.
Enshrining bad actors in a position of power is likely to have much worse outcomes than letting individuals occasionally make bad decisions.