Not having a side is a nice idea until your opponent forces you into a side. Then they complain when you defend yourself or you must give into their side.

Like when someone is arguing that people with your attributes should be killed or should have less rights than other people you don't have a choice in the matter. If you have a coworker yelling about how gay people are inferior to their openly gay coworker, there's no getting out of choosing a side.