offending other people should never be the standard. By your definition we should ban South Park because it offends nearly everyone, so fsck your sensitivities.
The confederate flag should be banned because the kkk and other white supremacists adopted it, rebranded it as their own flag, and most importantly, no one cared to stop them. Try that with the Irish flag and see what the Irish have to say about it.