The latter approach certainly hasn't worked either. Look at the rise of white fascism in America. It grew out of the free-speech open dialogue wild west internet (4chan, 8chan etc) and became so radical that it's racked up quite a body count thus far.

I don't think cancellation is about changing the person-being-canceled's mind on a given opinion. I believe it functions entirely to signal to the rest of society and culture that such positions and actions are unacceptable.