I think you'll find that cancel culture is vigilantism. There's nothing to moderate how severe the "consequences" are.
But (IMO) there are two separate issues at play here. 1. everyone is entitled to free speech 2. every action has consequences.
[truth] 1. People who get "cancelled" complain that they don't have free speech [which is obviously not true], which is the crux of the discussion here. The right/wrong of the opinion doesn't matter in free speech.
[opinion] 2. "Consequences" (IMO) is a mix of {a sense of righteous indignation, mob mentality, schadenfreude} - but more importantly, also very much free speech. Vigilantism implies taking the law into their own hands, which "cancelling" is not.
When I say "deserve a backlash", (IMO) it is because people who get "cancelled" are often people with a platform/entitlement/privilege. Taking that away is pretty much the only leverage that people have. Even after that, the right to free speech is still not lost.
(But do I think "cancelling" is ok? That's context-specific. I agree with some cancelling, some not.)