From what I've heard people fear the complexity, not the increased rates. Also it's hard to get health insurance or a mortgage when self-employed.
But I am annoyed my insurance costs are going up by quite a bit, but in the grand scheme of things I still pay very little.