One can still have freedom of organization without having official recognition of those groups. The tragedy to me is the extent to which we have raised the concept of political party into law. Why are there slots on the ballot reserved for parties? Why are party affiliations even printed on the ballot?
Any block of people should have full right to form a political party to espouse their shared views, but I don't see any reason we should do their advertising and organizing for them. I take it as an affront that when I register to vote that I'm asked to declare my allegiance to private social group, just as I would take offense at being asked by the government which of two official religions I belonged to.