I think this exact sentiment is explained over and over why people leave the Culture in the books. And why they don't actually have to -- full freedom to do literally anything is given to you as an individual of the Culture. There's effectively no difference in what freedom of personal choice you're afforded whether you're a part of the Culture or whether you leave it.