Someone born in a particular house with a third party claiming ownership rights faces exactly the same keep paying up or vacate dilemma as someone born in a particular country. In both cases, they might find it difficult to find a piece of territory that is both appealing to live and doesn't involve paying someone else for their right to occupy it.
The relationship governments have with their citizens is unique.
It goes like this: give us a chunk of your entire income forever or else we'll throw you in prison. Don't like it? Say good-bye to all your friends and family and enjoy surviving in a country where you don't understand anything anyone is saying, which will probably just give you the same ultimatum.
That's simply mafia-style protection-money robbery.
There is another option: voluntary association and voluntary taxation. This is just one possible form it can take: http://groupcurrency.org
Finding a jurisdiction in which English is widely spoken and there's no income tax is actually surprisingly easy...
When you're a child, and your parents have custody of you, they make that choice for you--you're a citizen here, here's where you'll live, be politically involved, and pay taxes.
When you're old enough to make that choice for yourself, that's when you can stay--with all that entails--or go.
It's therefore impossible for any US citizen to legally avoid paying tax at least once, assuming sufficient income to have to pay tax in the first place.
Do note that this law also applies to foreign born US citizens, who are citizens by blood but who have never even visited the US. For obvious reasons, many of these citizens either don't know about their obligations under US law, or deliberately ignore it.
Or, don't work until you decide to leave, and then leave.
That it requires very quick timing or unusual planning to end your obligation does not mean that you don't need to pay it when you are obliged to.
In any case, the current cost to renounce your citizenship is $2,350. You might say it's a "fee" and not a "tax". That label is irrelevant in the overarching context of financial obligations imposed upon a person. Otherwise the US should just charge everyone a fee for having US citizenship and forget about the whole "tax" issue.
Moving just within the United States can end up being an expensive endeavor as is. Moving out of it isn't any more affordable, meaning that your argument basically amounts to "tough luck, poor people".
Not that I believe taxes are inherently bad, but the implication that "you can always leave if you don't like it" is misleading at best (and realistically outright false for a rather large segment of the American population).