>You can leave the country and stop paying taxes, you just can’t come back.
No, you can't. You have to renounce your US Citizenship to stop paying taxes in the US, no matter where you live. Renouncing requires acquiring citizenship in a new country, which isn't that easy: just ask anyone who's tried to naturalize in the US.
>Again, you’re part of a society and you can’t opt out, even if you don’t want to pay.
You're not really a part of the society if you don't live there, but if you're a US Citizen, you're still required to pay both federal and state taxes, even if you've never lived in the US in the first place.