Failure to pay taxes is illegal, not theft.
Theft is stealing the property of another. If income is the product of your physical labor, and the state owns your productive output (making tax avoidance theft, as you propose) then you are a slave.
How would you effectively pay for services that maintain the community behind the scenes, i.e. police and fire department? Swipe a card every time they show up?
How would we pay for the military without taxes? Just sell contracts to private security firms? How do we hold those firms accountable when they commit war crimes, as we recently saw from a former Blackwater contractor[1]? But then how do we pay for the justice system to hold those accountable if not through tax revenue? Make it so only those who can pay can participate?
0: https://www.latimes.com/nation/la-na-hanford-cost-20161216-s...
1: https://www.nytimes.com/2019/08/14/us/politics/blackwater-ni...
Also, you are overestimating my desire to pay for foreign military adventures. Your point about the police is actually great: the quality of customer service is so bad, racist and ineffective, that if they were a private enterprise, they'd have gone bankrupt.
Generally, I'm not saying the govt should have no role in things, I'm just in favor of reducing its current involvement by about 80%, along with the productivity it robs from workers and business.
So, no, you can't "just leave" if you don't like it.