Aren't taxes a zero sum game? If one "dodges" any amount the state must make up that revenue somewhere so the actors with finances and ability to best take advantage of the system then end up not paying their fair share?
It seems that you're beginning with the assumption that the expenditures are all sensible or necessary, thus the "need" to make up the revenue. To be sure, a deficit is a deficit, but when we start talking about people paying their "fair share", they're probably going to want to know, "fair share of what, exactly?" If politicians keep spending themselves into debt, regardless of how much more money they extract from the population, can we speak of one's "fair share" of the irresponsibility of the people holding the public purse?
If the taxes not paid creates increased growth then tax will simply be made up in the future.
Imagine two countries each with a $1000 economy, both have a 50% tax rate, but in one country every person 'dodges' 50% of their taxes, if that country grows its economy at say 3% instead of 2% then after 100 years two economies will be at $7,386.22, $20,068.17 thus the country with the tax dodgers will collect ~$5K in revenue while the other collects ~$3.5K