If we replace "taxes" with "paying for an army, paying for police, etc." than the above statement seems strange. I can understand "I don't like what society choose to build (with the taxes)", but some things can't be let to the free market.
> I have to go do the fucking IRS's job
That's a society choice. I have no experience with IRS, but other countries have tax systems that are much more automatic (come prefilled with most info for example). But taxes are maybe a bit higher (because if you don't do the job, someone needs to maintain the systems that allow the automation).
In most countries filing taxes is a painless process that takes 5 minutes and is more or less completely automated for you.
This is an expression of personal preference. It is a common preference but not universal.