Because focusing on the IRS ignores the fact Congress can alter the IRS's mandate at-will to remove any of the policies of the IRS that is considered "bad". The President and his appointees can alter things within the mandate.
Get angry at the people that make policy, not the people that enforce it.
Sure, the IRS could be nicer. However, as someone who had to handle a policy enforcement role from time to time...I know the people I had to enforce policy on hate me, personally. However, I'm just enforcing policy. That enforcement is basically an email saying "No, disregard what X told you because Y is current policy."
People at the bottom usually aren't assholes "just because". We become assholes because people get angry with us over stuff we have no control over [unless we quit].