Java seems like a good choice for the IRS. There are legions and legions of programmers well versed in Java, since its among the most commonly used programming languages to introduce programming at universities. This means the IRS should be able to focus its hiring on "People who know tax law and can use Java" instead of "Java experts that know enough tax law to be dangerous".
Its not the flashiest language, but I think its the right choice for them.