OP should count themselves very lucky they're (eventually) getting an actual cash refund and not some sort of IOU only redeemable against future income tax due.
When you deal with an entity that has the monopoly on legitimate violence, your bargaining power is - unsurprisingly - borderline nonexistent.