There's a difference between saying "It's my right not to do this thing" and "I can't do that thing". Either can be true or false. In the case of the taxes, he said the latter, and it was false.
No, you are wrong. He's simply choosing to not reveal it.