If the other side claims you can control them to score points when they go up, why would you not point out when they go down?
Integrity?
It's not lacking in integrity to point out your opponent's criticism is no longer valid, even if - especially when - that criticism was made in bad faith.