GPL grants you the right to redistribute those binaries and sources. The contract is terminated if you exercise your GPL rights.
One of GPL clauses is that you can't add more restrictions. Terminating a contract seems like a restriction to me.
Also, terminating a contract is very different from not renewing one.