I'm not aware of any term in the AGPL that requires that. The AGPL says it applies only to modified versions or derivative works of the original codebase, and in general, just using an API -- even if it's unique to that product -- is not enough to make something a derivative work of that product.
(Otherwise, for instance, all software written in Java would be subject to Java's license agreement.)
The AGPL says that all clients interacting with the server through the network must be offered the source code of the server, not that they have to be AGPL themselves.
Of course, if you build your own client that uses code taken from the server, your client must be distributed under AGPL, just as if you did that with the old version, you would have had to use the GPL.