I'm not relying on the contractual barrier, I'm relying on the interpretation of the clause as applying strictly to the behavior of the program as designed and embodied in the source code, and arguing that there is no moral requirement stated in the license for ensuring that that behavior is ultimately exposed to end users, and in fact, such a requirement would go against the spirit of the AGPL as a copyright license and it was explicitly written not to require that.
Some people might disagree with that interpretation, which is okay because it's the kind of fine point nobody will ever agree on until it ends up in front of a judge. Which is why I bring up the two party example, to sidestep that question.