So when you compile a program and distribute over the network as a binary, you are also distributing the compiler that created the binary?
> if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network [...] an opportunity to receive the Corresponding Source of your version [...]
From the Merriam-Webster dictionary:
> interact, verb: to act upon one another
When distributing a binary from a compiler to a user, the user does not act upon the compiler. So the user is not interacting with the compiler, so you don't need to distribute your modified compiler's code any more than you would with the GPL.
The disputed claim is that users who interact with a remote program are, by that interaction, being distributed the program.
The GNU AGPL itself explicitly disagrees with this; check its definition of "propagate" and "convey".