From section 13 of the license (https://www.gnu.org/licenses/agpl-3.0.en.html):

> if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network [...] an opportunity to receive the Corresponding Source of your version [...]

From the Merriam-Webster dictionary:

> interact, verb: to act upon one another

When distributing a binary from a compiler to a user, the user does not act upon the compiler. So the user is not interacting with the compiler, so you don't need to distribute your modified compiler's code any more than you would with the GPL.