It is not going to sweat binary vs. Source distributions, and in fact, as long as an argument is sufficiently cogent, a court of law can probably mavigate multiple layers of indirection.
The law deals with people; and is not an ignorant machine. There is flexibility. There is also a principle in law that is basically expressed as "the court will hedge against inefficacy". If there are competing interpretations, the one that actually best upholds the intent of the legal construct will be gone with.
AGPL is considered Anti-SaaS pixie dust, as you call it, so the courts would look at it as "have you met your obligation under the license to make available to the user all code to allow the user to recreate your system's deployment?"
If yes, solid. If no, you have issues.
Not a lawyer, not your lawyer, but read way too many books on the nightmare that is our legal system to entertain your interpretation of the AGPL being unnavigable by a court of of law.