Not really.
iOS already, today, for no other reason than to prevent downgrades, verifies every firmware upgrade/restore with an Apple server, which sends back a per-device signature. Technically, it would not be difficult for Apple to have that same server authorize a special spy firmware only for specific device IDs. The only way this could result in a mass compromise is if Apple were hacked, but hacking Apple already gets you that - the hard part isn't writing code to spy on people (especially if you've hacked them and thus have full iOS source code), it's signing it.
This is different from exploits for code vulnerabilities, where there's always at minimum a secret (the location of the bug) that can't get out, and in some cases going from there to a working exploit is also difficult.
At worst, if Apple went beyond compromising that one phone and set up an ongoing process to compromise phones as requested by law enforcement - then there might be some sort of online portal, and maybe it could be hacked either directly or by stealing a legitimate agency's credentials. Maybe then the spy system could be used by unsympathetic governments against their enemies, though only by stealthily submitting individual requests; there would be no way to exfiltrate something from law enforcement that would compromise everyone. I don't think that's what most people are thinking when they talk about a back door "getting out" or whatnot.
Personally, I agree with Apple's refusal to create and sign a spy firmware on ethical as well as pragmatic grounds. But there's a lot of misinformation about the issue.