The secure enclave isn't subject to these constraints, allowing for more conservative design decisions.
You've found a privilege-escalation attack that can let sandboxed apps escape their sandbox? Still secure if the chip can't run apps in the first place. You've found a bug in the USB disk mode emulation code? Still secure if the chip doesn't have any USB code on it. You've found a bug in branch prediction? Still secure if your chip didn't use it. You've found a way to abuse the third party developers' debugging interface? Still secure if your chip provides no such interface...
https://blog.invisiblethings.org/2011/09/07/anti-evil-maid.h...
BTW, Google uses hardware-based security on all of their servers:
https://www.theregister.co.uk/2017/01/16/google_reveals_its_...