Rooting an arbitrary Android phone is not easy, and rooting an arbitrary Android phone (especially the Nexus/Pixel flagship devices) without wiping the storage is really not easy.
However, it still remains that the secret is sitting in plain text on a hard drive on the phone. If you unplug/unsolder the hard drive, you could just read it, like a PC.
Another advantage is that if the secure enclave is hardware/firmware linked to authentication (fingerprint / password), then there would need to be a vulnerability in that hardware process for a remote users to get a break.
This is the second factor of a 2FA, so I agree that in most cases, it won't be a large issue. Someone who phishes your password over email would empirically be unlikely to hack your phone.
"In plain text"... on a disk that is fully encrypted. Full-disk encryption has been available in Android for several years, and required for almost all of that time.