Doesn’t this mean you have access to all the private keys and can take all the crypto from the wallets? Even if you of course pinky swear you would never take it.
For testing, dev work, and ephemeral agent flows that's a reasonable tradeoff for the convenience. For valuable wallets it's not, and you should generate locally, until TEE attestation ships, at which point these keys become safe to use for valuable wallets too.