We're not talking about annoyance here, we're talking about data breaches on the one hand and permanent account lockouts on the other. There is no world where the majority of people remember distinct passwords for each service, and there's likewise no world where the majority of users carry a Yubikey or even download their recovery codes at all, much less carry them with them.
So we're left really with two choices at the extremes: do we go for the failure mode where people reuse passwords and lose money and data that way, or the failure mode where a company is physically incapable of restoring a customer's account? Given the choice between these extremes, companies will always choose the former.
Now, these extremes aren't the only choices, but then you get to the crux of OP's argument: passkeys stored in a password manager aren't any more secure in practical terms than random passwords stored in a password manager. Your auth system is only as secure as its recovery mechanism (which is the observation that led Slack to initially not bother with passwords at all).
There's one way in which I disagree with op, which is that passkeys prevent users from typing in a password and storing it in the password manager, which many users do—they effectively enforce random passwords rather than it being just a best practice that needs to be taught.