To sidestep this, create a new AWS account to be your master SSO administration account (separate from whatever account you're already using), and enable "AWS Organizations" in it. Join the existing AWS accounts to the new "AWS Organization".

Once you enable AWS SSO for the "organization", you can then set up SAML SSO to your existing identity provider (e.g. G Suite, which allows multiple hardware 2FA tokens per user). You do this in the new master organization account.

You can create the corresponding users in AWS SSO, and grant them the appropriate permissions in the appropriate organization accounts.

Then you do the 2FA auth to your IdP (G Suite, or selfhosted, or whatever) and then AWS just trusts the auth from the IdP, and you get to sidestep the terrible morass that is AWS MFA configuration.

If you get stuck, email me.