The gist of it (pun intended) is that GitHub issues GitHub-issued (not issued by your cloud provider!) OIDC tokens to your job runs, and then your job exchanges those tokens for permissions with your cloud provider, which you have configured to trust tokens from GitHub.
From experience, OIDC authentication to cloud providers from GitHub actions boils down to:
1. Get your GitHub actions workflow to be able to work with its own (GitHub) tokens by configuring workflow permissions.
2. Configure your cloud provider to verify tokens issued by GitHub actions OIDC server thingymabob (technical term).
3. Configure your cloud provider to grant permissions in that cloud provider (or issue its own tokens with those permissions) based on certain values in the verified GitHub tokens that is presented with. If your cloud provider trusts GitHub, then it can treat the values in the token (workflow name, branch run from, owner and name of GitHub repo) as trusted.
4. Use a GitHub action from your cloud provider to do the negotiation with your cloud provider when your job runs, performing the exchange configured in (3).