So add a wrapper for that, a quick script that checks which branch and revision you are publishing from. The issue here is publishing from a CI you do not control that well and with automated events.
You can run the exact same script locally as you do in CI, with the only difference being the addition of a 2FA prompt.
That's a good point, I would lose package provenance that way. I guess that is fine since it didn't prevent anything here.