You step on my foot. I look at you, and say "sorry", and pull my foot away. You abashedly also "sorry" and we move on.
Am I apologizing? Hardly! The full sentence is, "Sorry, did you just actually step on my foot, what the hell?" and your reply is "Sorry, yes, I did do that, didn't mean to, apologies". Then the problem is solved and we move on.
> But in the US, I've been pounced on for a casual use of that word, as if I'd announced failure and that I owed something to the other party.
It is precisely the opposite in circumstances like the above!