Yeah you lost me there I guess, then. There are numerous instances I can think of where we censor our own language and it has nothing to do with removing someone's agency. And of course you can intend to not offend someone and still offend them; that doesn't mean your intent was wrong.
We can agree to disagree here. In the end, I would rather converse with someone who errs of the side of not wanting to "offend" me, but can still freely say whatever they want vs. someone who speaks with no regard to how their words might affect me and makes no apologies for it.
The outcome is the same, but to me, one person is thoughtful and the other person is a jerk. I have never thought that someone trying to be thoughtful with their words was patronizing me, but that's only my experience.