When it occurs more than once, it starts to move from the realm of "accident" to "deliberate", and I can easily imagine that in such a scenario, the corrections would become less gentle and less polite. That doesn't strike me as unreasonable, nor does it strike me as a free speech issue.
But notably, you're giving rather more power to free speech than is usually ascribed, which I find interesting. You're asserting that your right to free speech trumps my right to self-definition; if you want to call me a woman, you have that right under free speech, and if I want to correct you by explaining that actually, I'm a cisgender male, I can't do that because you have the right to free speech.
This seems patently ridiculous to me, and hopefully it does to you too. I don't see a way to reconcile your desire for free speech with my desire for bodily autonomy and self-definition. Can you clarify how allowing other people to choose the proper referential terms for me doesn't erase my ability to define who I am?