I know exactly what you're talking about and have been in similar situations many times. For me it's not just limited to speech, but lots of other aspects. Sometimes it feels like being a ghost or some weird Star Trek like phase shift. E.g. one person walks out of a board game night to get some food, everyone stops playing to wait for the person. I walk out, things just continue.

What helped me a lot were 2 things:

1) There are ways to improve your conversation skills. Big topic, with lots of branches. Your speech matters. Your tone. Keep collecting interesting anecdotes. Culture some depth to your personality that is unrelated to work (e.g. interesting hobbies). Essentially train your charisma.

2) The big eye opener for me was the discovery that different groups of people actually react differently to me. I.e. I was simply friends with people who ... didn't care as much about me as I cared about them. The blunt fix here is to change your social circles. Not easy, but doable, slowly, over time.

The "interesting hobby" part of the charisma training actually helps there to connect to different groups of people.