Last week I was in Paris. Even with a refresh on grammar and vocabulary, I struggled a lot to communicate in French.
The most difficult thing, for me, is the sentence structure. I have enough vocabulary and grammar to choose the words I want, but I have a lot of trouble putting them in the correct order. And this is not a rare thing; it shows up in all sorts of simple sentences.
For example, in English I might ask "Where is (X)" or "Is this street (X)". In French, I need to flip those around to "(X), where is it" or "Here, this is (X)?" The sign to hang on the door of the hotel room would, in English, say "Please do not disturb" -- in French, it says "Thanks of not to disturb".
All of those words are, of course, easily recognizable, but the structure is completely alien to an English speaker (especially the way French uses infinitives in so many situations where English does not, giving rise to stereotypes of French speakers attempting English and always saying "Please to tell me..." and such).
I suspect that it would take me many months or perhaps even years of immersion and full-time speaking and thinking in French to get to the point of being able to construct most sentences in "natural French" structure on the first try.