Might be correct for reasonably narrow definitions of language and thought, but it falls a bit short in considering the extended mind thesis. A whole lot of our thinking happens with pen&paper, their digital successor or other items out there in the world. We don't solve complex problems in our head alone, we solve them by interacting iteratively with the real world, and that in turn often involves some kind of language, even if it's just us reading our own scribbles.

Another issue is that a lot of tasks in the modern world are rooted in language, law or philosophy is in large part just word games, you won't be able to get far thinking about them without language, as those concept don't have any direct correlate that you could experience by other means.

Overall I do agree that there are plenty of problems we can solve without language, but the type of problems that can and can't be solve without language would need some further delineation.