There were 84 million people displaced in 2020 alone. A single flood in Bangladesh can displace 1-2 million people. In 1998, when the country was a lot poorer and more fragile, 30 million people were displaced due to floods. But the country didn't collapse.
Even hundreds of millions of climate migrants worldwide, over decades, will be a challenge, not "civilizational collapse." Many of these will be internal migrations, which is all the more reason for countries like Bangladesh to rapidly industrialize so it can handle these internal population movements. From 1995 to 2020, 350 million people migrated from the Chinese countryside to cities. Yes, climate change will require people to move. But it's manageable.