And it's heavily used by people with deeply unpleasant opinions, which tend to be more heavily promoted. The network effects of communication mean that your entire experience becomes more filled with that unpleasantness.
People left because Twitter was no longer enjoyable to them. The changes shifted the equilibrium that made the community work, and they left.
Some of that was just exemplified by Musk himself, who is a deliberately abrasive person, and made changes that amplified other abrasive people. He made a big deal about the right to speak, and so he has it. But he seems to have missed that other people have the right to not listen, and people's fear of losing that right drove them off the platform even before their feeds degraded.