Twitter can leverage a huge amount of data about personal preferences, interests, relationship status, political leanings, income, usage profiles, message habits and content, social graph, and location.
If you specifically wanted to clamp down on certain demographics, it would be an excellent way to locate them.
I don't believe for a second that Elon Musk really cares about free speech for anyone but himself and he is right now probably not limited by twitter's policy but by other externalities
In any case, does twitter not already have a lot of "assholes" and people with "violent ideas"?
My point being that I don't trust Elon Musk to make better moderation decisions than whoever is in charge right now.
Past attempts of social media with low barriers of what is allowed have absolutely turned out worse than Twitter if you ask me(e.g. 4chan, 8chan, gab etc) That's why I think the best case is "Twitter but more aligned with whatever Elon Musk thinks at the moment"