Simple solution. If twitter is not (or no longer) acceptable to you, stop using it. Otherwise, by using it your content. It’s Musk’s toy. You know that so you’re making a choice.
The fact is that women get harassed far more than men. Taking away blocking will negatively affect them disproportionately. Ignoring that isn't being "evenhanded." It's being willfully ignorant of discrimination.
Each hostile environment we create for some group has little impact on society. But it adds up.