Academia has never been dispassionate. The very idea of an academic conference started because academics hated each other so much that they needed a mechanism for them to see each other as people.

Further, a marketplace of ideas is a marketplace. Marketplaces are not emotionless voids where consumers dispassionately select the product that will provide them with precisely the best utility-to-cost ratio. They are emotional places where concepts like marketing and signaling are extremely important. Similarly, we'd expect a "marketplace of ideas" to be an emotional place and for human emotion to be a consideration when adopting ideas.