He didn't "talk" to Ben, he broadcast Ben's ideas to millions of listeners (the majority of which happen to be white).
As an example, I listened to one of those white supremacy former academics, can’t remember who, and he quips midway through the lecture that Western society developed faster because... we are unique in having a sense of shame. I kid you not. It doesn’t take a neurosurgeon to detect bullshit.
Rogan talks to interesting guests all the time, if you label him adversarial, you will have rejected a lot of people.