People hate on him because he's an excellent host, and his lack of "outrage" on controversial topics goes against the fiber of their beings.
If you want to see guests arguing with their hosts, well that's a form of entertainment you can get on cable.