Let's imagine a state had 100 representatives (to keep the numbers nice). And in this state 35% of the population most strongly identified as republican. 34% most strongly identified as democrat. 16% most strongly identified as green, and 15% most strongly identified as libertarian. And this state had a really good geographic mix of people, so everybody was completely evenly distributed. As a nice aside there you don't even need to worry about things like gerrymandering - any sample of any area will have an identical mix of people as any other area. That's good... right?
What would be the most reasonable result for the makeup of their representatives? What would we actually get? I think most of everybody would say the same thing for the most reasonable result. 35 republicans, 34 democrats, 16 greens, and 15 libertarians. Few would correctly answer what we'd actually get - 100 republicans. And in a nutshell, this is the real problem. District based plurality elections, be it first past the post or IRV will trend towards domination of two major parties. The only solution is proportional representation. You get 10% of the votes and if there's at least 10 seats then you're guaranteed to get at least one.
Proportional stands next to no chance of being implemented since it would require the current political establishment to sign their own [political] death warrants, but if we're going to change the electoral system I think we should at least look at solutions that would actually solve the problem.