I'll say it. Been using the iPhone 6 for several weeks. Love it, but the screen is just a little too big for my tastes. It's awkward and I miss the smaller screen. It's enough of an annoyance I'm looking at other phones for the first time.
I'd say that 5" versus 6" is a qualitative difference. People don't buy a 6" phone thinking, "Eh, whatever". You don't hear a lot of people complaining that 6" phones are too big for the same reason you don't hear a lot of people complaining that Linux is too hard to use -- because you only got there by having already decided the answer to the question.