I bought into Steve Jobs' idea phones should be usable one-handed. I think the sweet-spot for that is 4 inches or less. A premium phone at these dimensions doesn't exist anymore, the last one was probably the iPhone 5S.
I went from iPhone 3GS -> iPhone 5 -> Sony Xperia Z3 Compact and honestly the Z3 Compact is larger than I would like (4.6" screen). I'm honestly considering getting a 5S for my next phone because I don't want a phone as big as the iPhone 6.
I noticed recently that the One Plus 2 Mini is going to have a 5" screen, that seems insane for a "mini" phone.