Can't go wrong with the Nexus phones. 5X sounds like your ideal phone. Or perhaps the new pixel phones.
OP, grandparent is correct, I was in your situation and a 5X was perfect. Pixel and Pixel XL look like they'll be too big, but you should wait for the October 4th event to confirm.