At the moment, Photoshop is difficult to use, but has thousands of features (I have no idea exactly how many) that presumably professionals make use of, at least to some degree or another.
Try making a modeless version of Photoshop like you describe and most people would have to look up the key combination before doing anything, at least until they've learned all the modifier keys (which sounds remarkably like emacs). Not exactly the paragon of usability.
As with almost everything, UI is a tradeoff between several different factors. In the case of modes, the applications that people like them in tend to be used by professionals or experts for a very specific purpose. In such a case, the utility, power and expressiveness is much more important than the initial ease of use.
If your aim is to create something that doesn't have thousands of features and should be usability immediately (or relatively so) by new users, then modes are likely a bad idea; if your aim is to create something powerful for experts who will be using it 8 hours a day for the next 5 years, modes may be worth considering.