How many of us would use a music player that just had start and stop buttons? Why would this seem as an inconvenience? If the product is already familiar, then it would be better to improve on the already familiar interface. When it is a completely new technology, you get an opportunity to make a fresh start.
Similarly, many niche products have power users who would be unhappy if the interface is too "dumbed down". The classic case of an IDE and UNIX text editors come to mind. An IDE is quite obvious to use, but many would not find it as efficient.
Cognitive overhead is a good guideline but we need to understand our target market first.