When I was a part of a team that did enterprise-ish management software, one of the complaints of our initial design is that it's too simple and it takes too long to perform actions (>1 clicks from the main UI), and actually making it far more complex (users had to learn where the buttons are).
Also, "Taste" is pretty subjective. Personally, I find software that's easy to use to be of good taste. A high learning curve (lots of options) is not necessarily bad as long as I can be efficient when I'm familiar with the software (think: Vim vs Google Maps)
There certainly are cases where usability falls by the wayside for looks, and plenty of cases where the people involved are oblivious to everything and are just trying to get something based on mainframe or first wave client-server technology to be accessible by the web.
The learning curve issue is a real one and it has a few dimensions. One is the case of an app you use every day where click minimization is the number one thing. The other is the app that you use only occasionally and the user's long term memory is the bottleneck. One trouble is that even the app you use every day probably has some corner that you go into every six months, so even that kind of app has learning curve issues that don't go away.