If I had to evaluate stories, my vertical axis would have 4 parts. From top to bottom:
* Unpredictable and irrational
* Surprisingly Unpredictable
* Tolerantly Predictable
* Predictable and stale
The closer the story is to "Surprisingly unpredictable" the better it is.
Cinderella, Kafka, Hamlet - all reach that point.
(But to be fair - Cinderella reaches that point only because its amongst the first few stories you ever hear.)
Titanic reached that stage for many teenage girls by killing the hero.
The best book I've read - Foundation Trilogy by Isaac Asimov reaches that "surprisingly unpredictable" point too.