I think a lot of the problem stems from the new page to front page algorithm. Only a small subset of readers on HN upvote stories. And only a small subset of those upvoters ever look at the new page. And both groups are self-selected and not necessarily representative of the whole group. And when looking at the new page, the top stories on that page get looked at more. And when people actually click on a link, the majority don't actually read the story. And on top of that, stories only show there for a fixed amount of time, and in some cases, it isn't that long.
The result of all that is stories are looked at by very few people before a decision is made (algorithmically) to promote it to the front page or not. Often it only takes one upvote. I've submitted a lot of stories, and some from my own blog, so I know the amount of traffic you get from the new page, i.e. the # of people actually reading your story while it is on that page is quite small.
One idea would be to randomly give x slots of the front page (under say, the top 10) to new stories. These stories would be randomly chosen from the past y hours to give them more exposure. A different selection would display on each page load. Presumably, once a story gets one upvote it would stop showing in this random process. Perhaps such an algorithm could also take into account how many people clicked on the article, and also do something with that information as well.
Thank you for asking.