Comment by ColinWright | Hacker News Reader

Comment by ColinWright | Hacker News Reader

Fascinating to look back at the reactions from when I posted this 500 days ago. One person said:

    while this is a classic, my snark is thrashing against
    its bonds. I could have sworn it said 'news' up there.

And another:

    SENSATIONAL HEADLINE ALERT!!!

And that was it. No further comments to be had. Both those comments were downvoted, but even so, interesting that no one else commented, and very few voted for the submission.

For reference, the headline I used was:

    Engineering pornography - underground
    power cables gone wrong

... which I felt was more informative.

So on this submission, now even older, we have 16 comments (and counting) and 60 points (and counting). Has HN changed? I'm off to check some records ...

(pause)

A bit of hunting shows that this got at least one vote while still on the "newest" page, but it looks like it never hit the front page, which will part explain why it never got many comments or votes. Still, the tenor of the comments is interesting. I continue to learn.

As what becomes popular on HN has a large dosage of randomness, I would not draw conclusions based on this one data point.

There was an interesting study from a few years ago that investigated what factors make a song a hit. They seeded various songs with various levels of initial popularity, and their general conclusion was that popularity begets popularity. Hence, initial conditions matter a lot, and they are largely random. (And if anyone can find that study, please let me know. I have tried multiple times to find it, but I can't.)

Yes it does have a large dose of randomness, and I'm not taking this single data point. I have more data. Quite a lot, actually.

And the challenge in the analysis is to find the factors that affect it, and then add the randomness to get the right results. To find the underlying behavior, even when the data are affected by randomness. It is kinda what I do, and that's partly why I'm working on it. I'm learning things as I go, both about the techniques and methods, and about the HN "community".

Interesting study you mention - I've made a note both to look out for it, and to let you know if I find it.

Thanks.

These are the droids that are being referred to I think: http://www.princeton.edu/~mjs3/musiclab.shtml "Experimental study of inequality and unpredictability in an artificial cultural market."

This wiki article might be of interest too: http://en.wikipedia.org/wiki/Matthew_effect_(sociology)

Is this the study you had in mind? http://www.princeton.edu/~mjs3/salganik_dodds_watts06_full.p...

I read about a similar experiment in the book "Automate This" by yc alum Christopher Steiner. They seeded a music website with unknown songs and it turned out that whatever was liked first benefited from the Matthew Effect. ie, people tend to like whatever is already labelled as "popular". Definitely an interesting study, but I fear we've gone far off topic.

I think it is. Thanks!

You may also be intrigued by the comments from when it was posted 1677 days ago:

http://news.ycombinator.com/item?id=193112

Well found - thanks.

So you are complaining that you didn't get the karma 500 days ago that wallflower got? Or are you just pondering the mysteries of stories getting picked up? I see it isn't the latter since in this comment http://news.ycombinator.com/item?id=4097127 you indicate you've done research on that topic.

So what are you saying?

    So you are complaining that you didn't get the
    karma 500 days ago that wallflower got?

Not at all. Karma is a proxy measure for something else, and since this submission proved "successful" by the karma measure, I'm happy to see that the submitter got the karma for it.

    Or are you just pondering the mysteries of stories
    getting picked up?

In part, yes, exactly this,

    I see it isn't the latter since in this comment
    http://news.ycombinator.com/item?id=4097127
    you indicate you've done research on that topic.

I have, and continue to do so because I have, as yet, come to no definitive conclusions. The investigations continue, even if only in a sporadic fashion,as and when I have the time.

    So what are you saying?

I'm saying that sometimes submissions fail to attract any attention, even if subsequent submissions achieve significant success. This is evidence that the current system is, by some measures, sub-optimal. I continue to investigate the dynamics (as best I can), and continue to work on something that I think will improve the situation.

The dissonance between the comments there (even though downvoted) and the positive reaction here is also of interest. I thought I'd share that, hence my comment.

Well in a very real way the system has to intentionally obfuscate the 'way to the front page' because being on the front page is so valuable, thus a target for manipulation.

What I'm saying is that as soon as you find a fool proof way to submit something and have it show up on the front page, pg will 'fix' it so that you can't do that in the future just like linking the discussion in a blog post was 'fixed'.

I typically read the site from an RSS feed, which means I see everything that is submitted that makes it to the rss feed, regardless of whether it is on the 'new' page or has fallen off. I don't know what percentage of folks do that. There are also 'super powers' for some users but those are for the standard sorts of things (like make sure a YC company looking for people gets to the front page for a while).

Can you say more about what "improving the system" would entail?

I'm not looking for a guaranteed way to get to the front page. I'd guess that most of the things that don't make it, deserve not to make it, in the sense that while they might be interesting, even if most HN people saw it, it wouldn't get many votes.

My concern is that things that are clearly of interest (in the sense that they do eventually do get the upvotes in a subsequent submission) sometimes don't get noticed and don't get the upvotes they deserve (by some definition).

Or something.

    > I typically read the site from an RSS feed, which means
    > I see everything that is submitted that makes it to the
    > rss feed, ...

Do you apply some sort of filtering, or do you literally scan everything by eye?

    > Can you say more about what "improving the system"
    > would entail?

My ideas are very badly formed at the moment, and I'd rather not try to put them in written form in an open forum. I wouldn't care if people stole them, I'd care more that they would look pretty stupid, because I couldn't explain them well enough. Yet.

> This is evidence that the current system is, by some measures, sub-optimal.

Aren't you assuming the value of an article is a constant? In practice, context changes and the value of an article goes up and down. Maybe today is a slow news day and 500 days ago was very busy. I wouldn't expect your average article about (say) Scala to do equally well on the day Facebook has its IPO and some day nothing interesting happened on. A controversial article about gun control is not going to be perceived as having the same value the day after a big massacre as leading up to an election or on a boring day in mid-January.

An interesting point. As it happens, when it was submitted 500 days ago it was a slower day (in some sense) than today. The "newest" page had fewer items per hour back then.

It's an interesting observation, though, that for some items the "value" changes over time. This particular item, I suspect, doesn't fall into that category. Context might be more important, but the analyses I do try to take that into account.

Eh, it's starting to sound like sour grapes again. There isn't an objective standard for value you can discover if your analysis is just a little better.

Sorry, but I can't parse that.

You're right that there isn't an objective standard for "value" (or at least, I don't know one, nor how to get one), but multivariate analysis can assign a number to an item which is broadly aligned with my intuition of the concept of "better". As a result, I'm trying to home in on finding items that my system predicts should be "of value" so I can read them, even if they don't make it to the front page.

Consider, I can train a Bayesian Filter with the page text of many submissions, and get that to predict what I'm likely to find "of value." But that means I don't see what the community might have found interesting but would be outside my core interests. These, too, are "of value" because they stretch me. I can't just train the filter with what makes the front page, because the evidence is that many things that "should" (by some definition) don't, and that would damage the training.

So it's complicated. I'm trying to find ways to improve the community and help me get more out of it (obviously) with less effort. I'm trying to help, but the hassle I get is just encouraging me not to make these observations out loud.

Based on this, your profile, and the rest of the comments in this thread I would suggest that whatever it is you're doing is vastly more effort than you're going to get back in added value.

This is a classic signal-to-noise problem in today's content generated websphere. I still don't believe we are anywhere near "solving it". More of my thoughts on it: http://www.techdisruptive.com/2012/09/18/we-are-far-from-sol...

A sample size of 2 is much too small to inductively reason.

Reply on news.ycombinator.com