coderrr

That's a great idea. I think it would probably work, but I can think of one potential case where it would introduce some complications.

If user1 sees 'pg' as the submitter and user2 sees 'blah' user1 might comment something which gives away that it is actually pg to user2. Now user2 might realize this and act on their pg bias.

Can you think of a way to prevent this?

I agree... no meta questions would be compared.

You can get around the problem that you mention by only comparing votes which are made from the main page and not from a "follow" page.

I think you raise an interesting point. There are actually TWO problems here. One is that there is possibly a bias based on the content submitter/creator.

But the other is that, people who are followed will naturally get more votes, just because they get more views. Is that a bad thing? I dunno. But what if it got to a point where there were enough different people being followed that the only way onto the first page is by being one of them. That kindof sounds like a problem too.

This is true. It's a heuristic our brain uses so it doesn't have to reevaluate things every single time we experience them. Whether or not this is good or bad is very debatable and depends on the context.

The heuristic is something we evolved through evolution to help us make choices quickly and survive better. For example we know to just trust whatever the group leader says when there is danger rather than having an open debate every time he speaks because that could get us killed.

In the case of HN, does it matter if we have to fully evaluate the content every time before we vote (w/o knowing who wrote it)? I don't know. I think there are probably cases on HN where that heuristic is still to our benefit. I don't think theres black or white here, the costs on both sides just need to be weighed...

Also I think everyone should realize there is a small parallel with the ad hominem attck which is DH1 from PG's post about how to disagree with people (http://www.paulgraham.com/disagree.html). In this case it's exactly the opposite, vote up because of who it is instead of based on the merit, as opposed to vote down (or attack) based on who it is as instead of based on merit.

This is true. But the experiment could take this into account. It could only count the votes which occur directly from the main page.

How is it impossible to measure? It has nothing to do with how his content quality ranks compared to other people. Since he is writing both the content under his name and the content under a different name, given that he doesn't know which one he's writing under until after he writes it, the quality should be at the same level (on average).

If the quality is the same but the points are different (between his real acct and anon acct), then you've found a bias.

I'm surprised no one has mentioned this yet. If you are going to do this experiment, it needs to be _double-blind_. Meaning PG cannot know which username his post is going to show up under until after he has written it. Same goes for all the comments he writes (if you're going to measure those too).

If PG knows before hand what name his content is going to show up under there is a high likelihood of a bias on his part which he would be _unable_ to ignore even if he consciously tried. This is standard psych experiment stuff.