Comparing quality of search results: Google vs. Bing

startpage.com

3 points·karinakarina··4 comments

I realize this is not some high-stakes, formal research project or anything. That said, I don't know what to make of the 'results' because of the design of the experiment

What I'd like to see the author do is this:

1) write a script to conduct the searches on Google and Bing, combine and save the top 20 results of each search to a spreadsheet, in random or alphabetical order.

2) using the spreadsheet alone, follow each link and rank the order the author thinks is most relevant.

3) compare the author's rankings with Google's and Bing's actual results

The fact that the author is judging search results while knowing which engine produced them... it makes something that is subjective to start with even more subjective.

That's a fair critique of the experiment. I like your method. Have you thought about doing it?

Yes, but I almost certainly never will get actually around to it ;)

You're not wrong (I am the author of this piece), but with time constraints (and my inability to write such a script) I modeled this "study" with that in mind.

It's why I chose terms in which I knew what results I wanted to begin with, and came up with the criteria before searching for them.

To run your study, I would have chosen to search terms I knew nothing about, and looked for results that better educated me. Through my method, I knew by searching for "Rust" I was looking for the programming language.

My method was not meant to be scientific, but I used as much of the scientific method as possible without the ability to a blind or double blind study.

Reply on news.ycombinator.com