What metrics would you use? I would be fascinated to know.
What metrics would you use? I would be fascinated to know.
Then prescriptive.
The idea would be a framework so someone can make (or grab their current data) the test different engines for different search terms against the data and mark which is correct or better.
You don't have to 100% agree what search result is better. But to just get to the stage where you can discuss that would be a great start.
And I suspect not agreeing what is better is not as contentious as one thinks.
I also think Google does search real well. So use that for a set point of the truth.
Color me interested! There’s so many layers to measure: from natural language understanding to pagerank-style relevance ordering, filtering duplicates to presenting the different permanences of content (breaking news vs books on the same topic).
Yep. It's interesting. How do you deal with versioning. The same document 10 times on a CMS might overpower the search results. Does uniqueness rank higher? There you need a human judge at which search engine / search engine defaults is better.
> natural language understanding
I'd like to know if any search engines do this. I feel like it's out of scope. If they use the words "natural language understanding" I think they are running a scam and mean statistics.