Comment by carschno | Hacker News Reader

Comment by carschno | Hacker News Reader

Parent Full thread

This is exactly the problem of "general purpose" search engines as mentioned by GP. In classic information retrieval theory, a search query is a textual representation of the a user's mental search intent. A search engine's task is to interpret that textual representation in order to serve the user's need.

Basic NLP starts with simple techniques like stemming, synonyms, typo correction etc. But many users expect more, e.g. contextual disambiguation or broader synonym/similarity resolution. These users would see a search engine as dysfunctional if it could not even find pages in which a plural form or a spelling variation of a query term occurs.

However, there is an inherent contradiction to those users who want to find a specific string. Google, Bing and others apply complex algorithms ("weird Black Box AI") in order to interpret the user intent and the meaning of web pages in order to optimize towards the majority of internet users. To some degree, they do try to also serve users of (large enough) minority use cases in which the user feels like they knew exactly what words (strings) they were looking for. But again, the different use cases pose contradictory interests that are probably not resolvable in a "general purpose" approach.

I follow that, but I still have to ask: Why does it often ignore the quote marks it suggests to me? It's supposed to search for that exact string, but it never does and shows the same rubbish regardless. The quote marks should resolve the contradition between synonym searching and specific strings and they don't for some bizzare reason

Because their products are a complete fucking MESS.

Quotes used to be an exact search, but it now overrides that basically everywhere and continues to search for what it thinks you mean (and has for like a decade now). Even though the "Missing: ___ ‎| Must include: ___" popup just wraps that word in quotes in the search. Which is infuriating, since they know damn well adding quotes won't actually search for that word.

In reality, you need to click "tools" -> "all results" and change it to "verbatim" which does at least still appear to work... for now.

Today I learned... Thank you!

pose contradictory interests that are probably not resolvable in a "general purpose" approach.

How are they not resolvable? If Google respected quotation marks (as it used to do, in the old times) that would be enough to accommodate both kinds of users (or rather both kind of search needs - I'm the same person and do both).

Came here to say this. Quotation marks were hugely useful.

>Basic NLP starts with simple techniques like stemming, synonyms, typo correction etc. But many users expect more, e.g. contextual disambiguation or broader synonym/similarity resolution. These users would see a search engine as dysfunctional if it could not even find pages in which a plural form or a spelling variation of a query term occurs.

Any morphological variation would already be covered with stemming alone, so this particular point doesn’t speak in favor of more advanced interpretations, does it?

Having an path which uses advanced AI powered tools doesn’t necessarily imply one should drop any ability to search with more technical query interface for advanced users. And it’s doubtful big companies wouldn’t have the budget to maintain even two completely distinct tools, would it be required.

Reply on news.ycombinator.com