Comment by nostrademons | Hacker News Reader

Comment by nostrademons | Hacker News Reader

Ex-Google search engineer here, now using hosted ElasticSearch extensively in my startup.

This is a really good overview. If there's one part I want to highlight, it's that you should expect to spend a lot of time fine-tuning your ranking function for your particular product & corpus. The default ElasticSearch ranking function kinda sucks. It was changed in ES 5.0 to Okapi BM25, which is the current academic state-of-the-art in non-machine-learned ranking functions. However, search is one field where the current academic state-of-the-art is at least a couple decades behind where things are in industry. When you use a service with good search that just works, chances are that there's been a lot of engineer hours devoted to identifying exactly which signals are most useful in your corpus, how they relate to subjective evaluations of relevance, and how to clean them up so that noise doesn't dominate the signal.

These signals are tightly tied to the corpus you're working with. One slightly unpleasant surprise striking out on my own was just how little the stuff I learned about websearch is relevant to working with different corpora (though I guess I should've predicted this, having seen how many different teams across Google Search operated and how all their signals differed from core websearch). I'll reiterate the article's point about process and method being more important than any specific function or algorithm; the most useful stuff I learned at Google was actually a process for taking a vague domain where I'm not sure what's out there, and then learning how to turn that into a system for getting useful information out of it.

And then once you've got users, you probably want to feed usage data back into it through some sort of machine learning. In that order, though; with all the hotness around AI, it's really tempting to think "Oh, I'll just read the latest LtR papers, Mechanical Turk a bunch of query evaluations, and feed the data into the algorithm", but without a deeper understanding of the particular corpus you're working with and how users want to access it, it's unlikely an initial LtR system will perform well enough to attract enough users to bootstrap the system.

> When you use a service with good search that just works, chances are that there's been a lot of engineer hours devoted to identifying exactly which signals are most useful in your corpus

I imagine at a place like Google, your "corpus" is just about everything under the sun, as are your queries. (i.e. less chance for a subject-specific tuning)

What happens then?

Google's search departments were for years organized by the type of content being indexed, i.e. Web Search, Image Search, Video Search, Product Search, Book Search, Blog Search, Discussion Search (RIP), Google News, Scholar, etc. For a long time these distinctions were apparent in the UI (they were de-emphasized after I left, which IMHO was a very good thing), but the differences go way down through the search stack, with different indexes, different ranking functions, and in some cases different ingestion mechanisms.

As for what happens when these become inadequate for the queries users ask...well, buy Metaweb and rebrand it as the Knowledge Graph. ;-)

> I imagine at a place like Google, your "corpus" is just about everything under the sun, as are your queries. > What happens then?

You accept the fact that a good search engine is inevitably going to be a multi-layered beast that requires constant effort to improve, with heuristics, machine learning, magic constants, special lists of words, and regular expressions.

After that, it's just a matter of picking a commercially useful objective function ("instant search", "query autocomplete", "maximize user satisfaction") and pointing large teams of well-resourced PhDs at it. Probably a whole weekend at least.

If you can identify categories of queries or documents that interest people, you can treat tuning that part of the search engine as if it was a new product.

what does your startup do?

Cross between an RSS-reader and a search engine - it lets you subscribe to a topic instead of a site, so that you don't have to manage hundreds of subscriptions, and if a new discussion pops up somewhere on the web that's relevant to your interests, it'll find it and let you know. As forums get boring or off-topic and new ones spring up, it adjusts automatically, so you don't need to do the "Does anyone know of other places on the web like X?" dance.

Still under development, but drop me an e-mail if you (or anyone else reading this) is interested in beta-testing. I'm starting out e-mail first, so the initial UI is just that you get a daily digest of links & snippets to threads related to your interests.

Ah, so something like ElasticSearch percolater queries but applied to the web.

Yep, pretty much. The percolater queries are fairly handy in this instance, and one of the things we didn't really have at Google.

I just read https://stackoverflow.com/questions/21536599/what-does-perco..., and was strongly reminded of Google Wave's "search for documents that haven't been created yet, and they show up when they appear" functionality.

Without trying to get too much off track, I gotta say that it would be so nice to be able to use the original theme data used in Wave (very obviously sans branding). Wave In A Box is... blech, in terms of design, I have to admit.

Google News topic search has an RSS. How is your product better ?

Google News concentrates mostly on news sites, which have an economic incentive to generate content that emotionally grabs you and spreads virally (i.e. clickbait). I get the sense that people are getting fairly outraged at all the outrage in the MSM & popular blog-based sites - I know I am - and are fatiguing of that. I'm concentrating on forums, blogs, and other user-generated content, which has no economic incentive to reach millions of people and so can be both more authentic and more informative.

For example, my current video game obsession is Factorio, which has about 200,000 users and will likely never appear in a major news story, because it's un-economical for a news outlet to write a story that has an audience of at most 200,000. Despite this, there are 4 active forums dedicated to it, which generate a few hours worth of interesting reading each day. This content is a lot more interesting to me than anything that appears on Google News, but it's a lot less interesting to the millions of other readers of Google News. But the beauty of computers is that we can match content up to precisely the users that care about it.

> But the beauty of computers is that we can match content up to precisely the users that care about it.

In theory. I often wonder about pathologically impossible-to-query statistics like "who is snoring the loudest right now?" "show me a global map of everyone waking up right now (and a graph tracking how many people are waking up per second); and provide a second-to-second pinpoint of the person who feels the most refreshed." "what is the single most relevant set of webpages for this highly obscure, domain-specific query?" etc.

Heh, it sometimes takes me actual effort to calm myself down about the fact that, beyond a certain threshold, we literally cannot collect enough entropy (data) to direct a database to the most relevant results - and that we similarly won't connect users to the data they're most interested in, beyond that point.

I do totally get what you mean though.

Very interesting. Indeed, Google News is silent on english-content on Factorio. But then how is your product better at finding specific forums ? Furthermore, why wouldn't I just add this particular forum's RSS to my reader?

Are you considering adding sentiment analysis/clustering as one feature? You know, to let people read about different opinions, most often, they only need to read one post in each cluster.

Reply on news.ycombinator.com