Hacker News Reader

Comment by andrewhillman | Hacker News Reader

Yeah, this sounds all well and good in theory, but after visiting thousands of sites over the years, it might be a better idea to help engineers build a search engine for their own site/data first. I can't recall many websites that have amazing search. It's a problem when I have to use google to find what I want on xyz.com because if I go search for what I am looking for on xyz.com I cannot find it even if I know its on that site.

It would be so nice to go to xyz.com and actually find what I am looking for in under 1 second.

View on HN

I precisely ranted about this issue as a way to challenge Google: http://blog.databigbang.com/letters-from-the-future-challeng...

I think we need to optimize local search engines and aggregate (in some ways) all this into a global search engine.

As a site owner you can pay Google for this:

https://www.google.com/work/search/products/gss.html

I used it several years ago to power this site search:

http://www.poetryfoundation.org/search/articles#qs=ginsberg

It works pretty well (disclaimer: I'm not sure this is still being used on that site)

Yes. It's been around for years. I have used it but this misses my point or proves it. Either way, sites need to figure out how to power own search.

This is customized for the site to 'power their own search'. The query sent to Google is based on the keyword but also on the article type (in this example) and then the xml that is returned is post-processed to have a custom render per article type.

Since we had all the data the results from Google were compared to a local search I setup in Sphinx (http://sphinxsearch.com/) and the Google results were more relevant and it was a lot cheaper to deploy.

I'm pretty sure Elastic is on the right direction in this regard, don't you think?

I think "full-text indexing" is a very different problem than "search."

Full-text indexing (what ES provides) has been around for almost forever, ES just does a way better job of productizing/delivering it.

However, Google is far more than a text index. Ranking, currently is still very difficult and requires messing around with facet and weighting parameters.

I don't think so. I just went to their site and the first case study i opened was from theguardian.com. I went to theguardian.com and went to do a search. Guess who they are using to power their search function? Google.

In my opinion which means nothing, sites need to figure out how to power their own search. Using a third party isn't going to work for most. Maybe people need to focus on building custom architecture that indexes the data in a more structured way rather than cobbling systems together that ultimately hinder search efforts when its time to get the user what they want. I don't know the answer but somebody eventually will. Maybe wordpress will create a powerful search for all those wordpress sites.

Maybe I'm beating a dead horse but I feel like you could create a pretty compelling search engine using ES. I used it this summer at Goldman and it seems like it will really change the landscape... It's just so fast. I mean full-text-indexing seems to be a pretty integral part of search. Maybe it's just the first step, with the second step being a well written ranking function, but that's just my thought.

Elastic is more like a quick way to search through huge amounts of text, not a way to rank which hits are more relevant.

It lets you define your own ranking /scoring function I think

Reply on news.ycombinator.com