Customizing Web Search | Hacker News Reader

Customizing Web Search

notes.npilk.com

32 pointsnpilk11 comments

From the post:

> In the end, though, Google is heavily invested in users landing on their result pages so they can be served ads. So it's hard to imagine they would voluntarily redirect people away, even if they are clearly searching for a specific page.

The sad part of this is that Google, in the "10 blue links" era, actually did optimize for users clicking through to a result (and not immediately coming back to click another, which indicates the page wasn't satisfactory). This extended well into the early AdWords period where optimization extended to showing highly relevant ads in the sidebar or at the top of the results.

Things seem to have started to go wrong first with a series of design changes that made it harder to distinguish between ads and search results, and accelerated with the introduction of info boxes sometimes scraped from external sources and integrating results from other Google services (such as maps).

Since those changes, Google seems to have been optimizing for two use cases: either the user does a search and then stops (indicating they received the answer they were looking for) or they click through to a Google property or an ad and don't come back to the search engine. Clicking through to a search result is a lower priority.

The problem, as I see it, is that the first case of searching and then stopping also indicates "these results are garbage, why even bother" which dilutes the signal. You can potentially distinguish the two cases of 'completely satisfied' vs. 'completely unsatisfied' with a bunch of other weaker signals (clicking through to the long tail of SERPs, trying a bunch of search variations one after the other), but most users today won't try further no matter what (Google, ironically, has all along been training users that the first result page of the first query they try satisfices).

In short, Google has been shooting themselves in the foot by muddying the strongest signal it had for search quality.

I imagine that Google's solution to this involves behavioral tracking/profiling and analysis as in "is the user in the 'easily frustrated' bucket" to amplify or discard the weaker signals appropriately, but clearly whatever they're doing isn't working as well as the unambiguous signal did.

"You can potentially distinguish the two cases of 'completely satisfied' vs. 'completely unsatisfied' with a bunch of other weaker signals (clicking through to the long tail of SERPs, trying a bunch of search variations one after the other), but most users today won't try further no matter what (Google, ironically, has all along been training users that the first result page of the first query they try satisfices)."

Arguably, Google's training of users has worked. Bing offers few results, with no option to receive, e.g., 50 or 100 per page. When Brave announced their new "web search engine", they offered only one quarter page of five results max, regardless of whether on desktop, laptop or mobile sized screen.

It is perhaps worth considering where ads (paid "results") appear. Pre-Google, AltaVista used to mix them into the "organic" search results. Google separated them out. How many ads per page are feasible if paid results can be mixed with organic versus if paid results must be separated out.

Even more worthy of consideration perhaps is that once users are trained to only look at the top of page one, then that becomes more coveted real estate for ads, and arguably allows the price of the ad to rise as it encourages more competition for the top position, in most cases the only place eyeballs will be found.

I didn't know about the `!` operator, that's so cool. Thanks for sharing your whole approach. My dream search engine had 3 levels, you just added a 4th.

1) I'm feeling lucky (I will use `!` for now, but will look at setting up something like yours)

2) Search within bookmarks

3) Search within preferred domains (ideally the list is autogenerated from roots of bookmarks, but would like to explicitly include/exclude)

4) "Rest of the internet"

2-4 could be shown in one page, it would just affect the ordering.

Side note: A quick fix your issue with names messing up things like where the wild things are – check for a single quote mark at the beginning and treat that like `"Where the wild things are" !`. So, `"where the wild things are` would trigger the feeling lucky result.

I googled a bit more and found these links -

- https://duckduckgo.com/bang

- For the Instant Answer API: https://duckduckgo.com/api

An example of the API request with !bang: https://api.duckduckgo.com/?q=!imdb+rushmore&format=json&pre...

This returns a redirect key of `"Redirect" : "https://www.imdb.com/find?s=all&q=rushmore"`

For wikipedia query, the returned redirect URL is: "https://en.wikipedia.org/wiki/Special:Search?search=tom%20ha..." which redirects to => "https://en.wikipedia.org/wiki/Tom_hanks"

One thing I really liked about Firefox was that it used to do two different types of searches. The search bar acts as it does today, but the nav bar used to redirect to the first result. This was very useful because sometimes you just know what you're looking for (as per the OP's example, "tom hanks imdb"), while other times you are going to do some digging.

Now that I'm using Kagi as my main search engine, I wish I could tell Firefox to use another search engine in my nav bar, and when I'm incognito. Mozilla used to let us modify a lot of things (via the UI, or the about:config panel), but from a few years it has been removing lots of customization options.

> One thing I really liked about Firefox was that it used to do two different types of searches. The search bar acts as it does today, but the nav bar used to redirect to the first result.

You can actually do this yourself if you set your search engine to DuckDuckGo and prepend your search with a backslash (\, e.g. "\bats" or "\tom hanks imdb" in your case -- works great). I discovered this by accident; it's very useful.

Nice! I had discovered that adding a '!' does this on DDG, but didn't know about the backslash - cool tip.

I will say that it's nice to be able to add the exclamation point at the end, instead of putting it at the start like the backslash. That way, you can decide after you type your query whether you want to go directly to the first link or see the full results.

Nice idea! And your search bar to test it out worked well for me.

I'm in the process of collecting local HTML files from all kinds of pages I find interesting (I use an FF plugin called WebScrapBook for this). Eventually, I want to be able to have some locally running search bar in my browser (thanks LAMP stack) that first checks my local library of information, and then gives me the option to search online with DuckDuckGo.

Re: iOS Safari default search conundrum: You could use something like Launch Center Pro[1] to pass queries to your search URL in Safari. Minor convention change (eg. initiate search from LC vs. directly in Safari) but it's still at least one less step.

[1: https://contrast.co/launch-center-pro/]

Thank you.

I’ve been noodling with this sort of idea for years. Never got around to it.

Thanks for a simple starter. I will be modifying this for my needs at my leisure.

Great! I'm glad to hear this is helpful.

Reply on news.ycombinator.com