Hacker News Reader

Comment by wumpus | Hacker News Reader

Yep. A lot of folks appear to be pretty confused about retail search engines verses the actual index -- Google, Bing, and (at lower quality) Yandex are the only viable English-language search indices right now. Almost every search box you find on the Internets is powered by one of the three.

View on HN

Did You check Cliqz already? Independent search index: https://beta.cliqz.com/

There is a lot of technical details about how it’s built on the tech blog: https://0x65.dev/

(Disclaimer: I work at Cliqz)

I've been using your service as my main search engine on desktop and phone for more than one month (I made an xml file to load it as search engine.) It's okish but not very good at dealing with misspelling, with news and with images. Sometimes I touch the Google button at the end of the page to find what I'm looking for. I hope you keep track of the keywords that go to Google and use them to improve your index. Anyway, it's a good service, thank you.

Hey, thanks for using Cliqz. We're working hard on improvements for each one of the things you mentioned (in that order as well) - so things will definitely get much smoother over the next couple of months.

I looked at Cliqz a few months ago, and the crawl didn't seem competitive with Google or Bing: https://news.ycombinator.com/item?id=21676600

You mean they are not as competitive as a billion dollar company 20 years in business?

My grandparent was saying "Google, Bing, and (at lower quality) Yandex are the only viable English-language search indices right now", and my parent was asking whether Cliqz counted. I think the GP is correct and Cliqz doesn't.

Hey, I am working on Cliqz Search [0] and would be very thankful if you point me to quality issues. I use it daily myself (in English) and will make sure those issues are addressed.

[0] https://beta.cliqz.com/

If you click through my link above (https://news.ycombinator.com/item?id=21676600) my process went like:

1. Search for a blog I follow

2. Look at the snippet for the blog's homepage

3. Figure out how long ago the page must have been crawled to get such an old snippet

I only tried a few, but they were months to years out of date.

This is absolutely fantastic! A REAL alternative not just a search collector like ddg which are endorsed by Google or Bing as the traffic and ad opportunities remain unafected.

Maybe unrelated but what are the server costs and hardware used etc for such a thing?

SwissCows has its own indexer and I believe they don't rely on any external search source like Bing. They should push for better marketing, less nationalism & better UI like duckduckgo and they would go pretty far imo, their results are often better than DDG

edit : my bad, actually they do use Bing in addition to their own crawler

Yacy. Legendary yacy. Still around. Also gigablast

Qwant has their own index (or atleast parts) IIRC.

> Google, Bing, and (at lower quality) Yandex are the only

And DuckDuckGo.

DuckDuckGo is just a metacrawler. Depending on the search, it uses either Yandex or Bing.

According to DuckDuckGo,

> To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).

Do you view that as misleading/incorrect?

Yes, as we know nothing about the percentage of traffic which is coming from google, bing or any of the remaining 398 ressources. I can only assume the contribution is abysmal, given the almight of the one trully.

I read that as "Over four hundred sources for instant answers (the big box at the top) etc. What you usually think of as search results ("traditional links") comes from multiple partners, mostly from Bing (and never from Google, which only leaves Yandex and niche providers)".

That fits well with what GP claimed.

Doesn't it maintain it's own index on certain terms based on user actions/feedback that changes the bing results resulted? I thought there was an improvement feedback loop.

That doesn't seem to be the case; the only reason they use r.duckduckgo.com is for redirecting without the REFERER according to their docs, so there doesn't seem to be a way for them to do that.

No. Ddg gets most of its index from bing.

Reply on news.ycombinator.com