Comment by freediver | Hacker News Reader

Comment by freediver | Hacker News Reader

Parent Full thread

The article mostly talks about IA (instant answers) which are notoriously hard. The recent advances in machine learning have made the technology more approachable, so startups like Kagi Search (disclaimer: founder) can also leverage latest advances in NLP and compete on this ground.

To give just a few examples:

Query 1: how many stars in the usa flag

Google: https://cln.sh/63sVzh

Kagi: https://cln.sh/bFEHsD

Pretty surprising that Google would get something like this wrong.

Query 2: when did moon explode

Google https://cln.sh/fUhdJS

Kagi https://cln.sh/5wDvXG

Both engines feature the same article but for some reason Google decides this is not fiction, and gives a (wrong) answer.

Query 3: do most rabbits have short or long ears

Google: https://cln.sh/JuOeqq

Kagi: https://cln.sh/BkZi6O

Both engines use the same article for source, but Google completely misses the context.

These examples show that a search startup has a chance to go neck-to-neck with Google and compete even in technology as sophisticated as instant answers. We invested considerable resources in the Kagi Search AI capabilities, discussed in some detail here https://kagi.ai/last-mile-for-web-search.html

What is mind boggling though from a product management perspective is that Google had nearly a decade head start and a cash purse of hundreds of billions of dollars to get this right.

To be fair, it is likely that the vast majority of queries are answered correctly, but only the outliers get the public attention. Also Kagi is not without its own share of silly mistakes too, but just being able to be considered in the same basket as Google is already a huge thing for us.

My favorite is, given that I have a baby and I am a trained scientist and I live in the US, I find myself converting milliliters to fluid ounces a lot.

Right now the Google Assistant will correctly transcribe the request to a Google search... Only for the search to interpret “ml” as “miles” and, faced with the discrepancy between the length and a volume, cube the miles. So I am expecting an amount that is like 1 oz because I am converting like 30 mL... and I instead get 4 quadrillion ounces (exact number is 1 mi³ = 140,942,994,870,857 + 1/7 oz because of course it's got that extra 7th in there what were you expecting from our ridiculous US system, haha).

Ten years ago Google Calculator used to work perfectly. I don't understand how they've broken it and why no one is responsible for fixing it.

Perhaps everyone who understood the codebase has left?

No one is going to get promoted for maintaining a calculator, so it rotted.

Clearly they should have put a chat app in it.

At least we can still use duckduckgo and use the !wa bang (wolfram alpha)

Sometimes even without the !wa, as duckduckgo itself often provides the result

It's not that broken. If you type "30ml in fluid ounces" in to google (i.e. the same thing you did 10 years ago) it works as expected.

Don't be "fair", instant answers have been useless misinformation in every instance I've triggered them. And they've been broken for years.

No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.

> No one at Google is responsible for these half baked and largely irrelevant widgets or wants to stake their career fixing them.

You're just ... wrong about this. There's an entire team of dozens of people (maybe hundreds now) focusing on this specific web answer feature. I personally worked on the team (not this feature, though).

I don't understand why people say things they know nothing about.

I think it largely stems from this very popular comment from a couple of years ago: https://news.ycombinator.com/item?id=19553840 From an external point of view certainly seems to explain a lot of Google's issues with the stagnation and death of many of it's products.

The fact that there have been plenty of other comments from Googlers to back it up since shows that, in some parts of Google at least, there's a grain of truth there. It might not be the whole story, many teams might be proud of their part, and many Googlers may not be focusing on promition and shiny new things, but that doesn't really matter to Google's users. What we see has been plenty of Google products being killed off, or left to rot. Now even Search has people complaining about it. Startups are getting traction competing against it. A decade ago that would have been unthinkable.

If you're right and teams in Google do actually care about the older, less shiny things they build then Google has a significant brand and reputation problem. If you're wrong then Google has a massive engineering culture problem. Either way, Google has a problem.

I think people say this because some problems stick out like a sore thumb and stay that way for months if not years. The line of inference is that there must be no incentive inside the company to fix them.

Isn't this just the software engineering equivalent of the fundamental attribution problem? My backlog is long because I have important stuff(TM) to address, whereas their backlog is sign of dysfunction.

> I don't understand why people say things they know nothing about.

Well, in all honesty I wonder how this exact same thing when reading the "answers".

I'm having visuals of someone repeatedly trying to throw something and missing the wall entirely.

Lets examine the original product implemented in wetware:

http://answers.google.com/answers/index.html

> "At the bottom of every question page we provide a link to answers-support@google.com. We encourage you to use this link whenever you see questionable content posted to the site. In your email, please provide information about the question, its ID number, and the reason you find the content questionable."

This one went through the wall!

You have no further questions.

And yet despite all of those people, the quality is bad and seems to get worse. A startup seems to be outcompeting them.

Sometimes industries or teams have effectively negative value, or their preconceived notions about how something works based on teachings from their field is wrong. This is the case for chiropractors (the whole field is useless / a net negative), vs back doctors.

We see this happening today with Google moving away from traditional and high quality techniques like direct keyword ranking, bm25, and pagerank and move towards lower quality methods based on hype such as BERT/other "semantic search" based on LMs, query rewriting, and using these dense vectors directly in pagerank (and this degrading it).

The amazing power of language models in certain domains (text generation) has unfortunately caused a proliferation of them in a place where they are still pretty bad (information retrieval and search).

Google is full of search chiropractors when they need search back doctors.

Instant answers is terrible, its someone’s interpretation of something rather then letter me decide.

Every instance? Come on, instant answers are bad but let's not exaggerate things.

Let me phrase it this way:

I'll happily say bye bye to instant answers across all search engines just to get my working search engine back (preferably with a blacklist).

Weird, two of those queries are right for me: Query 1: https://imgur.com/6cy3jBg.png Query 2: https://i.imgur.com/MLF8jpT.png

The third query is wrong for me too, though.

> The article mostly talks about IA (instant answers)

Not really. TFA is discussing the search results. If google/bing want to put instant answers then that is what will get judged. If your ML/AI is not good enough yet to provide natural language answers, don't make it the most prominent part of the search results.

Fwiw, for "how many stars in the usa flag" Google gives the correct answer for both logged in and not.

I got the correct answers from Google when I did the first and second search.

Same results as you on the third one though.

Reply on news.ycombinator.com