Hacker News Reader

Comment by comeonbro | Hacker News Reader

I would propose another mechanism: even the free-tier models have already completely saturated what most people are capable of appreciating.

View on HN

Google's free AI responses are moving the bar on what I expect from search. But maybe not in the direction you think.

It's so bad. 3 out of 4 answers for me are wrong.

Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.

God damn. I respect you for being honest, but… god damn. That’s a pretty low bar.

Just in the last few days I had an example where the Google Search version of Gemini had a much better and more comprehensive answer, by far, than ChatGPT. It had to do with Amazon’s use of Data Matrix barcodes on shopping bags. Gemini was able to fully describe how they’re used in Amazon’s logistics chain. ChatGPT essentially said it didn’t know. Your preconceptions may not be accurate.

They might be, but still, I’ll be difficult to convince that Google’s shitty-ass 10B params instant answer models are good at anything other than cookie recipes and producing farts real fast for real cheap.

For what it’s worth, any time I see that feature, it’s all farts.

It depends on what you’re searching. If you only ever need really basic information like keyboard shortcuts or “is it & or &&?” then it’s pretty good, and definitely faster. And if it gets it wrong, then the shortcut doesn’t work or the code doesn’t compile and I search again and try the links.

People are really slow to update their priors after bad experience with an early version

Yes. Which means they’re really not equipped to deal with what’s going on right now.

It's just a google search but you can explain what you're searching for. AI summary may contain misleading details if you read it and take it literally, but 90% of the time it gives me pointer for the thing I need and I don't have to scroll through results that often could be just as misleading if not more.

It's demonstrably not. This summary, somehow, often fails to search, or fails to read what it searches and gives you the opposite answer. You shouldn't trust it.

In their world, it gives them the wrong answer that kills their dog, but it's fine because they never know because they don't look anywhere else. The dog dies, they're sad for a while, get a new dog, get on with their lives.

There's no reason not to trust it because since they trust it, they don't percieve anything wrong. They just take it and whatever happens happens, no annoying conflicting info and certainly none of that old grampa thinking and working and giving a shit about anything.

A large portion of the time I google something that I know a lot about the AI summary is misleading if not outright wrong. Frightening when you consider how many take their so-called AI as gospel.

To give a specific example of where I've seen this utterly fail... A few months ago one of my pets had an unexpected reaction to a new medication. It was late at night, well past our vet's hours. Google's AI response was basicaly "They are going to die, rush to an emergy clinic now".

But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.

Realize that all the Google thing is doing is having an LLM summarize the search results in the page. If the page results are crap, the LLM content will also be crap. It's the LLM version of the "quick result" boxes that would shove wikipedia in your face.

If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.

But it did not? It said something completely differemt then the resources.

The other day I asked which trains stop at a particular station and it pointed me at a map that isn't real.

It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.

It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.

It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.

It is better than it was but it is not adequate yet.

Or maybe needing.

And I would propose another: the supply hasn't matched the demand. We are still compute hungry and cannot afford the 90% to let their agents run 24/7.

Reply on news.ycombinator.com