Hacker News Reader

Comment by rolosa | Hacker News Reader

> i truly have tried to use llms before, just to try it out. nothing i was going to publish as research, but i really couldn't and it left me never wanting to use an llm again

> i don't know how more people don't experience this, but staring at an llms output be confidently wrong eventually to get there in the process, or looking at the commands its running and see some really stupid things its doing drives me crazy to the point i will just stop it before it answers, i cannot handle something being that confidently wrong (even though it'll probably still get there in the end)

When people say stuff like this, that they've used LLMs before and hated it, and then list off a bunch of stuff about how they're confidently wrong and all this other stuff, my only question is when? When was the last time that they used an LLM? three months ago, six months ago, one year ago, two years ago?

View on HN

I'll call myself a late adopter. I tried early models and wasn't impressed, work has made some things available but wasn't forcing us so Ive been keeping my fingers on the pulse for a bit but watching from the sidelines. In October I decided to give it another shot and Opus 5.5 has blown me away, I've written a whole PR for a sparsely documented project and was like write the PR description for me with like one sentence of background, no Claude.md, no context other than the repo and it completely nailed the description. Most of my apps are in the tens of thousands of lines, so nothing huge but still the capabilities of the frontier models is crazy good. They've come a long way from even a year ago.

This is always the first response to criticism of LLMs, and it's starting to feel banal, because it's a counter that you can pull reflexively from your back pocket at any time without engaging with the actual content of the criticism. I'm not saying it's wrong necessarily, but rather that the added information value of "have you tried GPT Cosmica 15" in a discussion on LLMs is approaches zero. Because it's such a generically applicable response.

Well, it's just a rephrasing of "I don't share this experience at all."

And it's an easy explanation for why one person can be getting "confidently wrong" results still, and another person hasn't experienced it in a long time.

I'm one who hasn't experienced it lately. I haven't even thought about it in quite some time. Maybe a domain thing, maybe I'm not doing fancy enough work, maybe that guy is using some shitty model.

There are so many other explanations.

Maybe this person is working in a different domain where these tools work less well. Maybe this person is a particular kind of expert where they detect incorrect or suboptimal output that would be difficult for others to detect. Maybe the mechanism they use to evaluate the output is different than what other people use.

Seconding this. It's come to the point where I tried out gemini (argon) and saw it was giving me a wrong answer and was immediately shocked how far behind google obviously still is.

Well, no: it’s a shorthand for “I don’t share your experience AND ALSO your experience is invalid.”

Well yeah. If your exposure to AI is Google ai search then I don't really care what you have to say.

Every time I get pulled into a conversation with someone who is an extreme AI skeptic in this latter half of 2026, it’s eventually revealed that their only experience is one of:

1. The AI box at the top of Google

2. The voice assistance on their phone when they tried it a long time ago. Some question they asked Siri in 2025 and got a bad answer for.

3. ChatGPT free tier.

For a while you’d find someone who got assigned an AI tool at work but their administrator limited them to the cheapest model for cost savings, but most companies are wising up.

I also talked to a developer last month whose company had never let them use anything beyond Sonnet. When I told him I used Opus for everything his eyes went wide because he assumed I was spending thousands of dollars per month. However now that the latest Sonnet is very good, even this category of exposure is decreasing.

I think in 2027 as advanced models become more common we’ll see less of these claims about AI not working well when they tried it. There will be holdout group of people who refuse to try anything new, but the world is moving on.

"You haven't spent enough to get the point" is a weird bar to set for a growing industry.

If we steelman their argument for a second I think they aren't setting any bars. LLMs go from google search to Opus/Fable, and people praising/criticizing them are mostly using opposite ends.

I was setting a bar, but I think it’s a reasonable one:

If someone is writing confident takes on a topic, I expect them to have some experience with the topic.

Writing confident dismissals of something after not getting it or not really taking time to learn makes for uninteresting writing.

That’s a perfectly reasonable bar to set for someone writing hot takes about AI.

If I’m going to read someone’s opinion on an a growing industry, I expect at minimum that they’ve given it an honest try.

These articles where someone tries AI once, doesn’t get it, but also thinks they understand it better than everyone else are everywhere right now.

They’re not very interesting because the author hasn’t taken enough time to understand the subject they’re writing about.

Microsoft copilot is the other one. Copilot has done so much damage to AI's reputation I think AI companies probably despise Microsoft.

Last week and tonight I tried asking a frontier model a couple of basic questions about a language with 60-70 million speakers with that I'm an expert on. It gave absolutely brutal, confidently wrong answers. It was completely confused and about the basic structure of the language.

Which model? What language? What questions?

Do you have a share link to the session?

Why is every confident claim about a frontier model being brutally wrong about something presented without any details or evidence? It’s so trivial to share even details like what language was used, but even that had to be obscured for some reason.

This is an interesting request. Every claim of LLM being wrong has to be proven.

How about the opposite? Share a normal session (by normal, I mean something longer than, say, 4 turns, not absolutely trivial) where LLM doesn’t do a single silly thing. Including in thinking portions.

> This is an interesting request. Every claim of LLM being wrong has to be proven.

Providing even a minimal set of details is important because “an LLM” could mean anything from the Google search header to Fable or Astra.

It’s like someone making a claim that they tried a diet and didn’t lose weight, therefore all diets are bad and shouldn’t be trusted. The details of the diet, how long they tried it, how strictly they followed it, and the rest of their situation are pertinent.

There is an audience who demands content like that: People who want do hate diets and believe they don’t work will eat that commentary up because it levels up the belief they want to hold. Everyone with critical thinking skills or experience with dieting knows that one person claiming diet failure means nothing.

We know diets are a set of tools and they’re not perfect, but they are used successfully by people who can understand their limitations. Now replace diet with LLM and my statement is the same

> How about the opposite? Share a normal session (by normal, I mean something longer than, say, 4 turns, not absolutely trivial) where LLM doesn’t do a single silly thing. Including in thinking portions.

How about Terence Tao sharing his ChatGPT logs having ChatGPT work with him through understanding the Jacobson conjecture?

https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed...

just starting to read all these; that specifically was both Claude Opus 4.8 and GPT 5.6 Sol and GPT 6 Astra

the main point of the article isn't this and a lot of people seem to be focusing on it though, it's me describing why i personally don't want to use an LLM

and what i'm describing specifically there is the concept of the CoT and the tools it's executing. i see it doing something stupid and my instinct is to stop it, it does usually eventually get there in the end

as i point out right below, llms are super powerful. i'm not arguing that. i'm arguing that we should stop disregarding ethics for marketing and also generally have fun ourselves without llms in personal projects (not work) specifically.

My wife is a historian. She and her colleagues are generally skeptical of AI but have also given these tools serious attempts at improving their work. In the last six months she has made serious attempts to use these tools for paliography and transcription, identifying likely productive archives to visit, and for synthesizing the state of prior research.

In each domain the tools have done some things well but also confidently reported simply incorrect things. They misunderstand the development of these archives and recommend unproductive visits. They invent content when transcription in difficult.

They are still confidently wrong often. It’s harder to spot with Opus 5.5 effort medium+ but AI search overview is rife with this

I think the places the average person starts out with AI do look like this. Most people are not paying a $20 a month Anthropic subscription just to try it out. They're using free chat apps, reading the garbage llm summary Google puts at the top of every search page, dealing with the low quality tools foisted on them in their email app or word processor.

Have all these gotten a lot better in the last year? Sure! But there's still a several-orders-of-magnitude difference in quality between whatever garbage tier of GPT-6 Luna you can use for free on `duck.ai` (just for one example) and Opus 5.5. Or even GLM 5.3 Flash. It's a way bigger difference than the improvement in ~free models has made up for in the last year, even though those improvements have been substantial.

I think a certain sort of person, if they do try to use something better along with a coding harness, as also immediately inclined to read along with the reasoning traces - and understandably gets freaked out by the nonsense it contains. That's a mistake, I think; not just because models behaving "reasonably" doesn't depend on the rationality of their reasoning traces, but also because I think if you transcribed my brain activity into a serial stream of text, I'm not convinced it would be more coherent.

> garbage llm summary Google puts at the top of every search page

I assume they don't want to pay for something that would make search results better rather than worse.

> When was the last time that they used an LLM?

And also, which one? There is a very large difference between Google AI mode in the browser, Claude Sonnet, Claude Opus, running Qwen 3.x locally...even running the same version of Qwen 3.8 27B locally with different configuration can lead to wildly different results.

I mean... this has been the most common response to pretty much all LLM criticism since GPT 3? I have yet to see any substantial critique of LLMs that isn't immediately met with "but have you tried the latest models? they're so much better now!"

The article is also clear that it doesn't really matter whether the thing "works" or not - the process of using it is no fun for the author, so why would they want to use it for things that are supposed to be fun?

The only thing that changes over time is that you need to know more about the topic in order to be able to spot it.

I've been on leave and since my work subscription would go to waste otherwise I've decided to go completely silly and vibecode (truly vibecode) a big project from scratch. Opus 5.5 with xhigh reasoning was repeated firing off hour-long test runs, of tests it had written itself from scratch on code it had written from scratch, until I finally noticed and said "hey, couldn't we run some of these in parallel?" and then the execution time dropped to under 10 minutes. So yes, even recent models do utterly stupid shit pretty frequently (this isn't the only example from this project, but it's the easiest to explain)

How long until people genuinely don’t know how to check if the LLM is wrong?

(I use LLMs for lots of things. They’re useful for lots of things. Gemini 3.8 also feeds me lines of complete bullshit every couple of days.)

Why does that matter? The argument still stands, they are still dumb. We learned some ways to bypass the dumbness and automate some major workflows, and we still see them fail at random things.

It matters the same way saying “solar panels are super expensive” has been historically true not too long ago, but prices have fallen so rapidly it’s simply no longer a correct statement - even if you will occasionally come across an overpriced offer here and there.

I feel like we're arguing over semantics. What op quoted, referring to what I imagine were things like: confidently running wrong commands, answering something without looking it up, getting stuck on long scripts and not self-correcting all absolutely still happen, so the statement isn't incorrect.

I would also wager that OOP knows LLM limits quite well, and a more interesting discussion would be for why someone perceives LLMs as dumb as opposed to speculating whether they're aware of the frontier or not.

> confidently running wrong commands, answering something without looking it up, getting stuck on long scripts and not self-correcting all absolutely still happen, so the statement isn't incorrect.

That is just my point though; these things still do happen, but much, much, much fewer than they used to, and this changes massively every month - both due to improvements in the models themselves, but also experience gained from operating them.

Forming an opinion on using agents for software work once, and then treating your experience as the status quo months later thus leads to wrong conclusions. If you give a model a way to evaluate its work in a short, closed feedback loop, you can orchestrate it to build correct systems pretty easily now. That definitely wasn't true even just three months ago.

If, however, you do not actually use this technology regularly, you will not be able to achieve good results, which objectively makes sense and wouldn't be different for any other skill either.

It matters because they used to be just dumb, but now they are at the same time smart and dumb.

I can work on ideas with LLMs, even within my own area of expertise. They know quite a lot, and can take an idea I have and run with it. I would never trust them to be correct, but if I give them a new idea and ask them to work out details and raise any issues they find, I can quickly evaluate their findings, and take them into account when developing the idea further.

I do not do this on all ideas, not even most. Most of the time it is still me, a sheet of paper or some code. But still, comparing to how it was a year ago, it is night and day!

A year ago I would just close the window after the first reply, shaking my head at its stupidity. I can still get that experience by using anything but Opus 5.5 or Fable. I tried the $20 ChatGPT tier last week and was immediately thrown back to that old annoyance! (I am sure the best OpenAI models are fine)

They are improving quickly

But not getting past the forefront of what's known and written by humans, right? (Aside from filling in math proofs by broad searching and brute forcing known lemmas?)

Peak humans? The best we have to offer?

Or my coworkers?

If $1 million on the line general knowledge question I had to choose between an answer from chatgpt Astra and the answer coming from anyone I know currently (or everyone I know answering if even 1 of them gets it right I win) I'm choosing Astra 10/10 times based on my experience. And I suspect I'd likely leave this game with $10 million.

You either don't know many people or the people you do know aren't that smart. Maybe both.

I know people who know lots about their own domain. The instant you go to zoology or niche astronomy or architecture or whatever I have large doubts on the breadth of their knowledge. I do not have my doubts with Astra at all. It'll just search the internet for a good source (or 50) and get the correct answer.

Not every answer is on the internet.

So, what you are saying is Astra is not really smart, it just has access to the internet while your friends don't.

I'm pretty sure 'Who wants to be a millionaire' and the likes would work very differently if the contestants could search the internet for the answer.

“They are still dumb” “We learned some ways to bypass the dumbness”

"We learned some ways to bypass the dumbness" "They are still dumb"

Reply on news.ycombinator.com