Hacker News Reader

My LLMs frequently find subtle typos and never make subtle typos. I don't think that's equally likely at all.

Just today I had an incident review meeting resulting from a prod incident where the root cause was an LLM refactor stripping ! from the front of a boolean statement "for no reason". Not to mention how often I've seen garbled output from the LLM.

Lots of things failed for that to reach prod, but the assertion that LLMs do not make subtle typos... is not congruent with my experience.

Which LLM? When was this code written? When was the last time you saw "garbled output" and what was the context?

I'm really trying to get to the bottom of these claims, because I see them a lot and it's so wildly different from most people's experiences.

If a lot of people make the same claim, then from their perspective, you’re the one with a “wildly different” experience. What makes you so certain your experience generalizes to “most people’s experiences”?

Well, I'm either a god with prompts or...

Reply on news.ycombinator.com