Hacker News Reader

Comment by lloeki | Hacker News Reader

It's not just the C-suite.

I keep seeing and talking to people that are completely high guzzling Kool-Aid straight from a pressure tap.

The level of confirmation bias is absolutely unhinged "I told $AGENTICFOOLLM to do that and WOW this PR is 90% AI!", _ignoring any previous failed attempt_, nor the 10% of humanness needed for the change of what ultimately is a 10 line diff and handwaving any counterpoint away as "you're holding it wrong".

I'm essentially seeing two groups emerging: those all aboard the high velocity hype train and those that are curious about the technology but absolutely appalled about the former's irrational attitude which is tantamount to completely winging it diving head first off of a cliff deeply convinced that of course the tide will rise at just the right time for skulls not to be cracked open upon collision with very big sharp rocks.

View on HN

> this PR is 90% AI!", _ignoring any previous failed attempt_

Someone pointed out that AI has a gatcha / intermittent reinforcement effect to it, which explains a lot of its power to capture minds. Sometimes it produces great results. The entire history of industrial process control is about trying to always produce good, or at least identical, results. These two are incompatible.

AI is right in the sweet spot of winning the demo but failing in mass production, and I think people are too optimistic about this being fixable.

You may well be correct that people are too optimistic about it being fixable, but I would say the "why" is a little more complex than that.

It is also possible for artisans to produce functioning goods in low volumes, which is why "hand made" has often been seen a positive quality sign rather than a negative quality sign, and AI can do "artisanal" (same root as artificial). With mass production we want all three of "good", "cheap", and "fast", and we've been able to get it.

But even that AI can do artisanal is not sufficient for AI to succeed, and likely still won't be sufficient even when AI is near the best you can hire in a human, because AI is a memetic monoculture: while the errors of many human artisans can be somewhat uncorrelated and therefore the failure modes different, if everything comes from a single AI mind, it all fails in the same general kind of way. Likewise the quotation about science advancing one funeral at a time: diversity beats even very smart geniuses.

Memetic monoculture is what I think can't be fixed with current approaches, because current approaches need too much information for it to be possible to give each LLM* (even from different providers) a statistically distinct training set to learn from.

* The image generators do seem to be able to get meaningfully distinct output styles. At least superficially, I'm not an artist, so I may be missing some common thread they share beyond the "indistinct blurry background item" that I count as an error rather than a style. But in this topic, even that error would count as a common failure mode.

> if everything comes from a single AI mind, it all fails in the same general kind of way

I'm reminded of https://qntm.org/mmacevedo ; even if we don't get brain uploading, we have a similar sort of thing going on with synthetic "personalities" of LLMs.

> diversity beats even very smart geniuses.

Some of HN will get very mad about that statement but it is often true. Of course, the best combination is a diverse collection of geniuses, which is why midcentury openish borders US did so well.

> The image generators do seem to be able to get meaningfully distinct output styles

Generally they can plagiarize in the style of any human artist you can name whose work is on the internet. Artists are, not unreasonably, upset about this.

> I'm reminded of https://qntm.org/mmacevedo

Mmm.

My inspiration was from Alastair Reynolds putting indoctrinal viruses into the Revelation Space series, listening to an audio book of that several years before mmacevedo was published. At the time, I was thinking of how AI might enforce group-think, rather than write the content itself directly: https://benwheatley.github.io/blog/2019/12/30-18.46.50.html

> Generally they can plagiarize in the style of any human artist you can name whose work is on the internet. Artists are, not unreasonably, upset about this.

Indeed, but so can LLMs of text styles. I meant more of the "voice" seems to be different between them — there seems to be more of a difference between getting SDXL and SD 3.5 (or DALL•E 2 and whatever it's called now it's baked into ChatGPT) to mimic, say, Hieronymus Bosch, than there is between getting GPT-4o and Claude Sonnet 4 to do a poem in the style of a Shakespeare soliloquy.

On the other hand, this may just be my limited appreciation for the arts, as per XKCD: https://xkcd.com/1015/

> The entire history of industrial process control is about trying to always produce good, or at least identical, results.

Oh, but the entire history of gambling is trying to produce highly variable results with exactly the characteristics you mentioned. That's what gets people addicted.

I bet there are also a good number of paid shills among those. If you look at how much money goes into the tech it's not too far-fetched to invest a tiny fraction of that in human "agents" to push the narrative. The endless repetition will produce some believers, even if the story is a complete lie.

Not necessarily paid shills, but it's a good way to get a promotions, and then when it will be revealed that it doesn't actually work, they got the promotion already and will jump on the next hype to get the next promotion.

don't forget the useful idiots.

In tech we seem to have a tendency to believe people are rational and behave predictably, but the truth is there are still truly just a ton of useful idiots even in software and other "high intelligence" areas

I don't exclude them and they have already been covered by my parent comment.

this is true -- the 'believers'

Sounds like you're pretty peeved about this, is your manager on you because your peers are out-delivering you?

You realize even if you're onboard the agentic coding hype train, you don't have to just blithely paste tickets to the agent and let them rip. You can have a long conversation about design and architecture, and have them write their own implementation plan based on that, then watch them ticking items off the list and review code for changes as they're completed while the agent forges ahead. A lot of times you don't even need to have a long conversation about this stuff, just write a readme that very clearly outlines what you're doing and how to do it, and the agent will read it and do just fine.

If by out-delivering you mean "they are producing low quality code that gets merged and then they can't fix all the bugs they've introduced so it's all up to you" then yes.

Reply on news.ycombinator.com