Comment by burkaman | Hacker News Reader

Comment by burkaman | Hacker News Reader

Parent Full thread

Can anyone knowledgeable interpret what was actually done here? It doesn't sound like "performed brain surgery" is accurate.

> Watching our surgeon do the craniotomy, which the agent is not allowed to do yet for safety reasons

I think the only action it took was sending commands to lower a needle to a certain location, inject, and then remove it. But then it sounds like it couldn't actually choose a valid location and this guy had to manually override it? Maybe I'm misunderstanding, talking about this post: https://x.com/OcanaSantero/status/2115442968874139867 (https://xxcancel.com/OcanaSantero/status/2115442968874139867).

I know there is always endless goalpost-moving in these kind of discussions but I really do think there's an important difference between "performed brain surgery" and "observed brain surgery and ran inject(3.9, 4, 4.3) at one point". And there's also no discussion here of whether this decision made any medical sense and what impact it had on the mouse besides not killing it.

From my reading of it, the override moment was because they made a coding error which cause manual adjustment to de-sync the arm position, the agent correctly refused to continue when it noticed that.

It does seem like there is minimal "surgery" involved but it's not much of a stretch. If a surgeon took the same steps but someone else opened the skull, it would still be considered 'brain surgery', I think.

Who made the coding error?

The mouse.

"You own the work"

No the company owns the IP. You own the blame.

I don't know what harnesses they run but when I daily tell Claude or others to do X and they indeed do parts of X and then also Y and break A on the way... I'm so tired of asking "but, really?" that I'm not even bothering anymore. The AI folks live in a different world than I live, and whether it's a real world or only in their minds (or only in their articles) I definitely cannot tell anymore. I simply gave up.

Some variation of “you’re just using it wrong” is usually the refrain I hear when I talk about things LLM’s are bad at. Anything is possible if you divine the right words to input and fix your harness and set pick the perfect model and…

> Anything is possible if you divine the right words to input and fix your harness and set pick the perfect model and…

this has been all of computers since day one, pick a better argument.

imagine this same response but towards a linker/assembler/compiler that was complaining that it couldn't compile your plain speech into exactly the thing you wanted.

I guess the presumption is this : "This thing responds and outputs to plain language decently well, that must be what it's for!" , you then decide to disregard all of computer science, programming paradigms, boilerplate, and any attached rigor that one would normally have in the crafting a computer program and get validated by garbage output : "See honey! I told you these LLMs are useless, it couldn't one shot "Make me a perfect compliant browser no mistakes." "

YES , it goes without saying that with regard to computers -- those things in which we have little choice but to interact with textually most of the time -- that the things you put into them, and the order of those things, and the tool in which you use to do the inputting matter

what a surprise .

Strange how geeks have been making magic-word incantation/wizard jokes about the practice since the 50s, huh?

The purpose of "LLMs" is to magically divine that intent. Else there wouldn't be "OpenAI model solves math" articles, only "Mathematician finds Lean really neat". Programming languages already exist to specify exact intent.

Is this an impossibly high bar? Yes! But that's literally the product pitch. If it can't act as a responsible colleage how is it supposed to replace you all?

Yeesh. Claude still does that? I haven't had that problem since GPT 5.

Are you being sarcastic? I genuinely cannot tell.

Reply on news.ycombinator.com