Hacker News Reader

Comment by softwaredoug | Hacker News Reader

The experience still matters.

I have had experiences out of my technical depth where I’ve instructed an agent to do something technically nonsensical. The agent tried to do it, creating a mess. Nothing works, I get frustrated, only to realize I have a fundamental misunderstanding of something technical.

A similar thing happens when you ask for something, but the complexity for the perfect solution is 10 x the complexity of a simpler, 90% solution. And sometimes you only realize that looking at code (or feeling lots of brittleness)

View on HN

Fwiw, that's not what the parent was talking about.

You're trying to middle road it. He was talking about actually treating it as a black box, so just give the model your expectations and goals it needs to meet, not tell it a rough solution for it to apply in detail.

The expectation/goal approach in a ai edited project works better then the middle road since opus5.5 ime. Depending on your project structure, it may not be viable however without large refactorings

Expectations and goals can be technically nonsensical. Agents have gotten really good, but they can’t magically solve intractable problems. If you’re out of your technical depth it can be easy to set unrealistic expectations without realizing it

Then you're still middle-roading it from my perspective. Treating it as a black box means you're only specifying what comes out, essentially.

But to preempt a "I can't do that in my brownfield distributed monolith I'm maintaining at $dayjob": yes, that's why I said it may not be possible without refactoring.

I think we are talking past each other a bit? I’m curious about GP’s perspective here

My point is that you can ask for unrealistic outputs even if you’re treating the system as a black box. This is the classic case of higher level managers asking for infeasible results from their engineering team because they haven’t considered they’re trying to solve the wrong problem. Depends what you’re doing but I don’t think agents tend to push back on badly specified goals unless you’re proactive about it

Sure theoretically. practically this isn't a real problem for me or anyone else i know

Yeah I think this is the main issue at play. Bad engineers are now 10x bad engineers with the ability to hide behind the mask cause they can output something that looks right, but they don't understand it. You can "make" an agent do ANYTHING. There is no rigor, there is no taste, there is no risk. Anecdotally I have seen this play out IRL. I worked with a 3-month boot-camper that somehow stumbled into a software developer role at a small company. The amount of slop that he was creating was unprecedented. No formal training, no idea of abstraction, letting/making the agent reinvent the wheel for everything. The LLM was doing the whole "you're absolutely right" so he was also under the impression that he was a real engineer doing the "right thing". Concretely he convinced our manager that we need to re-write our web-app in Go because it was so much faster(0 users btw) and safer than python. He had an agent build out this test suite and there was a metric like Go vs Python DB operations, Go being 107x better. Seemed sus, looked into it and he(his agent) removed connection pooling for the python side but for the Go side, 100 concurrent connections, 10 idle and a connection lifespan of 1 hour. His reason for that was that the python code kept crashing every time his agent adding pooling so he had the agent removed it. This sheet got printed as is and distributed to our C-suite to much praise. I know this is a very specific example but I fear this is happening at some level everywhere. Where in the past a bad engineer to this degree would stick out like a sore thumb if they were surrounded by competent people that have rigorous standards. An LLM/AI doesn't have to worry about what happens if things don't work and when it's confirming that YOU are a genius every step of the way. It's going to be a disaster when the reckoning comes. I am not a AI luddite but software engineering is just as much as a culture as it is art and science. Without that backstop I just think that we will see a huge degradation in quality because there is no incentive for LLMs/AI to write "good software".

Sounds like that colleague is on track for a Distinguished AI Builder title and promotion

Hopefully this is sarcasm cause if so, haha. Otherwise seek help :P

In the black-box approach, how do you know what is a reasonable task to give it? Just ask the agent about that also?

You don't give it tasks. You give it what you want. It decides the tasks, and then executes. Then you give it feedback. "black-box approach" here is just a fancy word for vibe coding.

I want to be rich, healthy, happy, young, smart, beautiful. Same for all my friends and family please. How many tokens?

PS: make no mistakes and tell me in advance if I forgot anything that I could regret later.

Or on a more technical side: I would like all problems that any app/company/product could solve to be solved in the best possible way, so that nobody has to even think about what kind of problems even could be solved. Go iterate on that but do not make any mistake, do not ask any questions, be done by tomorrow, do not interpret what I said in any bad way.

For me it's really hard to find a balance with LLMs. Even trying one to add .NET IR codegen to my compiler that already had a working C++, JS, and C# backend for it to use as a reference. It just kept doing really odd stuff and when we got to generics it had done such a bad job with the prior work it struggled to collect the information it needed from .NET to do generic calls correctly, and mid session decided to just make everything reflection based with no notice to me. It had a doc of the AST with notes on what the nodes mean and how different constructs are represented, it's a pretty typical ML family language. Plus a working C# backend to reference.

Other times it will one shot things near perfectly that I was expecting to just use as a reference. Demo pages for work I've done its so fast and gets good enough right out the door

Every time I've tried to use it to write something small in my compiler? :"( pain. It's the perfect intersection of looks good and covers 90% of cases fine but explodes far away because some subtle invariant is wrong. I can't always think of the invariants up front without writing the code myself

Reply on news.ycombinator.com