I've been using Claude Code for a couple of days
twitter.com
twitter.com
All the incredible performance and success stories always come from these Twitter posts, I do find value in asking simple but tedious task like a small refactor or generate commands, but this "AI takes the wheel level" does not feel real.
For small stuff LLMs are actually great and often a lifesaver on legacy codebases, but that's more or less where it stops.
Typed programming languages like Typescript, Scala, Haskell and so on will produce more errors -> you need to fix stuff manually. However it will also reduce bugs. So it is a mixed bag. For an error free experience python and JavaScript work very well.
When it comes to tooling, if you haven't used cline, roocode or aider (not as good) yet, you haven't seen what an AI can do.
A good starter would be starting fresh, by creating a README which describes the hole application you want to build in detail and let the AI decide the tech stack. You can most certainly build complex applications with an AI in blazing speed.
Where AI shines for me is as a form of a semantic search engine or even a tutor of sorts. I can ask for the information that I need in a relatively complex way, and more often than not it will give me a decent summary and a list of "directions" to follow-up on. If anything, it'll give me proper technical terms, that I can feed into a traditional search engine for more info. But that's never the end of my investigation and I always try to confirm the information that it gives me by consulting other sources.
Code? Nope.
Coders used to be more productive by using libraries (e.g: don't write your own function for finding the intersection of arrays, use intersection from Lodash) whereas now libraries have been replaced by LLMs. Programmers laughed at the absurdity of left-pad[1] ("why use a dependency for 16 lines of code?") whereas coders thought left-pad was great ("why write 16 lines of code myself?").
If you think about code as a means to an end, and focus on the end, you'll get much closer to the magical experience you see spoken about on Twitter, because their acceptance criteria is "good enough" not "right". Of course, if you're a programmer who cares about the artistry of programming, that feels like a betrayal.
If we're the luddite artisans, LLMs seem to represent the knitting frames which replaced their higher quality work with vastly cheaper, far crappier merchandise. There is a historical rhyme here.
Ive worked on all sorts of code bases filled to the brim with bugs which end users just worked around or ignored or didnt even encounter. Pre-product market fit startups, boring crappy CRUD for routine admin, etc.
It was horrible shit for end users and developers (me) but demand is very high. I expect demand from this segment will increase as LLMs drive the cost of supply to nearly zero.
I wouldnt be surprised if high end software devs (e.g. >1 million hit/day webapps where quality is critical) barely do anything different while the demand for devs at the low end of the market craters.
Over the last few decades the prevalance of more and more frameworks to handle more and more of what used to be boilerplate that everyone had to do for GUI apps has not significantly impacted the bottom end of the job market.
It's only, if anything, caused even higher demand for "just one more feature".
When's the last time you worked on a product in that space where the users or product managers didn't have any pending feature requests? Didn't add new ideas to the backlog faster than the team could implement?
Someone's gonna have to know just enough to make sure the LLM is producing shit that passes the (low) bar of adequateness. But that job will probably be even further away from the CS-knowing, performance-critical, correctness-matters roles than it is today. It's already a big gap, I just don't see the shit-shoveler job going away. The corporate world loves its shit shoveling.
We have not yet reached a point where LLM generated code can also be maintained by LLMs and the tooling is not there. Once that happens, your argument will hold more weight. But for now, it doesn’t. Injecting unreadable, likely bug-ridden code into your application increases technical debt tenfold.
They write as much code as you want, and it often sorta works, but it’s a bug filled mess. It’s painstaking work to fix everything, on part with writing it yourself. Now, you can just leave it as-is, but what’s the use of releasing software that crappy?
I suppose it’s a revolution for that in-house crapware company IT groups create and foist on everyone who works there. But the software isn’t better, it just takes a day rather than 6 months (or 2 years or 5 years) to create. Come to think of it, it may not be useful for that either… I think the end-purpose is probably some kind of brag for the IT manger/exec, and once people realize how little effort is involved it won’t serve that purpose.
I recently used Claude with something along the lines of “Ruby on rails 8, Hotwire, stimulus, turbo, show me how to do client side validations that don’t require a page refresh”
I am new to prompt engineering so feel free to critique. Anyway, it generated a stimulus controller called validations_controller.js and then proceeded to print out all of the remaining connected files but in all of them it referred to the string “validation” not “validations”. The solution it provided worked great and did exactly what I wanted to (though I expected a turbo frame based solution not a stimulus solution, but whatever it did what I asked it to do) with the exception of having to change all of the places where it put the string “validation” where it needed to put “validations” to match the name it used in the provided stimulus controller.
Creating a standard library stdlib with many (potentially thousands) of rules, and then iteratively adding to and amending the rules as you go, is one of the best practices for successful AI coding.
Born to early to explore the galaxy.
Born at the right time to write endless proompts for LLMs.
Is this an especially better (easier, more efficient) route to a working, quality app/system than conventional programming?
I’m skeptical if the answer to the way to achieve 10x results is 10x more effort.
You could easily end up with a lot of rules if you are working with a reasonably large codebase.
And as you work on your code, every time you have to deal with an issue of the code generation you ask Cursor to create a new rule so that next time it does it correctly.
In terms of AI programming vs conventional programming, the writing's on the wall: AI assistance is only getting better and now is a good time to jump on the train. Knowing how to program and configure your AI assistants and tools is now a key software engineering skill.
We could use it for LLM driven coding style linting. Generate PRs for refactoring. Business logic bug detector
Also, you can just tell copilot to write rules for you.
I've been using Claude Code a lot recently, and it's doing amazing work, but it's not exactly what I want it to do.
I had to push it hard to refactor and simplify, as the code it generated was often far more complicated than it needed to be.
To be honest though, most of the code it generated I would accept if I was reviewing another developer's work.
I think that's the way we need to look at it. It's a junior developer that will complete our tasks, not always in our preferred way, but at 10x the speed, and frequently make mistakes that we need to point out in CR. It's not a tool which will do exactly what we would.
This has been the case so far in both js for web (svelte, react) and python automation.
I feel like 3.5 generally came up "short" more often than 3.7, but in practical usage it meant I could more easily modify and build on top of. 3.7 has led to a lot of deconstructing, reprompting, starting over.
Sure, but man are there bugs.
They’re synonymous words and mean the same thing right?
Person who writes logic for machines
I imagine most of us here can agree that some elevate the craft and ritual to great effect while others avoid such high-minded conceits in favor of shipping whatever hits the expected target this week.
I’ve been both at different points in my career. Usually it’s a response to my environment.
The real distinction is programmer vs software engineer, or IOW do you want to write code or solve problems?
IMO, the vast majority of programmers wouldn't meet the definition you've put forward here. I don't know many that dig into the operating system, platform, or hardware all that much, though I work in streaming media so that might just be an industry bias.
That said, it's rarely pays out. Quite the opposite: I often spending lots of time digging unnecessarily. I'm not complaining, I am who I am and I don't want to change. Digging into internals makes me happier and satisfied. And sometimes it is useful. So Linux kernel internals are on the roadmap and hopefully I'll dig into it some day.
I agree that absolute majority of people I met are not of that kind. And probably they should not. Learning business side of the software is what makes someone really useful. I hate business side, I only love computer side.
I'm working in areas of systems programming where such knowledge and willingness to go deeper into the stack is helpful, even if rarely so.
I can't say I understand the kernel, I only scratch the surface of most of it and only dug deeper where I really needed to and only as deep as time allowed me and the need required though.
> Programmers laughed at the absurdity of left-pad[1] ("why use a dependency for 16 lines of code?")
Also, if you cannot tell the difference between code written by an LLM or a human, what is the difference? This whole post is starting to feel like people with very strong (gaterkeeper'ish) views on hi-fi stereo equipment, coffee, wine, ... and programming. Or should I say "code-as-craft" <cringe>?
If you go through and read the left-pad posts here on hn, you'll find people at both extremes: some fiercely defend left-pad as useful and worthwhile and think writing left-pad yourself is dumb-as-hell, and then at the other end you'll find some fiercely deride using left-pad as absurd when they could just write the code themselves. Here's a good thread to start with: https://news.ycombinator.com/item?id=11348798
Personally, I'd rather hire a "coder" than a "programmer" and consider myself more "coder" than "programmer" :)
The problem with this is that you will never be able to modify the code in a meaningful way after it crosses a threshold, so either you'll have a prompt only modification ability, or you will just have to rewrite things from scratch.
I wrote my first application ever (equivalent to a education CMS today) in the very early 2000s with barely any notion of programming fundamentals. It was probably a couple hundred thousand lines of code by the time I abandoned it.
I wrote most of it in HTML, JS, ASP and SQL. I was in high school. I didn't know what common data structures were. I once asked a professor when I got into late high school "why arrays are necessary in loops".
We called this cookbook coding back in the day.
I was pretty much laughed at when I finally showed people my code, even though it was a completely functional application. I would say an LLM probably can do better, but it really doesn't seem like something we should be chasing.
And I don't believe until I see LLMs can use real debugger to figure out a root cause for a sophisticated, cascading bug.
You can be over specified in your prompts and say exactly what types and algorithms you want if you’re opinionated.
I often write giant page long specs to get exactly the code I want.
It’s only 2x as fast as coding, but thinking in English is way better than coding.
Finding the right prompt to have current generation AI create the magic depicted in twitter posts may be a harder problem than most anticipate.
For example, I have mindlessly asked Claude Code over a large codebase "where is desktop app version stored and how is it presented on site". I have expected useless answer given how vague the questions was. Instead I have got a truly exceptional and extremely clear report that fully covers the question.
Another example. I have asked Claude Code to come up with a script to figure out funding rate time intervals on a given exchange and Code ended up in an almost endless loop running small test scripts in node.js to figure this out and came up with a super suboptimal and complicated solution. Turns out my prompt was too verbose and detailed and I have specifically asked Claude Code to figure out time intervals, not just get them. So it did. Instead of just querying the exchange via API and printing the list on terminal (3 lines script) it actually, truly tried to figure them out in various ways.
Sometimes you will get better or worse answers completely by chance.
I think Claude is pretty good if you have it write a function and give it the inputs, output and a data example. You can also put to ask clarifying questions as needed because there is a good chance there are aspects of the prompt that are ambiguous.
My prompts are always better if I write them in a separate text file and then paste them in too. I think I just take my time and think things out more that way instead of trying to get to the answer as fast.
A few things to note:
a) Use the "Projects" feature in Claude web. The context makes a significant amount of difference in the output. Curate what it has in the context; prune out old versions of files and replace them. This is annoying UX, yes, but it'll give you results.
b) Use the project prompt to customize the response. E.g. I usually tell it not to give me redundant code that I already have. (Claude can otherwise be overly helpful and go on long riffs spitting out related code, quickly burning through your usage credits).
c) If the initial result doesn't work, give it feedback and tell it what's broken (build messages, descriptions of behavior, etc).
d) It's not perfect. Don't give up if you don't get perfection.
Edit: there are also recruitment agencies with ex-engineers that do coding interviews, too.
Sorry this is a little harsh, but how do you get anywhere near 100 people before realizing the approach must be horribly flawed, and devising and implementing a better one? Surely it behooves you to not waste your employer's time, your own time, and the time of all those people you're interviewing (mostly pointlessly).
Candidates who use LLMs generally get through 4 or 5 steps in the interview question. Candidates who don't are usually still on step 2 by the end of the interview (with rare exceptions), without their code quality being significantly better.
(I end up in 1:1 interviews with perhaps 10-15% of candidates who take round 1).
I suppose, if you are specifically looking for coders to perform routine tasks, then you'll get what you need.
Of course, you could argue that ~90% of a programmer's work day is performing standard tasks, and even brilliant programmers who don't use LLMs will lose so much productivity that they are not worth hiring... Counterpoint: IMO, the amount of code you bash out in a given time bears no relation to your usefulness as a programmer. In fact, producing lots of code is often a problem.
I think it's a reasonably balanced interview process. Take home tests are useless now that LLMs exist. Code interviews are very time consuming on the hiring side. I'm a firm believer that hiring without some sort of evaluation of practical competence is a very bad idea - as is often discussed on here, the fizzbuzz problem is real.
Because it seems like tougher real-world technical problems (where there are tons of dependencies with other systems in addition to technical and business requirements) needs for the dev to have an understanding of how things work, and if you rely on an LLM, you may not gain enough of an understanding of what's going on to solve problems like this...
... Although, I could see how devs that are more focused on application development and knowing the business domain is their key skill, wouldn't need to have as strong an understanding of the technical (no judgement here, have been in this role myself at times).
Yes, definitely, though I lean more on the 1:1 interviews for that. I understand the resistance to this from developers, but there's a lot of repetition across the industry in product engineering, and so of course it can be significantly optimized with automation. But, you still need good human brains to make most architectural decisions, and to develop IP.
That seems like exactly what the person you're replying to is saying - that sounds like basic standard product-engineering stuff, but simpler, like any of a million examples out there that an LLM has seen a million times. "Here's a problem LLMs are good at, wow, the people using the LLMs do best at it." Tautolgy.
So it's great for finding people who can use an LLM to do tiny product things.
In the same way takehomes had all the same limitations. More power to you if those are the people you are looking for, though.
But it also sounds like a process that most people with better options are gonna pass on most of the time. (Also like with takehomes.)
Yes, product engineering, the thing that 90% of developers do most of their time.
But if you want to create a paradigm shift, you need to do something new. Something LLMs don't yet know about.
That's a mistake. There are plenty of people who are not good multitaskers and cannot effectively think AND talk at the same time, myself included.
IMHO, for programming, monotaskers tend to do better than multitaskers.
Each to their own and maybe their method works out, but it does seem whack.
This is why I wouldn't hire a person who doesn't know how to do this.
Same here. Most candidates I interviewed said they did not use AI for development work. And it showed. These guys were not well informed on modern tooling and frameworks. Many of them seemed stuck in/comfortable with their old way of doing things and resistant to learning anything new.
I even hired a couple of them, thinking that they could probably pick up these skills. That did not happen. I learned my lesson.
This is already a productivity boost. I'm more and more impressed about what I can get out of these tools (as you said, simple but tedious things). ChatGPT4o (provided by company) does pretty complex things for me, and I use it more and more.
Actually, I noticed that when I can't use it (e.g. internal tools/languages), I'm pretty frustrated.
Does it mean that we'll need less engineers to perform the same amount of work? or we'll produce better products? In my company, there's no shortage of things to do, so I don't think we'll hire less people if suddenly engineers are a bit more productive. But who knows how it'll impact the industry as a whole.
The way I prompt it is first I write the documentation of the module I want, following the format I detailed inbthe master documents, and ask him to follow the documentation and specs.
I use cursor as well, but more as an assistant when I work on the architecture pieces.
But I would never let an AI the driver seat for building the architecture and making tech decisions.
I'm using Claude 3.7 now and while it improved on certain areas, it degraded on others (ie: it randomly removes/changes things more now).
LLMs are cool, machine learning is cooler. Still no 'AI' in sight.
After interacting with this tool, I decided it would be nice if the tool could edit itself, so I asked (him ? it ?) to create its next version. It came up with a non-working version of this https://gist.github.com/sloonz/3eb7d7582c33e95f2b000a0920016.... I fixed the bug manually, but it started an interactive loop : I could now describe what I wanted, describe the bugs, and the tool will add the features/fix the bugs itself.
I decided to rewrite it in Typescript (by that I mean: can you rewrite yourself in typescript). And then add other tools (by that: create tools and unit tests for the tools). https://gist.github.com/sloonz/3eb7d7582c33e95f2b000a0920016... and https://gist.github.com/sloonz/3eb7d7582c33e95f2b000a0920016... have been created by the tool itself, without any manual fix from me. Setting up the testing/mock framework ? Done by the tool itself too.
In one day (and $20), I essentially had recreated claude-code. That I could improve just by asking "Please add feature XXX". $2 a feature, with unit tests, on average.
That said - I'm wary of reading too much into results at this scale. There isn't enough code in such a simple application to need anything more sophisticated than churning out a few lines of boilerplate that produce the correct result.
It probably won't be practical for the current state of the art in code generators to write large-scale production applications for a while anyway just because of the amount of CPU time and RAM they'd need. But assuming we solve the performance issues one way or another eventually it will be interesting to see whether the same kind of code generators can cope with managing projects at larger scales where usually the hard problems have little to do with efficiently churning out boilerplate code.
This is why expectations are all out of whack.
Keep in mind this is the commenters first attempt. And I'm surprised he paid so much.
Using Aider and Sonnet I've on multiple occasions produced 100+ lines of code in 1-2 hours, for under $2. Most of that time is hunting down one bug it couldn't fix by itself (reflective of real world programming experience).
There were many other bugs, but I would just point out the failures I was seeing and it would fix it itself. For particularly difficult bugs it would at times even produce a full new script just to aid with debugging. I would run it and it would spit out diagnostics which I fed back into the chat.
The code was decent quality - better than what some of my colleagues write.
I could probably have it be even more productive if I didn't insist on reading the code it produced.
It seems to me people’s perspective on code gen has largely to do with their experience level of actually writing code.
Just as a coworker used it to develop an AI code review tool in a day. It's not fancy - no bells and whistles, but it's still impressive to do it in a day with almost no manual coding.
Not sure it’s a narrow reading. This is my point, if it’s a basic or rudimentary version people should be explicit about that. Otherwise these posts read like hype and only lead to dissatisfaction and disappointment for others.
Reading something literally is by definition the narrowest interpretation.
Doesn't Code have a similar option?
Was this using technologies you aren't familiar with? If not, the output rate seems pretty low (very human-paced, just with an extra couple bucks spent.)
More importantly, the 100-300 lines was very low effort for me. That does have its downsides (skills atrophy).
1. It’s a learning experience 2. Looking at the chat transcripts, many of those dollars are burned for stupid reasons (Claude often fails with the insertLines/replaceLines functions and break files due to miss-by-1 offset) that are probably fixable 3. Remember that Claude started from a really rudimentary base with few tools — the bootstrapping was especially inefficient
Next experiment will be on an existing codebase, but that’s probably for next weekend.
I still have to write the requirements, design, and acceptance criteria.
I still have to gather the requirements from stakeholders, figure out why those will or will not work, provision infra, figure out how to glue said infra together, test and observe and debug the whole thing, get feedback from stakeholders…
I have plenty of other stuff to do.
And if you automate 99% of the above work?
Then the requirements are going to get 100Xed. Put all the bells and whistles in. Make it break the laws of physics. Make it never ever crash and always give incredibly detailed feedback to the end users. Make it beautiful and faster than thought itself.
I’m not worried about taking myself out of the loop.
There's also the question of whether I will enjoy my craft if it is reduced to, say, mostly being a business analyst and requirements gatherer. Though the people paying me probably don't care very much about that question.
How many devs could debug both a K8s network configuration issue and a bug in an Android app caused by a weird vendor's OS tweak? Not most of us.
Some people will be better at pushing the LLM things to generate the write crap for the MVP. Some people will be better at using these tools for testing and debugging. Some people will be better at incidence response. They'll probably all be using tools with some level of AI "magic" in them, but the specialization will be somewhat recognizable to what it's been for the past decade.
If you're on the business side you still want a team of people running that stuff until there's a step-change in the ability to trust these things and they get so good you'd be able to give over control of all your cloud/datacenter/network/whatever infrastructure and spending.
And at THAT point... the unemployed software engineers can team up with the unemployed lawyers and doctors and blue-collar workers who were replaced by embodied-LLM-powered robots and ... go riot and ransack some billionare's houses until they decide that these new magical productivity machines should let everyone have more free time and luxury, not less.
The takes are as different as (paraphrasing): "if a person can't create something with en empty text editor, I fail them", and "if a person can't speed run through an unrealistically large set of goals because they don't use AI-assisted development, I fail them".
I guess one should keep their skills at both honed at all times, even if neither are particularly useful at most real jobs, because you never know when you're going to be laid off and interviewing.
Yup, that's our job as software engineers.
The more seasoned you are as a SWE, the higher the orders you consider, and not just on the technical aspect, but the human and business sides as well.
Google has gotten worse (or the internet has more garbage) so finding code an example is more difficult than it used to be. Now I ask an LLM for an example. Sometimes I have to ask for a refinement and and usually something is broken in the example but it takes less time to get the LLM produced example to work than it does to find a functional example using Google.
But the LLM has only replaced my previous Google usage, I didn't expect Google to develop my applications and I don't with LLMs.
The LLM was obviously much faster and the information was much higher density, but it had quite literally about a 20% rate of just making up APIs from my limited experiment. But I was very impressed with Kagi’s results and ended up signing up, now using it as my primary search engine.
Then I got hooked by having a search engine that actually finds the stuff I need and I've been a subscriber for bit over a year now.
Wouldn't go back to Google lightly.
That being said I appreciate your suggestion and will consider giving that a shot.
Like a single page HTML J's page which does a few things and saves it state in local storage with a json backup feature (download the json).
I also enjoy it for doing things I don't care much but makes it more polished. Like I hate my basically empty readme with two commands. It looks ugly and when I come back to stuff like this a few days/weeks later I always hate it.
Claude just generates really good readmes.
I'm trying out Claude code right now and like it so far.
But I'm in the more 'bad programmer/hacker' camp and think that LLMs are amazing and really helpful.
I know that one can post a link to the chat history. Can you do that for an example that you are comfortable sharing? I know that it may not be possible though or very time consuming.
What I'm trying to get at is: I suck at programming, I know that. And you probably suck a lot less. And if you say that LLMs are garbage, and I say they are great, I want to know where I'm getting the disconnect.
I'm sincerely, not trying to be a troll here, and I really do want to learn more.
Others are welcome to post examples and walk through them too.
Thanks for any help here.
Respectively, are you understanding what it produces or do you think that's its amazing because it produces something, that 'maybe' works.
Here's an e.g. I was just futzing with. I did a refactor of my code (typescript) and my test code broke (vitest) and for some reason it said 'mockResolvedValue()' is not a function. I've done this a gazillion times.
I allowed it via 3-4 iterations to try and fix it (I was being lazy and wanted my error to go away) and the amount of crap (rewriting tests, referenced code) it was producing was beyond ridiculous. (I was using github co-pilot).
Eventually I said "f.that for a game of soldiers" and used by brain. I forgot to uncomment a vi.mock() during the refactor.
I DO use it to fix stupid typescript errors (the error blob it dumps on you can be a real pita to process) and appreciate it when gives me a simple solution.
So I agree with quite a few comments here. I'm not ready to bend the knee to our AI Overloads.
All my work now is in python and SQL now and though I've watched a lot of youtube videos and plunked at StackOverFlow for ~15 years, I've never had formal education in either language. Like, it takes me about as long to set up the libraries and dependancies in python as it does to write my code. My formal job titles have never had 'programmer' in them.
My code, as such, is just to get something done. Mostly this is hardware interfacing stuff, but a little software too. I've had my code get passed up the chain and incorporated into codebases, but that's only happened a handful of times. The vast majority of the code I write is never seen by others. My code hasn't had to be maintainable as I've never really had to maintain it for more than 5 years max. I've used git on projects before, but I don't really see the need these days. The largest program I've written is ~1M lines of code. Most code is about 100 lines long now. I almost always know what 'working' means, in that I know the output that I want to see (again, mostly working in hardware). I almost never run tests of the code.
I've had the same issues you have had with LLMs, where they get stuck and I have to try to restart the process. Usually this happens to me in about 20 back and forths. I'm mostly just pasting relevant code snippets and the errors back into the LLM for a while until things work for me. Again, I know what 'working' looks like from the start.
Typically, I'll start the session with an LLM by telling it the problem I have, what I want the code skeleton to look like, and then what I want the output to look like. Then it will give the psuedo code, then I walk it through each portion of the psuedo code. Then I get to errors and debugging. Usually about half of this is just libraries and versions in python. Then, I get to the errors of the code itself. I can typically find what line of code is causing the error just from the terminal outputs. I'll talk with the LLM about that line of code, trying to understand it from the error. Then, on to the next error. Repeat that process until I get the working output I desire. I'm never expecting the right code out of the first LLM interaction, but I am expecting (and seeing) that the time it takes to get to working code is faster.
The time it would usually take me to get through all this before LLMs was about 2 weeks of work (~80 hours) per project. Now it takes me about half a day (~4 hours), and it's getting faster.
I'm not in the camp of thinking that AI is going to take my job. I am in the camp of thinking that AI is going to finally let me burn down the list of things that we really need to do around here.
Thank you for the reply!
Then I went on github and found that it used some code written by someone in JS 7 years ago and just converted and extended it for my language, but that code was wrong and simply useless. We'll end up with people publishing exploits and various other security flaws in Github, these LLMs will get trained on that and people that have no clue what they are doing will push out code based on that. We're in for fun times ahead.
I wrote a worksheet for Cursor and give it specific notes for how to accomplish the task in a particular case. Then let it run and it’s fairly successful.
Keep in mind…it’s never truly “hands off” for me. I still need to clean things up after it’s done. But it’s very good at figuring out how to filter the HTML down and parse out the data I need. Plus it writes good tests.
So my success story is that it takes 75% of the energy out of a task I find particularly tedious.
My recent usage is oriented towards using pseudocode descriptions that closely map to Python to produce Python functions. I am very impressed with Claude 3.7's syntactic correctness when given a chunk of pseudocode that looks "python-y" to begin with.
My one concern is that much of my recent code requirements lack novelty. So there is a somewhat reasonable chance that the tool is just spitting out code it slurped somewhere in github or elsewhere in the larger Internet. Just this week, I gave Claude a relatively "anonymous" function in pseudocode, meaning variable names were not particularly descriptive with one tiny exception. However, Claude generated a situationally appropriate comment as part of the function definition. This was . . . surprising to me if somehow the model had NOT in its training set had some very close match to my pseudocode description that included enough context to add the comment.
Heck, if we think about it from the programming language perspective, all code is "simply" using already existing language functions to cobble together a solution to some specific set of requirements. Is no program novel?
There is probably a consideration here that maybe boils down to the idea of engineering vs artisanal craftsmanship and where a specific project falls in that spectrum . . .
Consider using Aider. It's a great tool and cheaper to use than Code.
Look at Aiders LLM leaderboard to figure out which LLMs to use.
Use its architect mode (although you can get quite fast without it - I personally haven't needed it).
Work incrementally.
I use at least 3 branches. My main one, a dev one and a debug one. I develop on dev. When I encounter a bug I switch to debug. The reason is it can produce a lot of code to fix a bug. It will write some code to fix it. That won't work. It will try again and write even more code. Repeat until fixed. But in the end I only needed a small subset of the new code. So you then revert all the changes and have it fix it again telling it the correct fix.
Don't debug on your dev branch.
Aider's auto committing is scary but really handy.
Limit your context to 25k.
Only add files that you think are necessary.
Combining the two: Don't have large files.
Add a Readme.md file. It will then update the file as it makes code changes. This can give you a glimpse of what it's trying to do and if it writes something problematic you know it's not properly understanding your goal.
Accept that it is not you and will write code differently from you. Think of it as a moderately experienced coder who is modifying the codebase. It's not going to follow all your conventions.
It is, after all, what many of us would do in our manual SW development. But when using an LLM that seems pretty good, we just assume we don't need to follow all the usual good practices.
You can configure it not to autocommit, although I suppose the "undo" command won't work in that case.
Can you provide a ballpark of what kind of $ costs we are talking here for using Aider with, say, Claude? (or any other provider that you think is better at the moment).
Say a run-of-the-mill bug-fixing session from your experience vs the most expensive one off the top of your head?
Twice I had a "production ready" throwaway script in under $2 (the first was under a dollar). Both involved some level of debugging. But I can't overstate how awesome it is to have a single use script be so polished (command line arguments, extensive logging, etc). If I didn't make it polished, it would probably have been $0.15.
Another one I wrote - I probably spent $5-8 total, because I actually had it do it 3 times from scratch. The first 2 times there were things I wasn't happy with, or the code got too littered with attempts to debug (and I was not using 3 branches). When I finally figured everything out, I started again for the third time and it was relatively quick to get something working.
Now if I did this daily, it's a tad expensive - $40-60/month. But I do this only once or twice a week - still cheaper than paying for Copilot. If I plan to use it more, I'd likely switch to DeepSeek. If you look at the LLM leaderboard (https://aider.chat/docs/leaderboards/), you'll see that R1 is not far behind Sonnet, and is a third of the cost.
how big/complex does the codebase have to be for this to be for you to actually save time compared to just using a debugger and fixing it yourself directly? (I'm assuming here that bugs in smaller codebases are that much easier for a human to identify quickly)
So I hope you can understand it when I say: Why should I waste my brain cells debugging when I can just tell it to fix its own problems?
Say you want something done (for personal use), and don't have the time to develop it yourself. Someone volunteers to write it for you. You run it, and it fails. Would you spend much time reading code someone else wrote to find the bug, or just go back to the guy with the error?
Yes, I have had to debug a few things myself occasionally, but I do it only when it's clear that the LLM isn't able to solve it.
In other cases, I'm writing something in a domain I'm not fully knowledgeable (or using a library I'm only mildly familiar with). So I lack the knowledge to debug quickly. I would have to read the docs or Google. Why Google when the LLM is more effective at figuring it out? Certainly in a few cases, the solution turned out to require knowledge I did not have, and I appreciate that the LLM solved it (and I learned something as a result).
The point with all this is: The experience not binary. It's the full spectrum. For my main codebase I'm responsible for at work, I haven't bothered using an LLM (and I have access to Copilot). I need to ensure the quality of the code, and I don't want to spend my time understanding the code the LLM wrote - to the level I would need to feel comfortable pushing to production.
But in my own work, those tasks are pretty rare, like 3 times a month? Often I start working on something, and the scope and definition of success changes while I'm in the midst of it. Or it turns out to be harder than expected and it makes sense to timebox it and do a quick search for workarounds.
As much as we joke about StackOverflow commenters sometimes telling a question-asker they shouldn't be doing what they're trying to do, you do actually want that (soft) pushback some of the time. Most modern LLMs will gleefully come up with a complete plan for how we're gonna get the square peg in the round hole.
I once accidentally asked a local DeepSeek distilled model to do the wrong thing by accidentally copy pasting the wrong variable name.
It told me how to do it, and then asked me if I was sure.
My local DeepSeek R1 model (deepseek-r1:7b) saw me trying to do something stuupid (I was working with the wrong variable). It told me how to do what I asked and then asked:
> _Is this modification part of a larger change you're making to the code? I'd like to make sure we're not modifying something important that might have side effects._
Looking at its though process:
> _The user wants to modify this string by replacing "-input" with "-kb". But looking at the ARN structure, I recall that AWS S3 ARNs are usually in the form arn<:aws:1151472526310103070>s3:::bucket_name RegionalPart path. The part after the bucket name is typically s3:// followed by the object key._ > _Wait, maybe the user has a specific reason to make this change. Perhaps they're formatting or structuring something for a different purpose. They might not realize that directly modifying ARNs can cause issues elsewhere if not done correctly._
Every single time they were doing something simple.
Just because someone has decades of experience or is a SME in some niche doesn’t mean they’re actually good… engineers.
I still find lots of use for LLMs authoring stuff at more like the function level. "I know I need exactly this."
Edit: I did however find it amazing for asking questions about sections of the code I did not write.
1. 1st prompt is me describing what I want to build, what I know I want and any requirements or restrictions I'm aware of. Based on these requirements, ask a series of questions to produce a complete specification document.
2. Workshop the specification back and forward until I feel it's complete enough.
3. Ask the agent to implement the specification we came up with.
4. Tell the agent to implement Cursor Rules based on the specifications to ensure consistent implementation details in future LLM sessions.
I'd say it's pretty good 80% of the time. You definitely still need to understand the problem domain and be able to validate the work that's been produced but assuming you had some architectural guidelines you should be able to follow the code easily.
The Cursor Rules step makes all the difference in my experience. I picked most of this workflow up from here: https://ghuntley.com/stdlib/
Edit: A very helpful rule is to tell Cursor to always checkout a new branch based on the latest HEAD of master/main for all of it's work.
Cursor w/ Claude has a habit of running away on tangents instead of solving just the one problem, then I need to reject its changes and even roll back to a previous version.
With a proper specification as guideline it might stay on track a bit better.
https://docs.github.com/en/copilot/customizing-copilot/addin...
The first thing I do for a new project is ask Copilot to create a custom-instructions.md for me and then as I work on my projects, I ask it to update the instructions every now and then based on the current state of my project.
Much less misses this way in my experience.
I just spent last night working with Cursor and Claude Code, both support different styles of custom rules.
Then I got to work and was lamenting what my corp sponsored Copilot can't do the same - but turns out it can! :D
EDIT: Just tried it and Copilot is completely clueless. It can explain the file, but doesn't know how to generate one. Claude generates its own CLAUDE.md on startup. Cursor can create its own rulefiles pretty well, even better if you add a rule for generating rules.
It seems like you would be the perfect audience for it. We're hoping the book can teach you what you need in order to have all those success stories yourself.
I tried to get it to build a very simple version of an app I’ve been working on. But the basics didn’t work, and as I got it to fix some functionality other stuff broke. It repeatedly nuked the entire web app, then rolled back again and again. It tried quick and dirty solutions that would lead to dead ends in just a few more features. No sense of elegance or foundational abstractions.
The code it produced was actually OK, and I could have fixed the bugs given enough time, but overall the results were far inferior to every programmer I’ve ever worked with.
On the design side, the app was ugly as hell and I couldn’t get it to fix that at all.
Autocomplete on a local level seems far more useful.
One of the biggest difficulties AI will face is getting developers to unlearn the idea that there's a right answer, and that of the many thousands of possible right answers, 'the code I would have written myself' is just one (or a few if you're one of the few great devs who don't stop thinking about approaches after your first attempt.)
A lot -- and I mean a lot -- of people who hype it up are hobby or aspirational coders.
If you drill down on what exactly they use it for, they invariably don't write code in professional settings that will be maintained and which other humans have to read.
Everyone who does goes "eh, it's good for throwaway code or one offs and it's decent at code completion".
Then there's the "AGI will doom us all" cult weirdos, but we don't talk about them.
I'm curious when we'll start seeing verifiable results like live code streams with impressive results or companies dominating the competition with AI built products.
I did a comparison between Claude Code and Aider (my normal go-to): I asked it to do clone a minor feature in my existing app with some minor modifications (specifically, a new global keyboard shortcut in a Swift app).
Claude Code spent about 60 seconds and $0.73 to search the code base and make a +51 line diff. After it finished, I was quite impressed by its results; it did exactly the correct set of changes I would have done.
Now, this is a higher level of task than I would normally give to Aider (because I didn't provide any file names, and it requires changing multiple files), so I was not surprised that Aider completely missed the files it needed to modify to start (asking me to add 1 correct file and 2 incorrect files). I did a second attempt after manually adding the correct files. After doing this, it produced an equivalent diff to Claude Code. Aider did this in 1 LLM prompt, or about 15 seconds, with a total cost of $0.07, about 10% of the Claude Code cost.
Overall, it seems clear that the higher level of autonomy carries a higher cost with it. My project here was 7k SLOC; I would worry about ballooning costs on much larger projects.
Thats the biggest part everyone misses. It’s all sunshine and rainbows until in a month you realize you start asking llm to think for you and at that point the code becomes shit and degrades fast.
Like with everything else “use it or lose it”
If you don’t code yourself- you will lose the ability to properly so it very fast, and you won’t realize it until too late
Which is approximately zero companies.
For example: thirty years ago, FX trading was executed by a bunch of human traders. Then, computers arrived on the scene, which made all of them practically obsolete. Nowadays FX trading is executed by a collection of automated algorithms, being monitored by few quants.
My question is: is the software development in 2025 basically like what the foreign exchange was in the 2000s?
With industrialisation blacksmiths were replaced by assembly lines. I'm sure that blacksmiths are more flexible and capable in almost any important dimension, but the economics of factories just made more sense.
I expect that when the dust settles (assuming that the dust settles), that most software will be an industrial product. The humans involved in its creation will be engineers and not craftsmen. Today we have machinists and industrial engineers - not blacksmiths.
Quality and quality assurance processes will become more important, I also expect optimised production processes. I think a lot of the software ecosystem is a baroque set of over-engineered (or over crafted) steps and processes and this will probably be refactored.
I expect code quality metrics to be super refined. Craftsmen don't usually produce artifacts to the tolerances that our machines do now - code will be the same. I expect automated correctness proofs, specification languages, enhanced type systems to have a renaissance.
The only reason I ever got into linux at all was because I ended up with some dinky acer chromebook for school but didn't want to stop making stuff. Crouton changed my life in a small way with that.
As I branched out and got more serious, learning web development, emacs, java, I never stopped feeling so irrationally lucky that it was all free, and always would be. Coming on here and other places to keep learning. It is to this day still the lovely forever hole I can excavate that costs only my sleep and electricity.
This is all not gone, but if I was just starting now, I'd find hn and so and coding twitter just like I did 10 years ago, but would be immediately turned off by this pervasive sense that "the way to do things now" is seemingly inseparable from a credit card number and monthly charge, however small. I just probably would not of gotten into it. It just wouldn't feel like its for me: "oh well I don't really know how to do this anyway, I can't justify spending money on it!" $0.76 for 50 loc is definitely nuts, but even $0.10 would of turned me way off. I had the same thoughts with all the web3 stuff too...
I know this speaks more to my money eccentricities than anything, and I know we dont really care on here about organic weirdo self teachers anymore (just productivity I guess). I am truly not even bemoaning the present situation, everyone has different priorities, and I am sure people are still having the exciting discovery of the computer like I did on their cursor ide or whatever. But I am personally just so so grateful the timeline lined up for me. I don't know if I'd have my passion for this stuff if I was born 10 years later than I was, or otherwise started learning now. But I guess we don't need the passion anymore anyway, its all been vectorized!
Over the past year I have accomplished so much with the ever patient LLMs as my guide. It has been so much fun too. I imagine there are many others in my shoes and those that want to learn now have a much friendlier experience.
I know the feeling. We still have access to the engineering thought processes responsible for some of the most amazing software feats ever accomplished (thru source repo history and mailing lists), just with access to the Internet. Of course there's a wealth of info available for free on the web for basically any profession, but for software engineering in particular it's almost direct access to world class teams/projects to learn from.
> but would be immediately turned off by this pervasive sense that "the way to do things now" is seemingly inseparable from a credit card number and monthly charge
To be effective you still need to understand and evaluate the quality of the output. There will always be a certain amount of time/effort required to get to that point (i.e., there's still no silver bullet).
> But I guess we don't need the passion anymore anyway, its all been vectorized!
We're not running out of things that can be improved. With or without these tools, the better you get, the more of the passion/energy that gets directed at higher levels of abstraction, i.e. thinking more about what to solve, tradeoffs in approaches, etc. instead of the minute details of specific solutions.
> this pervasive sense that "the way to do things now" is seemingly inseparable from a credit card number and monthly charge
…is true, but it only applies to experienced engineers who can sculpt the whole solution using these tools, not just random code. You need the whole learning effort to be able to ground the code the slop generators make. The passion absolutely helps here.
Note this is valid today. I have concerns that I’ll have different advice in 2027…
I wonder what it means for the open source...
Jokes aside, code tools are best used in the hands of someone who is already trained and can verify bad code, and bad patterns at a glance.
AI code passes many tests. So does a lot of code written by us, for ourselves. When the code gets in front of users, especially the kind of genius users who learn how to fly by forgetting how to fall, then we learn many good habits.
Money for a computer when I was getting into it was the credit-card part of it — there were no cheap Chromebooks then. (A student loan took care of the $1200 or so I needed for a Macintosh Plus.)
I suspect that's always the way of it though. There will be an easier way throwing money at a thing and there will be the "programming finds a way" way.
Sure, I can pay google or fastmail to host a mailserver for me, but that deprives me of the joy of configuring and updating dovecot/postfix/etc, writing custom backup scripts, writing my own anti-spam tooling, etc. I want to know how all those pieces work.
Sure, I can pay kagi to search its index of webpages for me, but that deprives me of the joy of creating and running a botnet to index webpages, storing 100s of terrabytes of scraped data, and writing my own search code.
Targeting hackers is indeed a sucker's game.
As someone who works in infra and dabbles in coding, this is a continual bugbear, because often I’ll find an optimization while troubleshooting “my” problem, and the dev team is disinterested in implementing it. Their endpoints are meeting their SLO, so who cares?
I'm 90% sure you're serious, but that didn't stop me having the best belly laugh for a solid few minutes at this. Thank you.
Should they pull it off, it's not at all a bad startup to build. However, you need to now invest in a sales force that can sell to the Fortune 500. As a tech founder with no sales trope, this will be incredibly hard to pull off.
I digress, but yeah selling to devs is almost always a terrible idea since we all want to build our own stuff. That spirit may also be waning with the advent of Replit agent, Claude code and other tools.
They are often selling to IT managers against the advice of the developers and IT folks, and then they mostly don't get used because they don't actually add any value to the process.
but automating something that takes minutes to do is Larry Wall's example of programmer laziness, and is a virtue.
of course - this needs obligatory conflicting XKCD comics
automation makes you do more work: https://xkcd.com/1319/
is it worth the time to automate https://xkcd.com/1205/
A small change to create a script file (20 LoC) was 10cts, a quick edit to a README was 7ct
Yes yes engineers make more than that blah blah but the cost would quickly jump out of control for bigger tasks. I’d easy burn through $10-20 upwards a day with this, or upwards $100-$300 a month. Unless you have a Silicon Valley salary, that’s too expensive.
I use other tools like Cody (the tool the author created) or Copilot because I pay $10 a month and that’s it. Yes I get rate limited almost daily but I don’t need to worry that my tool cost is going out of control suddenly.
I hope Anthropic introduces a new plan that bundles Claude Code into it, I’d be much more comfortable using that knowing it won’t suddenly be more than my $50/mo (or whatever)
I'm a bit nervous where I'd end up though - with code I'd "written" but wasn't familiar with, and with who knows what kinds of limitations or subtle bugs baked in.
I’m having a hard time coming close to the 500 included in the monthly subscription and I use it like, a lot.
Just curious how you’re hitting that 200-300 mark unless you’re talking about paying Anthropic outside of cursor. Which I just now realized is probably the case.
This does seem like quite a big downside. It turns every new feature into “implement this in someone else’s code base”. I imagine you’d very quickly have complete dependency on the AI. Maybe that’s an inevitability in this new world?
That's a feature, not a bug. At least from OAI/Anthropic's point of view.
I don't think there's any current AI that is fully trustworthy this way though.
I wouldn't even put them at 50% trustworthy
I think we are going to see a cliff where they become 80% good, and every tiny bit of improvement past that point will be exponentially more difficult and expensive to achieve. I don't think we reach 100% reliable AI in any of our lifetimes
Current AI is likely already beyond 50% trustworthiness, whatever that means.
People had this same prediction about offshore development
Those old school devs are able to find well paying work fixing broken software churned out by overseas code sweatshops
I predict if you can read and understand code without the help of AI models you will be in even higher demand to fix the endless broken software built by AI assisted coders who cannot function without AI help
It amazes me that more people aren't worried about this.
Also (most) engineers don't hallucinate answers. Claude still does regularly. When it does it in chat mode via a flat rate Pro plan I can laugh it off and modify the prompt to give it the context it clearly didn't understand but if its costing me very real money for the LLM to over-eagerly over-engineer an incorrect implementation of the stated feature its a lot less funny.
Right now with Copilot or other fixed subscriptions I can also laugh it off and just create a new tab with fresh context. Or if I get rate-limited because of too much token use I can wait 1 day. But if these actions are linked to directly costing money on my card, then that's becoming a lot more scary.
They absolutely do, where do you think bugs come from? The base rate is typically just lower than current AIs.
For exemple I can’t remember seing a PR with an API that seems plausible but never ever existed, or an interpretation of the specs so convoluted and edgy that you couldn’t even use sarcasm as a justification for that code.
Don’t take me wrong: some LLMs are capable of producing bugs that looks like humans ones, but the term hallucinate is something else’s and doesn’t fit with much humans bugs.
A PR is code that has already been tested and refined, which is not comparable to the output of an LLM. The output of an LLM is comparable to the first, untested code that you wrote based off of your sometimes vague memory of how some API works. It's not at all uncommon to forget some details of how an API works, what calls it supports, the details of the parameters, etc.
For an LLM-equivalent experience, go open notepad.exe and make substantial changes there, and then rebuild - and let the compiler tell you what's your base rate of hallucinating function names and such.
And we never made up APIs, as there just weren't that many APIs. We would open the .h file for the API we were targeting as we typed into the other window. And the LLMs have ingested all the documentation and .h files (or the modern equivalent) so they don't have a real excuse.
But I use the LLMs all the time for math, and they do profusely hallucinate in a way people do not. I think it's a bit disingenuous to say that LLMs don't have that failure mode that people don't really have.
It won't say "I think it should look something like this but I might be wrong," it'll say "simple! here is how you do it."
Hence hallucination and not error. It thinks it's right.
And it doesn't fight me like the OpenAI tooling does that logs me out randomly every day and I have to login and spend 4 minutes copying login codes from my email or answering their stupid Captcha test. And this is on their API playground where I pay for every single call - so not like I'm trying to scrape my free chat usage as an API.
Okay maybe need to clarify: I hit those limits when I do agentic stuff, which is what Claude Code does: So let the LLM automatically pull in files into the context it thinks it needs, analyze my codebase, follow imports, add more code, etc. It can quickly balloon out of control when the LLM pulls in too many LoC and the context window gets too big.
Then do a few back and forth actions like "let's refine this plan, instead of X pls do Y", or "hmm I think maybe we should also look into file blah.ts" and you quickly hit 500k tokens.
If I use Cody only, which has some agentic capabilities but is much more "how can I implement Y in this file @src/file1.ts db models are in @src/models/foo.ts", then I rarely ever hit any rate limitations. That's more similar to what you describe of copying code back and forth, except it's in the editor and you can do it by writing @somefile.
I'm not saying that "Claude Code makes me 300% more effective", but I guess it did for this (simple) task.
I don't know what you mean by "how do these times change if you are coding in a very complex environment", but to restate my original post: I'm fearful that Claude's additional autonomy will allow it to waste money (and time) pursuing useless ideas.
My scrum/agile coach says, by parallelizing prompts, a single developer can babysit multiple changes in the same time slice. By having a sequence of prompts ready before hand, a single developer can pipeline those one after the other. With an IDE that helps schedule such work, a single developer can effectively hyper-thread their developmental workflow. If the developer is epoll'ing at 10x the hertz... that's another force multiplier. Of course context switches & side-channels are of concern, but a voice over my shoulder tells me that as long as memory safety is guaranteed, everything should turn up alrigd3adb33f.
Although I’m not proficient in HTML, CSS, or JavaScript, I have some understanding of what good code looks like. Through several iterations, I managed to complete the task in a single evening, which would have required me a week or two to relearn and apply the necessary skills. Not only the code is better organised, it’s half the size and the website looks better.
* Whatever context the agent decides to pull in.
* However many iterations the model decides to run.
* Any result you get, regardless of how bad it is.
With pay as you go, the tool developer has no incentive to minimize any of these costs—they get paid more if it's inefficient, as long as it's not so inefficient that no one uses it. They don't even need it to be especially popular, they just need some subset of the population to decide that costs don't matter (i.e. those with Silicon Valley salaries).
With Cursor's model of slow and fast queries, they are taking responsibility for ensuring that the agents are as cost efficient as possible. The more efficient the agent the larger their cut. The fewer times that people have to ask a question a second time, the larger their cut. This can incentivize cutting corners, but that somewhat balanced out by the need to keep people renewing their subscription, and on the whole for most users it's better to have a flat subscription price and a company that's optimizing their costs than to have a pay-as-you-go model and a company that has no incentive to improve efficiency.
I don't mind paying per-request, but I can't help but think the daily API revenue graph at Cursor is going up whenever they have released a change that trips up development and forces users to intervene or retry things. And of course that's balanced by churn if users get frustrated and stop or leave. But no product team wants to have that situation.
In the end I think developers will want to pay a fair and predictable price for a product that does a good job most of the time. I don't personally care about switching models, I tend to gravitate towards the one that works best for me. Eventually, I think most coding models will soon be good at most things and the prices will go down. Where will that leave the tool vendors?
https://x.com/ericzakariasson/status/1898753771754434761
https://old.reddit.com/r/cursor/comments/1j5kvun/cursor_0470...
I am afraid that the endgame of programming will be who has the biggest budget for an LLM, further consolidating programming to megacorps and raising barrier to entry.
Just a quick heads-up based on my recent experience with agent-based AI: while it's comfortable and efficient 90% of the time, the remaining 10% can lead to extremely painful debugging experiences.
In my view, the optimal scenarios for using LLM coding assistants are:
- Architectural discussions, effectively replacing traditional searches on Google.
- Clearly defined, small tasks within a single file.
The first scenario is highly strategic, the second is very tactical. Agents often fall awkwardly between these two extremes. Personally, I believe relying on an agent to manage multiple interconnected files is risky and counterproductive for development.
http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...
For the moment, after a few sessions of giving it a chance, I find myself using "claude commit" but not asking it to do much else outside the browser. I still find o1-pro to be the most powerful development partner. It is slow though.
> - Architectural discussions, effectively replacing traditional searches on Google.
> - Clearly defined, small tasks within a single file.
I think you're on point here, and it has been my experience too. Also, not limited to coding but general use of LLMs.
I'd claim that if you're debugging the code - or even looking at it for that matter - that you're using AI tools the wrong way.
I'm guessing that the folks who read the output and want to understand it deeply and want to "approve" it like a standard pull request are having a very different perspective and workflow than those who are just embracing the vibe.
I do not know if one leads to better outcomes than the other.
I am serious and didn't think anything I said here was contentious. Which part are you feeling incredulity over? I'll try to clarify if I've been unclear or learn from your perspective.
I _personally_ follow the careful crafting and review approach. I think a deep understanding of the systematic ideas underlying a project is critical to continuing to improve it effectively and I believe code that is easy to understand is extremely valuable for that. I'm my experience so far the ai stuff seems to create more, harder work for me in the long run and I end up with worse understanding after than if I just wrote it myself but I recognize that's just my person anedata so far and we're at the beginning of this ai coding landscape.
I'm also open to the idea that lots of software we currently carefully craft by hand could likely be autogenerated ad hoc as needed. Moreover, good engineers are extremely expensive and most software is quite bad anyway so I expect there is a new balance to be found here.
For my 2 cents, I'm not removing my or my company's ability to understand our own software in exchange for vendor lock in with ai (even if they weren't all owned by the current set of megalomaniacs) but I see the angles and I'm not afraid to consider the tradeoffs or have the conversation.
The business domain is far more nuanced and complex, and your flimsy "does it compile" test for the logic doesn't even begin to cover the necessary gamut of the input domain which you as the expert might have otherwise noticed had you performed even a cursory analysis of the LLM generated code before blindly accepting it.
Nice to know that I'm going to be indefinitely employed fixing this kind of stuff for decades to come...
Again, you're using AI the wrong way.
It's exactly what Yegge describes. It runs in the terminal, offering a retro vision of the most futuristic thing you can possibly be doing today. But that's been true since the command line was born.
But it's more than Claude Code, in that it's backend LLM agnostic. Although sonnet 3.7 with thinking _is_ killing the competition, you're not limited to it, and switching to another model or API provider is trivial, and something you might do many times a day even in your pursuit of code that vibes.
I've been a vim and emacs person. But now I'm neither. I never open an editor except in total desperation. I'm an aider person now. Just as overwhelmingly addicted to it as Yegge is to Claude Code. May the future tooling we use be open source and ever more powerful.
Another major benefit of aider is its deep integration with git. It's not "LLM plus coding" it's really like an interface between any LLM (or LLMs), git, and your computer. When things go wrong, I git branch, reset to a working commit, and continue exploring.
note: cross-posted from the other (flagged) thread on this
Complete with computer people kayfabe!
:/
Prompt engineering (to the extent that it results in the difference between applied LLM success vs failure) requires the human to truly grok what they're getting help on - the current top comment notes that the model applied more or less exactly the same diff they would have done, based on deep understanding of the exact codebase being modified - and also have some level of intuition about how LLMs work.
Someone just treating the model and context like an obscure TV remote and pressing all the buttons to see what they do, will get a result, and it might seem interesting, but will it be ontologically correct? Good question!
For me, I don't know this person, which means that all the words are completely meaningless.
Which is exactly the point.
Quotation from https://www.logicallyfallacious.com/logicalfallacies/Appeal-...
If (hyperbole) Picasso wrote a tweet saying certain canvases work great for certain tasks, and some random internet dude counters with A single tweet with lots of analogy, with no images/painting examples whatsoever. These are just words. Are we just discussing art based on vibe? ...
I will read that comment in the comic book guy voice.
Because you don't know who Yegge is, the words are meaningless to you? So a body of text is meaningful only if you know who the author is? That's...lame.
People do literally call it vibe coding.
https://en.wikipedia.org/wiki/Vibe_coding (it turns out there is a wikipedia page already although suspect it'll be gone soon)
Sure, for fun and one-off private apps this is fine, but the second that some buzzed-on-the-sides haircut guy thinks this is the way, the chaos will rival a Hollywood disaster movie.
But actual software engineering does demand great skill. What "vibe coders" are doing isn't engineering. It's not even programming. It's like calling yourself a chef when you microwaved a frozen meal.
Great engineers know that code is 10-20% of the whole solution to a problem and it doesn’t matter too much. Code is not an asset, it’s a liability.
Then AI generated code is a time bomb. Maintainability? Who needs that if you can sell the company quickly enough, I guess.
But as more developers rely on AI and their skills atrophy, if entire products are being built with AI, eventually no one will exist who can fix the software when it breaks. Not only that, but anything that requires knowledge outside the training of the AI will be impossible (or at best, extremely difficult/costly) to implement.
It looks like it’s doing a lot, and at first I was very impressed, but after a while I realized that when it ran into a problem it kept on trying nonworking strategies even though it had tried them before and I had added to claude.md instructions to keep track of strategies and not reuse failing ones.
It was able to make a little progress, but not get to the end of the task, and some of its suggestions were completely insane. At one point there was a database issue and it suggested switching to an entirely different database than the one that was already used by the app, which was working and production.
$12 spent in a couple of hours later, it had created 1200 lines of partially working code and rather of a mess. I ended up throwing away all the changes and going back to using the web UI.
Even if you won’t use it again, that’s booked revenue for the next fundraise!
More about my experience on my blog: https://hartenfeller.dev/blog/testing-claude-code
As for the app itself, it included a simple UI built on React with custom styling and real-time support using WSS . I provided it with my brand colors and asked it to use chadcn. It also includes a Node.js-based backend with socket.io and puppeteer. I even asked it to generate a Dockerfile and Kubernetes manifests. It almost did a perfect job—the only thing I had to fix manually was updating my Ingress to support WSS.
After studying the K8s manifests, I learned a bunch of new things as well.. spent around $6 for this session and I felt that it was worth it.
Any LLM application that relies more-or-less on a single well-engineered prompt to get things done is entry level and not all that impressive in the big picture - 99% of the heavy lifting is in the foundation model and next token prediction. Many code assistants are based on something like this out of necessity of needing to support anybody's code. You can't rely on too many clever prompt chaining patterns to build optimizations for Claude Code because everyone takes different approaches to their codebase and has wildly differing expectations for how things should go down. Because the range of expectations is so vast, there is a lot of room to get disappointed.
The LLM applications that are most interesting have the model integrated directly with the product experience and rely on deep domain expertise to build sophisticated chaining of prompts, tool calling and nesting of conversations. In these applications, the user's experience and outcomes are mostly predetermined with the grey areas intended to be what the LLM is dealing with. You can measure things and actually do something about it. What was the probability of calling one tool over the other in a specific context of use? Placing these prompts and statistics alongside domain requirements will enable you to see and make a difference.
I asked it to refactor a medium-sized Python project to remove duplicated code by using a dependency injection mechanism. That refactor is not really straightforward as it involves multiple files and it should be possible to use different files with different dependencies.
Anyway, I explain the problem in a few lines and ask for a plan of what to do.
At first I was extremely impressed, it automatically used commands to read the files and gave me a plan of what to do. It seemed it perfectly understood the issue and even proposed some other changes which seemed like a great idea.
So I just asked him to proceed and make the changes and it started to create folders and new files, edit files, and even run some tests.
I was dumbfounded, it seemed incredible. I did not expect it to work with the first try as I had already some experience with AI making mistakes but it seemed like magic.
Then once it was done, the tests (which covered 100% of the code) were not working anymore.
No problem, I isolate a few tests failing and ask Claude Code to fix it and it does.
Now for a few times I found some failing tests and ask him to fix it, slowly trying to fix the mess until there is a test which had a small problem: it succeeded (with pytest) but froze at the end of the test.
I ask again Claude Code to fix it and it tries to add code to solve the issue, but nothing works now. Each time it adds some bullshit code and each time it fails, adding more and more code to try to fix and understand the issue.
Finally after $7,5 spent and 2000+ lines of code changed it's not working, and I don't know why as I did not make the changes.
As you know it's easier to write code than to read code so at end I decided to scrape everything and do all the changes myself little by little, checking that the tests keep succeeding as I go along. I did follow some of the recommended changes it proposed tough.
Next time I'll start with something easier.
Or... you can do that with the robots instead?
I tried that with the last generation of Claude, only adding new functionality when the previously added functionality was complete, and it did a very good job. Well, Claude for writing the code and Deepseek-R1 for debugging.
Then I tried a more involved project with apparently too many moving parts for the stupid robots to keep track of and they failed miserably. Mostly Claude failed since that's where the code was being produced, can't really say if Deepseek would've fared any better because the usage limits didn't let me experiment as much.
Now that I have an idea of their limitations and had them successfully shave a couple yaks I feel pretty confident to get them working on a project which I've been wanting to do for a while.
I generally follow the same approach these days, ask it to develop a plan then execute but importantly have it excute each step in as small increments as possible and do a proper code review for each step. Ask if for changes you want it to make.
There is certainly times I need to do it myself but definitely this has improved some level of productivity for me.
It's just pretty tedious so I generally write a lot of "fun" code myself, and almost always do the POC myself then have the AI do the "boring" stuff that I know how to do but really don't want to do.
Same with docs, the modern reasoning models are very good at docs and when guided to a decent style can really produce good copy. Honestly R1/4o are the first AI I would actually concider pulling into my workflow since they make less mistakes and actually help more than they harm. They still need to be babysit though as you noticed with Claude.
AI coding is like having a partner who is both the smartest person in town, but also a functional alcoholic, who's really, really good at hiding it. LLMs act like they are working in a dark warehouse with a flashlight, and Altzheimers.
They can get an amazing amount of functional code done in a few minutes, and then spend hours trying to fix one detail. My standard prompt begins with, "Don't guess, debug!" They have limited resources and will bs you if they can.
For longer projects, since every prompt is almost starting from scratch (they do have a limited buffer, which will make it easy to become complacent), if you get into repeated debugging sessions, it will start creating new functions instead making exisiting functions work, and code bloat is tremendous. Perhaps Rules work, but I've given up trying to get it to code in my style. I'm trying to have AI do all the coding so I can just be the "idea" guy ("vibe" coding), so I'm learning to let go and let it code in ways the I would hate to maintain. It's working from code examples that don't use my style, so I'm not going to keep fighting it on style (with some execptions like variable naming conventions).
I'm stealing this :-) (And from now on I'll be imagining coding alongside Columbo. Claude Columbo.)
I am not disposed to AI skepticism. I'd love it if a tool existed that worked as this guy claims. Claude Code is the best tool of its type that I've worked with but I'd put it at "on balance a time-saver in a lot of cases, way more trouble than it's worth in many others".
Cody does use Claude Sonnet, so they do have some aligned interests, but it's still surprising to see Yegge speak so glowingly about another product that does what his product is supposed to do.
Claude Code didn't feel that different to me, and maybe they have something that is better and when they do release it they can say hey look, we pushed hard and have something that's better than even Claude Code.
My test was to create a tower defense game that would run in a web browser. It took me about eight prompts, each time refining it, including agreeing to the suggesting that Claude recommended and seeing Claude agree with me on the bugs that I pointed out.
It was mind blowing. It’s not pretty at all but is recognizable as a TD game. I thanked Claude and said that was enough and Claude signed off as well saying, well if you’re going to continue to refine it, do these four things. I was really stunned.
You didn't do anything, just asked a different entity to do it for you. And nothing noble or original, just a copy of existing games. I see no difference between this and getting a $500 coupon to use at Fiverrr and ask a freelance engineer to do the same while you chat with them.
If they were doing it for an exam where their skills were being evaluated, that’s one thing. But if they were doing it as a means to an end, does it matter if they found a more efficient way to do it?
And what's it matter if someone else made it? Do you make your own bread or milk your own cows? Do you build your own cars?
Like who gives a shit, when the most important thing for 95% of people is, "Can i use this? Can I operate it? What can this do for me? How does this make life easier/better/fun?".
It's also interesting as it shows the reality we're heading towards with hyper-personalized media. Love it or hate it, it's coming.
About your last example, it's like asking a AI to write Harry Potter but where the character has a different name and be blown away for what I did in a weekend!
I recently interviewed a very pro-ai dev who's last position listed 'architect' and was stunned at how limited their knowledge seemingly was. I didnt have a coding question prepared because i assumed we could have a higher level 'architecture'-oriented chat but they had seemingly nothing to contribute... Thankfully another interviewer had a simple coding challenge prepared, which filtered the candidate out.
Long story short, i wonder if the folks who.genuinely enjoy coding are going to be the only remaining skilled technologists after the rest make themselves obsolete by overly depending on these tools, regardless of how good they supposedly are.
It's a personal attempt to see how much I can do with an automaton. I could pay someone to do my taxes or file them myself (I'm in the US). There is much more room for frustration but also lots of benefits to the latter.
In particular, with Claude Artifacts, I had a chance to see an amazing innovation. Have you ever wanted to see something new just because it's new? It changes you, which of course is one of the purposes of exploring novelties. By the way, this was my experience in July 2024.
Setting up a simple game on your local machine really isn't that hard. For example, you can probably take an open-source project and with some minor alterations have something working pretty quickly.
But most software development doesn't work like this. You are typically given very explicit requirements and when you're working for a corporate with real customers you have high quality standards that need to be met. Often this is going to require a lot of bespoke code and high-level solutionising which you're not going to get out of some open source project (at least not without significant changes).
Similarly, productionising products requires a lot of work and consideration which spinning something up locally doesn't. You need to think about security, data persistence, hosting, deployment, development environments, documentation, etc, etc, etc...
I think this partly explains why people have such widely different opinions on these tools at the moment. I acknowledge they write pretty good code, but for me they're almost useless in 90% of the things I do and think about as a software engineer.
Yeah it sounds a lot like marketing to me
But in my experience they haven't been great whenever asked to something more high level
> As long as the bank authorizations keep coming through, it will push on bug fixes until they're deployed in production, and then start scanning through the user logs to see how well it's doing.
I enjoy using these tools. They help me in my work. But the continual hype makes the discussion around them impossible to be genuine.
So I ask, genuinely, did I miss the configuration section where you can have it scan your logs for new errors and have it argue with you on PRs? Is he trying to say something else? Or is it just anthropomorphizing hype?
We’ll see, maybe my whole approach is wrong, I’m going to try with a simpler project, my first approach was relatively complex.
However, the extra time between sessions does give me the chance to consider where the AI might have gone wrong in the first session and how I could have phrased the initial prompts more effectively.
As others have stated throughout the threads here, I definitely recommend giving it smaller nibbles of problems. When I take this route and get working modules, I will start a fresh prompt session to upload the working code module files and ask it to integrate them into a simple solution with static inputs in a 'main' function. (Because providing coherent inputs from the router function inputs of a web service are simple enough, once these basics are covered)
Basically - I do everything possible in order for the AI to not get distracted or go down a rabbit hole, chasing concerns that humans are very capable of taking care of (without paying tokens for). I will write most of the tests myself and then perhaps afterwards ask it to evaluate the test coverage and provide more of them. This way the new tests are in the format and platform of my choosing.
I have found that prompting something like “do X and write tests to confirm it works” works well for what you’re describing. Or even you write the tests then it’ll iterate to make sure they pass.
haven't tried PR stuff yet though
There are other types of work where Copilot is useless. But it's up to me to take the good parts, and ignore the bad parts.
Until then I'm sticking to a combination of Amazon Q Developer (with the @workspace tag in Intellij), ChatGPT and Gemini online.
A coworker tried both and found Code to be 4x the cost because it doesn't easily let you limit the context.
Sometimes it takes the easy way out. If you look up your billing, you will see sometimes it uses the 3.5 instead of the 3.7 API, maybe this has something to do with it. It apologies ("you are correct", "yes that was hasty") but I'd rather have it try harder every time, not only when called out (and foot the bill that goes with that ofc).
But overall the experience is great. I hope to use it for a few months at least until something else comes along. At the current pace, I switch subscriptions/API/tool at least once a month.
It’s expensive and is correct for the easy 90%. It messes up the hard 10%.
I guess with it rapidly improving, I should wait 3 months. What’s frustrating is when you spend 20$ in credits on it writing non functional code.
That said, every programmer needs to at least demo these tools
I use Aider chat with Claude and my sessions are much smaller. You need to ask it much smaller tasks and work incrementally.
Maybe it will be at some point, but it is not yet a reasonable substitute for having to think, plan, or understand the code you are working with and what needs to be done. It's also still more expensive than it needs to be to use on my free time (I'm happy to burn company money on it at work, though).
If I was a better programmer I'd be working on that solution right now.
I currently use Cursor as a devops agent, I use the remote ssh extension to ssh into a VM, then Cursor will set up everything, I make snapshots on way in case it fucks up. It's been really great to quickly be able to setup and try out different infrastructures and backends in no time at all. It works well enough that I now do all my development using remote dev with ssh or remote containers on the a server. Having a virtualized dev environment is a great addition to just having git for the code.
Cursor’s “chat with your codebase” is a funny joke compared to Claude Code. Ask it questions, have it figure things out.
I had it analyze the openAPI schema and the backend that is serving the schema for the API I’m writing and write end to end tests for the API. Then I did my normal meetings and it was done with huge chunks of it, it had run the code locally and tested against the actual endpoints to understand if the end to end tests were working or it had found a bug. Then it fixed the bugs it found in the backend codebase. My prompt: “write me end to end tests against my openAPI schema”
That was it. 30$ in tokens later, pressing enter a bunch of times to approve its use of curl, sed, etc…
So think about the reasoning there. It read the API docs that said all the timestamps were 3339, it ran a bunch of working 3339 timestamps and was getting them back from most of the APIs, so it decided this was a standard, read the code on the backend and understood this endpoint was different, and decided on the right fix and implemented it. Didn’t need to ask me for help. That is pretty impressive if you’ve been using copilot.
On the other hand, I also get the sense that for a very large organization with siloed teams that sure, the org moved to bring all the endpoint timestamps to a standard 3339 and either you never got the memo or your endpoint being misaligned would get caught at code review time and changed then but the agent was proactive here and went ahead and saved some time. Like, truly, I get that the marginal gains there add up on the balance sheet.
But what about when it is important that your APIs timestamps stay unaligned as epoch? Do you now need to set an override to meta-program that field for the AI as an additional token in the context window every time it runs?
Part of me is impressed for sure about what’s being achieved, but then another part is like “is the boilerplate of copying in the database field names and then shift+alting to write 100 lines of { get; set; } at the same time really so terrible for everyone?” I’m being hyperbolic for effect but I think my point is clear.
There’s also a third part of me that feels like I’m watching the creation of credit-default swaps of tech debt. If no one has time to do the changes that seem like a lot of work to a large organization with inertia because no one has the breadth of mastery needed to implement them as efficiently as possible, how the heck is anyone going to audit and fix the behavior when subtle mistakes eventually cascade?
Is this just the coding version of Amazon outsourcing their shipping to third party contractors that definitely led to people dying but shielded Amazon from liability? Like, “oops, sorry we don’t know why the AI changed our code and your bank account got emptied, we’ll look into it, but it’s not our fault, just the vagaries of fate!”
Again, I do understand the possible business savings in the short term in terms of administrative overhead, but I really shudder at the possible downstream effects of agentic code at scale and I’m trying to understand the real savings for a user that outweighs the downside risk of blowing up your production system and then being in a position of no one suited to fixing it who is responsible to do so.
As long as it costs less than a developer, it will be used instead of the said developer.
He had more like it - search for “yegge chat oriented programming”
I didn't know at the start what should be done, and the code ended up having lots of duplication, but i refactored it a lot and now it is in the order of 4.000 lines.
I could make some screencasts about the development process, but it is very simple. I ask it to write some code, and I always provide some relevant context. When it is a method of a struct, i give it the struct, and/or a similar method, and i describe what i want the method to do. Also sometimes I give it the type signature of the method/function which has to write. When it has to return an error, i provide the error Enum.
In other words, by providing the context, i never use it zero shot, rather always aim for few shot answers. When i want another function to use, i just provide the type signature, and almost never the whole function.
One more detail, I give as minimal of a context as possible. I use as a context window, 100 tokens, 200 or maximum 300 tokens. 100 tokens per query, then delete the context for the next task, and provide new context. Never use more than few hundred tokens per query. Even 300 tokens is pushing it too far.
That's about it! Never use LLMs zero shot, always few shot, and never use more than 100 tokens per query.
[1] https://github.com/pramatias/fxp_videoclipper/tree/main [2] https://www.youtube.com/watch?v=RmmoMPu091Y
It's very cool but highly limited in many ways.
It feels like this same statement has applied to LLMs since their popularity explosion.
But as I’ve improved at Rust I have noticed I am using Copilot a lot less. For me now it has mainly become a tool for code completion. It’s not helping me solve problems now, it’s really just about saving time. I have estimated (unscientifically) that it probably improves my productivity 2-4x.
I gave it a huge Clojure codebase and told it to implement proration in my subscription system. Which currently doesn't have proration of any kind. Very poorly specified task, and yet what I got was really quite good — not working, ready-to-run code, but certainly a great starting point.
I was impressed by how it was able to follow my coding conventions, interface with my Stripe library to (correctly) create and confirm payment intents, and most importantly, by the `calculate-proration-amount` function which was correct in spite of how difficult it was (there are custom plans, plan overrides, extra users, etc).
It is the first time I felt an AI coding tool is genuinely useful. I still think you need to set expectations correctly: this doesn't "just do stuff", it is not a human programmer and will not produce flawless ready-to-run code in most cases. But it's a very good tool.
So why wouldn't I continue burning Cursor's VC money? LOL
Nothing fancy or special. It was built with streamlit in about 150 lines and a single file. But I was impressed that Claude code 1 shot it
https://techcrunch.com/2025/03/06/a-quarter-of-startups-in-y...
For example, I wanted a dbt[0] like tool, but written in rust, specifically focused on duckdb. Claude Code knocked it out[1] it without much guidance.
Also added support for all duckdb output options (e.g. write to a partitioned parquet instead of a table).
0 - SQL transformation tool (https://github.com/dbt-labs/dbt-core)
So far LLMs are great at producing code, but not at architecture or maintenance.
I'm just restating the obvious because LLMs can do exercise for you, but there is not much to be gained if one follows this path.
Unfortunately I fear we will just enter an era of general slop where people get away with creating without really understanding. It sucks for anyone who is actually passionate and curious and really investing the time.
Coding is the real market for LLM's - it is the "killer app". I'm starting to think that a possibility is that for most jobs it ends up being a "what was all the hype about" but for SWE's it will be carnage globally w.r.t jobs. They aren't generally intelligent (lots of hallucinations in other domains), but with RL can be trained on the mountains of open source software out there to displace software.
Software developers automating their own jobs away. The promise of "free software" with open source was realised - just not as people envisioned it.
No other profession would do this or at least not as fast. As a honest answer, as much as I want to be wrong, I don't recommend new comers join the industry unless they really want to do it and are happy taking the risk. The uncertainty is high right now, and the anxiety these tools are giving quite a number of people I talk to is very high. Capitalism doesn't reward value or how much is built - it allocates resources and rewards people who produce into scarce markets. AI makes the product of code significantly less scarce.
But Sonnet 3.7 actually seems dangerous to me, it seems it’s been RL’d _way_ too hard into producing code that won’t crash — to the point where it will go completely against the instructions to sneak in workarounds (e.g. returning random data when a function fails!). Claude Code just makes this even worse by giving very little oversight when it makes these “errors”
In other words: "do your job, developer!"
I’d love to sometimes chat to an agent and dictate what I want to happen.
For one project there’s a lot of boilerplate and I imagine AI could be really fast for tasks like: “create a new controller called x”, “make a new migration to add a new field the to users table called x” etc
https://aider.chat/docs/usage/voice.html
Thanks!
Something like that. Anyhow Claude needs to allow me to put in my own API keys.
He talks about it later of using autocomplete vs complete writing, I don't think he misunderstood the scope just talking about the difference of ways ai can be used for coding assistant.
For full stack, I would say Lovable is the best, for complexity, I think Cline is the best.
Claude Code with the several request I gave it just produced code that didn't run. I think it has a long way to go.
Let’s not forget, if you relied on hacker news to give you accurate takes on things, you would’ve never bought bitcoin at one cent.
I no longer mind referring to it as X, because that clearly outlines that it's a different website with a much more rancid vibe.
If the few services that are left become well-known, they'll cease to function just like the majority. Such is scraping Twitter now
While we encourage people to use AI systems during their role to help them work faster and more effectively, please do not use AI assistants during the application process. We want to understand your personal interest in Anthropic without mediation through an AI system, and we also want to evaluate your non-AI-assisted communication skills. Please indicate 'Yes' if you have read and agree.
And notice how few benchmarks will include Sonnet 3.5/7 in their new model announcements, because they’re awful compared to Sonnet.
What’s going on here?
If the bar lets Nazi's in, it's a Nazi bar.
> Claude Code keeps doing stuff. It keeps solving massive problems, one after another. I throw larger and larger things at it, and it is unfazed. Chomp. Chomp. Chomp.
So I understand his request as more along the lines of, can you explain this in a way that I understand it. For which summarization is the wrong phrasing.
ie.. It seems trivial on hacker news, but that post would be pure giberish for most of our parents.
Please like and follow the smiling non threatening avatar of the author