How I use AI (Oct 2026)
ben.stolovitz.com
ben.stolovitz.com
> My most successful work this year was hand-writing a tool, replacing ~4,000 lines of vibe-code with a few hundred. My system has proven more flexible, easier to test, and more generalizable than what it replaced.
> agents still need human taste and experience to produce good work.
Where do you think human taste and experience in software comes from?
> overstimulating & exhausting in a way that coding never was.
> for many of us, honing the code, finding the right design, seeing the parts mesh together — that was the meditative purpose we found in our jobs.
> That’s not the only time AI waylaid me. LLMs could not figure out how to annotate that model’s training data correctly. I spent days going back and forth with them
> LLMs are not yet good companions when brainstorming complicated things.
Why do we expect this to get better? The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
> Writing code is solved.
What a strange thing to say given all the above experiences, which I think we all share.
Why are we willing to give up so much in terms of purpose and understanding for perceived productivity gains?
This always reads like “writing english is solved” just because you collected snippets of famous works and have built a spell corrector.
Code is written to process data. The struggle was never to learn programming syntax (most grammar are only a few page long). The main struggle was that it’s hard to model a domain into an information system, identifying the components and their relationships, what is ingested, and what needs to be output.
Computers have integers and floats, but you need to implement units like distance, money, various counts,… The real world have words and sentences, but that needs encoding like ascii and unicode as well as the “string” concepts to make it work. Take HN, I’d bet that if you can model it in terms of data, writing code is not an issue.
For 85-90% of devs, yes this was the struggle. Creating and updating rows in a database from user input is all the “domain modeling” there is in most software. That’s what is solved by AI and it’s the majority of software dev.
If anything, I do expect the casual non-tech users to be making the most adaptable software because they have what they want in their head. Although, part of what the dev does is figuring out the weeds, and only a part of users have the patience to sit through detail corrections. And the AI user has to verbalize their wants exactly too.
RL is not dependent on training data. The mathocalypse didn't happen because they found better training data somewhere.
There are reasons to suspect that OpenAI may have intentionally or unintentionally trained on chat logs from mathematicians--mathematicians who were actively using OpenAI to try to to solve the same problems themselves.[1] If those mathematicians were on the right track, and getting close to a solution, that could be classified as finding "better training data somewhere."
[1] https://cybernews.com/ai-news/openai-accused-stealing-anothe...
As a side effect having to read the code at least once inevitably makes it way shorter and usually much better.
Guess what: because I said "both", Opus decided I was looking to pick out exactly two of the issues. If I hadn't read that reasoning log, I would've remained oblivious and thought there were only two it could find.
This is like writing `if (cond_x && cond_y) {}` and then blaming the compiler because it didn't realise that actually meant to write `if (cond_x || cond_y) {}`.
I'd be curious to learn how you're using them.
However, I'm still split. Because this could mean that we see a jump in code quality, due to bad coders getting someone better to code for them. But we could see a decrease in code quality, due to less people "caring".
There is still questions about if that jira ticket should have been written but from ticket to mergable PR it's better than me.
It's crazy realizing that Claude Code only started getting big in the last year or so. It feels so different now, with all the new models & tools we have to use. I hope you enjoy!
LLMs still mess up a lot if you just let them loose and don't guide what they do.
We're in our own bubbles where a good part of HN population woke up to coding agents being good enough ~1 year ago, there's a group of folks that found out ~6 months before that, and then the people banging their heads on these a bit earlier than that. If you aren't using a coding agent everyday most people are just using these things to apply an instagram filter, or sort a grocery list. Muse is probably the closest thing to 'full automation' a lot of people are seeing for the first time.
Unfortunately I don't think the bubble that exists at the moment is on the promise of automating so much coding as trying to penetrate consumer use cases, which I think are still lacking and not reliable enough yet. As much as I use coding agents every day I still will not let a claw-like near a bank account or a credit card.
While trying to capture that bug, I (ironically) asked Gemini (but the Gemini-via-Google-search Gemini, since Gemini proper was an outage) for the package name of a screen recorder (gtkRecordMyDesktop); the command it returned referenced a non-existent package¹. I then found an alternative, and then hit a bug in that screen recording software that I had to then work around to get the video file there.
The site I uploaded the video to, Imgur, breaks completely — i.e., it cannot even so much as surface an error message — if your browser refuses to let it store god knows what on your disks. (So as it is, I have an extension that says "why yes, of course you can have storage, and essentially provides a black hole. But since it looks like a "success" to the Javascript, the JS doesn't crash!)
This isn't, to me, some one-off interaction. Every interaction with software these days feels like this, more and more. Just breakages everywhere that leaves me wondering if "coding is solved", where is all the bug free software?
(¹forcefully prompting the AI to actually query the package listing netted me a classic "You're absolutely right! The package name I referenced does not exist!")
My experience is the same as yours. Every single service I use. Apps, websites, doesn't matter what. Every single one seems just softly degraded. Like I can't trust that features work any more. I could report the bug, but what's the point? It's going to go to some Claude triage agent who is going to "fix" it by slopping up the app even more. Do people even use their own websites and apps any more? Apparently not - the same people declaring software solved are having their AI agents doing computer use and going on these websites for them!
Claude, please read the Bible for me. Tell me what I need to do to get into Heaven. Oh, can you dictate it to me? I'm managing an agent swarm.
Bleh.
The comments to this article suggest that it's still disputed.
Personally, I think it's a bit of a silly phrase, because it implies an absolute, and all anyone needs to do to disprove it is to show that coding was not solved for problem X. That's still the case -- it's just that the set of X is rapidly shrinking.
I think it's more indicative to say how much you trust LLM code. A little over a year ago, I scrutinised every line carefully, and it was buggy as hell and I had to fix most generated code by hand. Now, models write clean code with fewer bugs (far fewer) than I would. A year ago, I was doing a lot of steering during planning; now, it's rare that I don't agree with the model's first proposed solution, and even if I don't, it's most likely to fix a minor change in understanding, not to propose a substantial architectural change as it was before.
The one thing I'm not sure about is how much this is due to better prompting on my part. I doubt that this contributes much, but I still suspect that continual usage does make you more aware of what works and what does not: how much detail to provide, how to describe a problem or change, how to propose solutions while still leaving other options open.
> my uncle can create software
Is considered great
> my uncle can create music or art
Is not considered great.
I guess humans have agreed, that art is a skill you must learn if you want to produce it, but engineering or software is not a skill worth learning and we are happy if computers can do that for us.
So it’s great and amazing if anyone can create their gizmos and software to help them but not if they create music or digital images, which apparently it is wrong to prompt, and not a sign of a great future.
If you specified to someone that the AI is replacing the farting-into-mic-core music genre, they would not care. Same with programming, the average person is not exposed to this
Big +1 to that.
...
>In case you’re wondering, I’m subscribed to: > Claude Pro ($17/ mo) > GitHub Copilot Pro ($10/ mo)
Nothing wrong there at all, amirite?
Maybe AI scrapped basic arithmetic and now those numbers are totally fine
That is both untrue, and should be something that occurs rarely.
Past Mid level, most dicussion in code review is centered around the conceptual level. Code is mostly use to highlight the point being made. I’ve never seen the need for someone to rewrite a patch proposed by another person. Add to it or correct misunderstandings, but never sweeping changes.
I think I would rather shoot myself in the head
I walked into a paint store recently and there was some interesting music playing on the PA. It was some sort of spanish-language 80s new wave I'd never heard before. It had a nice groove. I asked the guy at the counter about it and was surprised to find out it was AI.
So... was it as good as great music? No. But it beat the hell out of muzak. And it was genuinely novel in a way that pop music isn't. I will sacrifice a lot of pride for novelty.
AI music is a complete waste of compute. It exists only to further enrich an already wealthy stakeholder, and we should all be ashamed to have ended up in a place where it even exists.
And I think it can be uniquely fun, because it allows for the very rare combination of truly off-beat ideas with excellent execution.
In real life, it very rarely happens that somebody will spend a lifetime learning, gather a band, and then record something extremely off-beat. Because there are risks to that. Destroying your image, having to find people that are on the same wavelength, having to justify the time and money it costs to record something weird and not marketable.
Sure, people make weird stuff once in a while, but very rarely, and generally it takes unique individuals that find some particular weird niche to work in, like Weird Al or Max Raabe. And even then they tend to pick something relatively tame and marketable and stick to that.
But AI can bypass all that, and that makes things interesting.
My jaw literally dropped. Do you really believe this?
I think you just don't spend much time around people who love music.
It makes sense that someone like you would like AI music, I guess. You are clueless.
Sure. It happens in every field. When you invest a lot into something, it becomes risky or unprofitable to do some things.
Tech often drives things in the opposite direction. As it gets easier/cheaper/faster to create something, doing something odd and risky becomes easier to justify because there's less invested into it and there's fewer people whose cooperation is needed.
I'm not saying nobody does anything risky traditionally of course. But there's barriers to it, which become lesser once tech trivializes a good part of the process.
> It makes sense that someone like you would like AI music, I guess. You are clueless.
I like music in general, and am fairly indifferent as to how the goose is cooked. I listen to almost everything. There's no reason to exclude AI music. If it sounds good, it's good.
>it was genuinely novel
Look, I don't know the song so I can't say how novel it is, but to me, it completely misses the point if its made by a music generation model. The point of music is not to make a bunch of novel sounds. Music is a means by which a human can express themselves, the whole _point_ of art is human expression.
Bird's aren't making art when they sing, I still like listening to birds. Why can't the sounds AI make be in the same category.
Human expression with art can go both ways from producing and experiencing.
For my part, I mostly dislike AI music. But if there was to be some AI music that I didn't already dislike for its content, then I would dislike it specifically for being AI. Call me close-minded if you will. Some people don't really like music at all. In general, I love music.
I saw a live band last week (daft punk meets prince), they wrote a song about Ai and used an auto tuner for it
> And it was genuinely novel in a way that pop music isn't. I will sacrifice a lot of pride for novelty.
For one thing, "interesting" and "novel" are not particularly the point of art. An infinite TikTok feed of AI slop is in some sense infinitely interesting and novel.
But also, those concepts are subjective. Pop music has a ton of evolution and innovation if you break it down. Sure, not every hit is doing something clever, but plenty of them are. You might not find those features to be interesting but it takes ingenuity to stand out in an era where the barrier to entry is so low.
2. Call it slop
3. Claim victory
Controversial take: AI shows that people simply don't give a shit and just confirms that mass culture is a mistake and the death of everything that is good and sacred.
Controversial take: For ages, it's been clear that most people have never given a shit, it just took AI for most to realize it.
I don't know what to tell you, but me and my kid are having a blast writing ridiculous songs and plopping them with song descriptions and some of those make it to our drive playlists.
What does amuse me to no end, however, is how many craftsfolk in AI circles are loudly rejecting AI imagery and video in favor of humans. The only folks who seem fine with that are the grifters looking to slop out a bunch of adverts quickly for commercial gain rather than do actual art, or folks who want “art” but don’t want to pay a human to make it for them or learn the skills themselves. Both of those camps are viewed with disdain and loathing by both sides of the AI Boom/Doom arguments, so I’ll take whatever allies I can get on those fronts.
AI music…man, I live with artisans and musicians and we have a number of “artists” we suspect are AI but whose music is pretty solid. We’re waiting for the shoe to drop, but I’m trying to look at it as a kind of turbocharged Fruity Loops/Logic as opposed to a human replacement. I hope I’m right on that, but time will tell.
The game changer I think more doomers need to experience is agents. It sounds like a recipe for disaster (a bunch of models known to hallucinate and allowed to run roughshod over a computer unattended? What could go wrong?), and it often (hilariously) is for the scammers out there giving it full access to production and fucking off to hawk their latest Masterclass on X or LinkedIn, but that first time you give it a reasonably achievable prompt and come back to a halfway decent product you can review and refine into production in 15m or less?
Absolutely magical, doubly so when it’s just 27B Qwen via Hermes and not some souped up stack in a frontier lab costing you thousands a month in tokens. If I were to evangelize AI, that’s how I’d do it. Fuck subscriptions, fuck cloud compute, fuck frontier labs and token buckets and whatever slop Sundar/Scam/Dario/Elmo et al are hawking this week; run that locally, give it a reasonable prompt that solves a real problem, and show them the results.
Then do the right thing and go for a walk with the two hours of free time the AI just freed up for you. Don’t ask for permission to take back your time.
And I agree that "less AI" edit passes reduce a lot of the hallmarks, but I think these models truly struggle with expressing complicated ideas. Have Opus been good at that for you?
> I still believe I use AI fairly typically, but I’ve had a notable change in background. Last October, I was an engineer on Microsoft’s Windows AI team. I am now a research engineer at Salesforce AI Research.
Priest wears robes.
I guess it’s one of those nonsensical automatic rules.
> I am now a research engineer at Salesforce AI Research. I spend more time supporting scientists & building rapid prototypes, rather than working with a large team of engineers.
I appreciate the PoV of people with different challenges, but my full-time job is not about building rapid prototypes so the guidelines are worth a bit less to me.