This is my disconnect with this kind of speed, how are they avoiding the janky crap? Or are they just accepting it and hoping it won't be too crappy or be found soon enough?
I don't have any insight into Anthropic but it seems like that's the case which is just depressing.
Using Claude code for a few days gives the answer pretty quickly: they fully embrace the janky crap and ship it to their users without second thoughts. And as long as their users don't mind using their piece of shit, why should they do otherwise?
Not throwing endless resources and energies at a problem "because Twitter told me what the best practices are".
E.g. Maybe Claude Code could use half the memory, maybe, but maybe the correct estimation is that solving the memory problem would negatively effect development speed, or would cost many millions of engineering effort and no real world user would really care whether it's using 50 or 500 MBs of ram.
So no, you're not an engineer but a "developer", if you think that "quality" code is engineering, code has a development and maintenance cost, specific directions come with important trade offs.
But since it isn't (we know real users care and complain, we know it's not your made up "endless resources" and "millions of engineering effort" because "millions" of other software products exist that have been made without those) you're short of an actual argument.
> code has a development and maintenance cost, specific directions come with important trade offs.
Yet again you forgot to make a case how any of those trade off towards the current state.
In general, how would you ever determine whether the current situation is justified or not if nothing in your strawman questions anything, but just comes up with not very plausible justifications of whatever already exists???
That has no bearing on the product i maintain, where users very much have a choice, and we keep them with us by solving their problems and listening to their concerns.
But real users do that about pretty much every piece of software, correct? Free SW, expensive SW, open source, closed, SW by a single person, SW from a massive company, etc.
That's not true, you totally can: "gud enough."
Work in general is.
Heck, life in general is
There is an assumption here that everyone thinks of excellence the way you do.
There are also some pretty detailed videos out there on YouTube of Anthropic employees showing how they are using cloud code on several orders of magnitude better than where the bulk of usage is.
What actual bugs do people have with it, and why do we think it is worse than a human coded tool?
The fact is people at Anthropic won't notice it because they are running it on M5 laptops
It's easily noticeable. Codex don't take half as much most times.
Also, the fact that they managed to leak their own code base, twice, tells you all you need to know about software quality at Anthropic.
as long as something works "good enough" and the code will never be reviewed theres only incentive to blast through tokens generating superfluous code. god forbid they actually ship a vibe coded product then they can basically ransom you for tokens
im probably being overly cynical but it sort of just seems like standard capitalism to me
Yes, you'll pay hundreds to thousands of dollars in tokens per month for the rest of your career.
What is your point?
In the end, what matters is whether something fulfills its purpose, not how it does it. Code quality, understanding code, etc. are just a means to this goal.
Some level of poor quality is acceptable, and I assume they are hoping that model development (i.e. its capability to fix and clean up an increasingly convoluted mess) will outpace the tangledness of the code.
So far, it seems like they are winning. Yes, the UI is often broken (e.g. for a long time switching away from an ongoing conversation lost it), but it wasn't broken enough for people to switch to other companies that may have had a better UI but in exchange for that a less developed product. If Anthropic instead spent its resources on making the UI not janky and code reviewing everything, the overall product would have likely been worse because they couldn't build and improve features as quickly.
I've seen the flip side where "code quality" becomes its own independent goal and more time is spent gratifying various linters than writing the code, and the inability to get anything done in such an environment is a lot more depressing than janky UX that can probably be fixed by prompting the next generation model to clean it up.
All software is broken, but LLMs seem to make the wrong compromises consistently when working in large codebases.
Interface design is one of the things Claude is worst at in my experience (using it every day for the past year).
Ideally, you design patterns that enable quick replacing of parts that get to messy. The code is cheap, just start from scratch with the defined functionality pre defined instead of bolted on with jank.
My point is more about up-front investment paying dividends. To speculate, it seems like the reward hacking is not there for long-term thinking.
What would they be?
I agree with that in the pre-LLM world, but I don’t believe it’s the same today.
You can build something with “bad” code that works and is full of technical debt, but today paying this debt is getting cheaper and cheaper.
So if Claude’s wrong design decision leads to having to refactor several chunks of a codebase in order to implement a new feature, does it really matter if it’s Claude who is going to refactor it in a matter of seconds? Especially for a company like Anthropic which has a virtually unlimited number of tokens to spend on it.
Like before AI, that debt is never going to get paid. Because how exactly are you paying off that debt when the means of paying are debt generators?
This is a fallacy. Computers are very deterministic, a well designed program will not have any bugs or unforeseen failures at all. You're not planting a tree and going "oh god I hope it survives the winter", you're programming a human-made computer, made of hardware that undergoes such incredible scrutiny before it's ever powered on that you can trust that it executes the same code, the same way, every single time, for decades.
No idea why people who clearly don't understand the difference between poppy farms and computers come in here and claim the world isn't perfect and so we should all accept sloppy shit code that barely works. The world isn't perfect, but software CAN be quite close.
Every single instance of jank on computers is a bug that someone wrote, or an LLM wrote, that can be found, understood, and fixed. And only once you understand this, can you talk about software quality to other engineers. You need to understand that the baseline is that software is not fallible the way the rest of the world is.
See sqlite3 if you want to know what nearly bug-free software looks like. I can't point you to a potato farm or an artist who is perfect, but I can point you to that and say "this is where we go when humans do a good job".
I don't know where you think you've found a fallacy. Computing systems are expected to interact in various ways with the outside world, which does include poppy farms, and people, and other decidedly imperfect things.
IMO the issue with LLMs is they massively widen the "unknown unknowns" (skipping review makes this problem even worse). This is obviously much worse when an LLM is actually a part of the software rather than just producing software that is otherwise deterministic.
The thread is about jankiness, lack of feature cohesion and code quality.
What we are discussing here is: is it acceptable to accept poor code quality and trade it off with delivery?
I found that people that lean towards the latter also favor profits over customer satisfaction. Then they will try to defend themselves with a black and white view “it’s either this scrap or no delivery”.
1. Have you seen that? If so, which agent+model?
2. Is “per day” intentionally used instead of per implementation - i.e a coding agent may A) be far faster tok/s than a human, and B) work more hours per day, so it’s similar bug rate per implementation? Or did you mean bugs per implementation?
1. Are you referring to LLMs handling 100x as many patches as a human can, or that LLM implementations are 100x as long as a human's implementation?
2. In my experience, LLM implementations are often 2-4x as long, largely because they have a lot more error handling than I would have chosen to implement. However, I don't find they have correspondingly X times as many mistakes.
3. Part of how agents seem to avoid many mistakes or even catch mistakes is by testing things. For example, it caught issues in Pester functions on certain Pester versions, or will even analyze binaries to verify the right config settings to use, etc.
4. If you are seeing 100x as many bugs, which agent+model?
It's ludicrous that we have executives at tech companies that think this is possible.
One S&P 500 company I worked for signed a big contract to deliver bugless software for an enterprise web portal for the state of Minnesota. I was on the team that had to make that magic happen. I had to leave the company not long after that due to nearly life-ending burnout.
So I would throw scorn at the ivory tower inhabitants here who should know better than to suggest that anything of substantial complexity can be bug free
I do think the world would be better if commercial software had to meet some professional standard of fitness, with meaningful repercussions if not. I also realize that ship apparently sailed before i was born, and I'm not paid to do serious engineering, I'm paid to ship features.
(Not even saying I'm qualified to do the former anyway, i just also think every time national air traffic is shut down or like in France the other day and hundreds of supermarkets lose all their frozen inventory, or i can't do my job because some service failed, or yes, claude code is slow and heavy age can't work right in every terminal, that maybe we shouldn't be writing the stuff that the whole world relies on to function every day with zero agreed upon professional standards)
You can only say that of either a very simple program or perhaps one with formal verification (which is overkill for 99.99% of software that gets written).
Look at the Linux kernel, arguably a well designed system (unless you ask Tanenbaum) with top notch developers, and still is full of bugs and security holes.
But I also would like to point out that comparing the Linux Kernel quality and development to Claude Code is surely a joke. The former is an ocean of features, compatibility and history. The latter, in the end still one executable CLI tool calling endpoints.
They're not even comparable.
By the way, all hardware has bugs too; we just don't see them often because the programmers building the stacks that we run our code on work around them.
Nobody has to make the perfect program, but we can't chalk it up to "the world isn't perfect" when I click the windows start menu and it opens empty (an existing bug btw)
How many large github repos have no bugs, not many. How much succesful commercial software does not have a way to patch their code, not much.
Reporting bugs, patching systems, updating software is common to most software.
Yes, but this is a side-effect of trying to do good work and writing code that is understood and works.
When you ship sloppy, AI generated code, that no humans at the company have read and understood, you're missing the main part about engineering and are skipping straight to the life-support stage of the software, and you treat the (previously) side effect as your main job. Reporting bugs and shipping patches becomes your job, instead of engineering software, which happens to also come with that task.
I expect people who are skillful and enjoy writing good code, are among those who dislike code generated with AI tools, and perhaps feel they have had a role or pleasure denied to them.
Those who write shit code, now feel that can produce vastly better software in a fraction of the time, and feel a sense of accomplishment.
That's a thought-terminating cliche. When someone asks how can they can avoid generating a pile of slop (which will compound over time and get increasingly harder to deal with), they're not seeking "perfection", and framing it in those terms is dishonest.
Furthermore, can we step back for a second and remember that user and developer experience matters too?
Working with code like that sucks. Spending much of your day on polishing a turd that grows more unwieldy with every iteration is going to get old, fast. And I also think that's going to take a significant mental health toll on people eventually.
The business won't care about this but we should. AI was supposed to make things better, better quality, better reviews, better QA, better product, better UX - remember?
> janky UX that can probably be fixed by prompting the next generation model to clean it up
I'll counter by pointing out that "Next one will fix everything" has been a meme for at least 2 years now and the slop just keeps getting sloppier.
Monkey's paw curls
Which is what the developer should actually be paying attention to, but I suppose they are just adding features for the sake of adding features.
It might be fully automated and eventually the agents figure out all of the bugs, but I suspect the costs are so large that no sane company would ever write code this way, and probably WAAAY more expensive than just hiring humans to write and test it.
There was a point where they were also rendering images as base64 text.
Can you point out the hyperbole to me?
made me throw up in my mouth a little bit.
The root of this challenge comes from more text aka more code ~= more tokens, so these firms have little incentive to optimize for that mindset. Sure, they might have more optimized individual responses - but the end result is still giving people what they want: more features, more documentation, nice, long, verbose and well-formatted Jira comments, of sometimes questionable but sometimes undeniably impressive quality. The band marches on, and when your codebase outgrows the constraints of the tried-and-true tools, you can just invent your own. With documentation and an MCP or two, or ten -- to also pull into your context window.
[0] I work on an operating system, so a suggested retort is to say this is me pushing responsibility and logic to the people higher up on the stack ;-)
I'm not sure if this is true. The code is such a small part of the total output. Thinking harder, like what would be needed for architecturing better code, would use more token.
They're obviously just accepting, have you seen any of their results? No internal insights required for these observations. Check out their clauded home page - try to open the right hamburger menu and click on 'policy' - it's impossible to do from the top because "Research" will expand on hover and push 'policy' down, and when you move further down, 'research' will collapse and 'policy' will jump up.
Likewise, the AI apps are infamous for their poor performance and buggy nature. Correctness is the price of such velocity
on mobile it works (where I would expect the hamburger menu), and on usual desktop I have the usual menu bar, but when I change the window size on desktop I get the hamburger menu (and then the UX jankiness is evident, but for me easy to work around, as moving the mouse down closes the "research" block and then it's quite simple to move it up to be able to click "policy")
at this point I think Opus does something better by default
By the way, the hamburger shouldn't even be there as there is enough space to fit the full menu (and even the company name) at the viewport sizes I've checked
Current agents never take a step back to refactor and to create a good abstraction as the requirements evolve, they only care for delivering fast even at the price of garbage code which doesn’t scale
Certainly, it would be super super difficult to generate training data for it, so it's unlikely that it will improve.
Because of the scale that would be needed. Like, design stuff is much fluffier than code correctness or functionality, so really really hard to generate enough synthetic data to tune the models to do better.
In the limit yeah it would work, but in the limit we're all dead ;)
I e., a responsible developer who cares about quality will have to get their hands a little dirty with some maintenance now and then.
So basically -- it's all crap.
I regularly run "experiments" on Azure to figure out what it actually does.
I'll save my nice words elsewhere.
Is my suspicion wrong? Are models getting better fast enough that they'll be able to add new feature (or fix bugs) and clean up even despite a mountain of AI tech debt [1]
[1]: I say AI tech debt because it tends to have some characteristics that are different from human tech debt, such as apparent lack of intentionality (which makes Chesterton's fence evaluation much harder). Perhaps if we make the AI write down somewhere, as briefly as possible, why it wrote each chunk of code...
Well-configured agents do, in comments and commit messages and docs.
But many people are just stuck thinking in GPT-3 era, where the prompt was all there was to it. It's not. You write a 1000 token prompt describing what you want, and the agent pulls in 10k tokens of system prompt, then spends time exploring the codebase and calling tools and attending to 100k of more tokens from this, possibly including output of sub-sessions that do the above, and somewhere along the way, having read lots of code, run lots of tests, perhaps doing a spike or two, the agent then decides some chunk of code needs to be written in this way, and not in some other way.
The reason is not contained in your original prompt at all. It's almost entirely based on what the agent learned about the current state of your codebase and environment.
In my case, “Factory” is hundreds of architectural tests built over thousands of application specific tests, and a couple MD files here or there. The MDs are mostly reference material and common patterns. The tests are what actually enforce feature/module isolation and proper architecture.
Lately, I’ve been getting overzealous with protobuffers and service discovery as well. I would never use these tools so frequently if I was designing for humans. With AI I’ll slap annoying boundaries in to enforce separation of concerns without thinking twice.
Making a new project is a pain at this stage, but you can just point to CI and the general structure in an existing project and AI will clone the important bits. I suspect high level architectural rules with programmatic enforcement will eventually be encoded into tools for easily starting new projects. Kind of like pnpm installing specific development practices and architectures.
I used to do a lot of work with reflection, compile hooks, and dynamic source generation before AI. You can do a lot to compensate for the architectural blind spots of AI if you get creative.
How about code blind spots like endless code duplication? Reading the whole DB to read one record? Slow algorithms? Bad data access patterns? Excessive defensive programming? Inventing new issues just because your "high level architectural creativity" mentioned something off-hand? Etc. etc.
On top of that, where are your "hundreds of architectural tests built over thousands of application specific tests" coming from? Also written by LLMs I presume?
All of these are endemic problems in this industry, openly discussed for decades before LLMs were a thing.
But the pragmatic reality is the same for people and agents - those "mistakes" are common; juniors make them because they don't know better, seniors make them for good reason (pragmatic tradeoffs). Good teams know how to prioritize, and generally fix them when they start causing problems. Agents are very much capable of the same, fully autonomously these days.
Personally, I've had plenty of cases where an LLM came out with a sleak, and arguably over-optimized algorithm or DB query - but when I looked closer at what it did, I saw it first came out with something dumb and claimed a win, then run it, realized it's stupidly inefficient, and optimized it on its own, before giving the final version to me. All in a single turn, without being explicitly prompted to work like this.
Agents don't learn. They rebuild a lossy context about your codebase from scratch every time, and then produce lossy output based on that context.
> You write a 1000 token prompt describing what you want, and the agent pulls in 10k tokens of system prompt
And at this point its context is so polluted so it doesn't matter
Same is true about employees and interns, from any company's POV. And in large companies with large projects, it's true from POV of employees too.
--
But let me spell out my original comment, to clarify:
Agents are like interns: they come to do the job, having none of the knowledge specific to your codebase and company. They have no time, to properly study the whole thing. They get a task, and are expected to figure it out (with help of documentation and people). Then they do the thing, and regardless of the outcome, are gone few months later, and you never hear from them again.
(In many companies, for various reasons, juniors are effectively interns with larger context window.)
In large companies with large projects, everything is moving so fast that no one - no one - can possibly understand the whole thing, and the work paradigm is purposefully structured so they don't need to. In some sense, they're like perpetual interns - each time they start working on different area of the project, they know next to nothing about it, and do not have a chance to fully comprehend it. They just need to figure out enough to deliver on their tasks.
(I know, because I like understanding things, and in one place I was told explicitly it's a common trap for people like me, and I need to learn to suppress it if I am to be effective at my job.)
Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well. It may not be optimal, but it works, which means agentic version of it has every right to work and scale as well.
> Agents are like interns: they come to do the job,
They don't come to do anything. Stop this infantile antromporphising of LLMs.
They are nothing like interns. Interns learn. And companies don't develop products by getting a new batch of interns every day.
> Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well.
No, it's not even close to proof. The conclusion literally doesn't follow from anything in your prevuous text.
If your agents are "perpetual interns" then this is proof that no, it doesn't work and it doesn't scale.
As for large companies... You don't have to know the whole thing, but you do have to know your area. You don't come into work at a large company always asking "what is it I am doing" from scratch every single day.
> They are nothing like interns. Interns learn.
And then they go elsewhere, so no, from POV of the company, if you treat "internship period" as "session", they don't learn. But they leave behind documentation and changes to code.
I guess another way to put my point is: the idea of code being self-documenting isn't completely flawed. It works to a degree; in fast-moving teams in large companies, you don't have any other ground truth anyway (documentation is always stale, design docs become obsolete the day they're approved, etc.). People manage. And so can agents.
> but you do have to know your area
It so happens that LLMs know nearly every single area of human endeavor better than most practitioners :).
No, it shouldn't. If you compare agents to interns then you have a new batch of interns not even every day, but every few hours (with every new session, and every new subagent).
> so no, from POV of the company, if you treat "internship period" as "session", they don't learn.
So,
- interns are a very tiny fraction of a workforce at any given company
- no company is ever building the bulk of their products with interns alone
- from POV of the company interns never learn, and the new interns should be taught everything from scratch, again
Somehow, from all this, your conclusion is that "it's proof that it works and can scale"
> It so happens that LLMs know nearly every single area of human endeavor
They have very approximate knowledge of those areas + this knowledge is rather randomized in the output. On top of that they are trained on publicly available code which is shit. Plus they know nothing (and can never learn anything) form your particular code, and your particular project.
Nope you are 100% correct. All of the models will code themselves into local maxima then spend more and more time chasing their tails as every change introduces new bugs, the agent adds more complexity, and the code base just spirals. Best case you end up with a bunch of parallel implementations that all work slightly differently.
It helps a lot of you pay attention to what it is doing and impose order, then regularly spend time refactoring - even if you are having claude do the work itself - you have to spend the time on the overall architecture and design.
I just experienced this with an internal productivity tool. Without claude this tool wouldn't exist - there would never be the justification to take the required time to build it. On the other hand all the interlocking refreshing rules for sourcing data from its inputs got so byzantine it was impossible to add new features or change anything without causing new regressions. Claude built a web of ad-hoc timers, refresh kick-offs, monitors, etc to solve each feature problem or fix some bug but the whole thing was impossible even for claude to understand. It took a special large workflow session to find most of the mess and even then during the refactor it kept finding new things.
I had to stop all feature work for going on four weeks and burn a ton of tokens to build a refreshing/syncing engine that could track dependencies, priorities, etc and fully test it and just now today I am finally able to get feature work going again.
On another feature an agent came up with a state object containing 55 inter-related fields with all kinds of interlocking dependencies. Something so complex no one could hope to understand it. And I find the agents aren't really much better at understanding such things than humans are. If you leave it like that the agent will just perform worse and worse on that part of the code as time goes on. It will make assumptions that force it into bad design corners, miss edge cases, and flat out miss things no matter how well you prompt. Agents benefit from encapsulation and simplicity just as much as humans.
If you cranked up turns and only used fable level model for every interaction, you would get way way less jank. Plus when you have essentially unlimited tokens, you can always do extra AI review passes to reduce jank.
I think for us, outside of Anthropic, it is hard to imagine the scale of AI agents that they are running internally . We are talking about probably 10s of millions $ worth of spend monthly if not more.
For now at least, I'd say churning code faster and faster cannot widen either of these bottlenecks much.
Today, I use Claude heavily, but review all the code. I'm faster, but not by the multiples people speak of. And that seems ok to me.
Keep in mind that Anthropic is first and foremost trying to sell you something.
I'm very curious about that because that has not been my experience at all. I used to find janky crap from older models and I had to review every line of code very carefully. With newer models, and especially large ones like Fable, it just doesn't happen anymore. The code that I get is what I would have written. Actually, it's usually better because it covers more edge cases that I wouldn't have thought about.
At this point, I guide Claude Code, provide general direction, and make design decisions, but I rarely write code anymore. The code that I get from the LLM is just better.
I do not fully understand where the disconnect comes from. I do believe people that say that they get crap from LLMs, and I suspect I can guess some of the reasons for that, but I cannot be sure.
> I rarely write code anymore. The code that I get from the LLM is just better.
Coding chatbots excel at leaping low bars.
Same here. There is an introspective comment below by khazhoux asking the question from the opposite experience. Why do developers end up on different sides of this coin?
LLM capability is obviously important variable. But those with bad experiences report using top tier models, so that isn't it. Likewise, level of experience can be ruled out. Each side of the coin have experienced developers reporting.
A lot of the other variables are not reported but I can go from my own experience.
Greenfield vs brownfield? The first is indeed usually easier but I get great results updating large code bases with decades of development. And I suspect developing greenfield with LLM is easier as the code is better written from the start.
Rarity of the subject matter? In principle, yet another XYZ app or web page is well represented in the training data so the LLM should have an easy go at making similar. In the past with low-end LLMs, this was noticeable. Working in relatively obscure programming languages produced just garbage. But, that was prior to the inflection point. And today I get good results working on relatively arcane scientific/mathematical algorithm as I do when I ask the LLM to barf out some web app.
Different levels of testing and validation and expectation? Even best LLMs make mistakes. The better LLMs make fewer and expose fewer still to the human as they create internal tests. Maybe people have different expectations and those with bad experience expect perfection from a single prompt? Perhaps, but any experienced developer knows bugs are a part of software development.
What else?
Whatever the answer, it's not a simple one and it may be a confluence of many smaller things. Maybe different developers have a multi-variant function in their brain that pops out a different quality-of-software score and also a different threshold that divides good from bad and all we see in their reports is that binary conclusion.
They aren’t. Their source leak showed what a mess of a codebase they’re working with. That was, I assume built with unlimited access to frontier models as well.
For example I've made a first screen with a lot of elements, and now I want to create a 2nd similar screen, so I tell it to use the same code as we've created for the first to make the 2nd. With that as input it will create it and a) use e.g. the variables and setup from the first b) copy everything into the 2nd c) copy some things d) make a generic parent for both to inherit from
A human would have realized that this is something we're going to be doing a lot, so the last is the correct answer. Make generics for all elements, e.g. the screen's panel, effects, buttons, text etc. then implement those along with possible overrides etc. for all.
But it seems that it has no clear methodology. Depending on your exact wording along with today's weather you will get any combination of the options.
As expected. That woukd require domain understanding, and that's way beyond the scope of stochatic parrotry.
Because you are using LLMs to write code that you would be writing by hand.
Use it to write thing you wouldn't have time to write by hand and that are not mission critical: custom linters, support tools and so on. You know, each time you think to yourself "hey, I would write something to automate this chore, if only I could set aside one or two days".
You can also use it to document silly things you wouldn't take time to but that provide context to agents. Ask it to do a performance audit and to give you a few SQL queries and bash commands to gather stats about prod and logs. Then commit this document.
Will you read it? No. But your agent will use it to write code that's fit.
Keep doing this and 1/ you will be more productive, 2/ your agent will progressively improve so that you will let it do more and more.
The secret is their code is shockingly bad but they have unlimited budget to fix it.
By "Real" in this context I mean companies that aren't Anthropic .i.e. don't sell any kinda AI product and existed at least 10 years before the AI boom. Esp. in regulated or at least sensitive fields like finance, banking, medical, logistics etc.
Have fun playing with it. I’m not responsible if it looses your data :)
They are just shipping the jank. Claude couldn't make sense of its own built in tools for like, months, if it's even fixed
I think it is mostly this and my experience is that agents feel much more at home here. But on the other hand monorepos disadvantages become even more pronounced with agent work. It will become an unmaintainable mess, even if your agent can iterate through it regularly.
I still think building a pipeline that allows for separation of concerns is a requirement for maintainability. Sure, Claude excels to build software if I just give him a server where he can run wild. Any new requirement is an extension of the old repo.
But at some point you just have a giant mess and I am not really convinced of this approach, even if it certainly fits agents.
Just do this for every type of "janky crap" and your codebase converges to perfection.
It's not that hard, but it takes a lot of tokens, and setup.
It's always total hallucinations. It could only see local minor issues.
It happily starts to introduce fake abstractions and other junior level "refactoring" making it even worse.
My understanding is that they have manual reviewers in phase 1 of their projects. Their goal is to refine the automation of reviewing, writing skills/rules, etc, enforcing architectural choices, etc.
Then in phase 2, the review phase is AI-driven itself and humans only get to be decision makers of escalated issues.
And yes, they likely accept that not every line of code is clean.
But honestly (and unfortunately), average source code is nothing stellar either. I do SW dev for 30 years. And most projects I have seen were somewhat ugly.
“Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.“
The difference between intent and implementation seems like a very common source of bugs?
If intent and implementation was the same thing programmers as a profession would have been left behind 20 years ago.
That’s the neat part! They don’t!
Your bar for quality must be pretty low if you feel this is better than average?
My favourite is Claude Code's /rewind feature, which gets so many things wrong for having such a simple UI. (Does selecting the default option still wipe your filesystem changes by default when going back to a previous entry? I found a poorly-documented way to circumvent that "feature", which only adds to my initial point.)
Theyre usable but leave alot to be desired. SOme other tools like opencode seem sometimes to have less feature (if exact numbers) but at least always work.
Somehow with Claude and some others, random simple thigns dont work. Esp long solved problems like TUI rendering, special chars, breaking changes every release etc.
Surely as always with software this is my experience and might be not yours.
PS: Ive used IntelliJ since... always?. It was quite decent. But these last years it is just so diferent, somehow way less quality, so many random bugs that Im afraid theyre doing AI agentic coding which sucks for such an important tool.
IDEs should be build from real professional crafstman with an exact vision in their heads, not some wild AI.
Im starting to think in a year or two "We dont use AI in developing our products" will be some kind of marketing trick since people will start looking for that.
Not. The web UI is the jankiest web UI that I've ever seen. E.g. its mobile responsiveness frequently crops dialogs leaving the close box unreachable.
I mean given all the sandbox escapes, and the leaked Claude codebase, I would assume they stopped reading their own code a long time ago.