For example, you can imagine, at the extreme, that agents could get caught in code change loops: a code change seems appropriate, it is implemented, it turns out there's some external factor which makes it not appropriate, the change is reverted, then that same code change seems appropriate again, etc.
Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
Can you even do the latter without the former?
So yes you can and it works, sometimes…
Then again, my standard instruction is something like "{bug description}. Find the root cause and propose possible solutions" so I guess I fit your profile
This is true, but at 30,000 merges a day we're talking about nearly 1 million merges a month, and purely autonomous agentic merges tend to be noisy. Is this still a tractable problem, even for an agent? Do revisions older than a few days, other than perhaps release snapshots, have any relevance for such a process?
Of course that does require some folder structure, and agents are not necessarily great at that, but they're still decent enough that you do end up with many folders not being impacted hour or even daily.
I'd wage that volume of changes is not leading to forward progress, but is a clear sign of code churn.
This in turn is a clear tell they are dealing with spaghetti code.
At the end of the day it's "just" a management problem. If you can do something like, say, distill development activity into a hierarchical summary (i.e., this merge does "this," and these collection of merges do "these" things based on the "this" that each is described to do, etc.), then, with enough agents operating in parallel, I don't see why managing that isn't at least theoretically possible.
In practice this seems like a mess, but they're the ones with intelligence on tap lol
This is a great idea, now they just have to use/do it. Maybe they can use it effectively by cloning the repo and having all the history at hand, rather than continuously scraping web pages with git content formatted for human brwosing.
It's the right approach if your business is selling tokens and the customer doesn't know any better.
most of the harnesses are checkpointing most of the data of a git commit aside from the formal release and messaging, anyway; thats why they all have a similar problem with SQL dbs exploding in size after a bit of time.
Right now we're in this weird transition where most of the harness systems are layering git tool use w/ the constant snapshotting AND a memory subsystem often, probably leaves a lot on the table in saving wasted token use to just ditch the vcs all together and rely on the already persistent and existent checkpoints and memories for reconstruction.
Or code comments which they are already doing a lot of. I used to get annoyed by their verbose comments but at this point I've seen them help a few times and it's not like I'm actually maintaining the code myself anymore so it doesn't really bother me
1. they have no-cap token spend allowance internally
2. their agent swarms are amongst the largest if not the largest, again because #1 above.
3. their git repo is enormously large (100GB uncompressed git history on main, 25GB compressed).
4. their merges per day count grew an order of magnitude from May to September to 30000 times per day, they are not seeing it slowing down anytime and probably only going to accelerate.
5. all of their code agents are networked and has network access, full local pull down is not needed at all and too slow (see #3)
6. their design of the Scion is based off prior art Google’s Piper and Meta’s Sapling.
Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts. But this was NOT a complete redesign from scratch - so it's an full online download only what you need type of source control system. The problem is the version control software itself but its prior tradeoffs are misaligned with their scaling needs.
PS, if anyone noticed, Anthropic starts to concealing their employee full names and replaced with only first name + last name initiate, not sure why.
PPS, the project Scion is so bad, scn is worst, it reminds people of prior SVN, which is bad by itself.
I might be completely out of context, but why is it only 4x compression? Do they commit a lot of binaries? I just don't understand why would code compress so badly.
I mean 30k CLs per day isn't much. That's one every few seconds. Well within what a decent perforce setup can do with enough replicas.
It's always a question of opportunity cost. The culture that led to Piper also led to them creating their own IDEs and everything else, often with questionable outcomes. Google had the profitability to get away with that kind of profligacy but do the AI labs really want to go the same route?
Cider and Cider-V were both forks of existing IDEs with Google-specific stuff wedged in. They weren't greenfield projects.
Blaze was absolutely worth it. Nothing else comes close.
I'm sure not _every_ tooling project was a win, but I don't think it's fair to frame it as "every Cider engineer could have been ramping up remarketing and advertising instead." Google's early days involved trying everything and keeping what worked. The tooling improvements worked.
But... there's a lot of decisions that make sense for a company like Google or Meta that don't make sense for other companies. Google, Meta, Apple and a few others like them are nearly unique. They have extremely profitable quasi-monopolies that have proven impregnable to competitors, and the second price auction system they use automatically sets prices to whatever the market will bear.
Thanks to that they they can pay enormous salaries, fund many high risk/low ROI projects and still have lots left over. In such an organization projects like creating your own build system, VCS, IDEs, email system and so on can be rational, even though they can't actually measure the productivity impact of a developer tool to one percentage point of precision.
All this is nothing like normal business. Normal businesses face peer competitors, meaning pricing is under continuous downwards pressure. Across all industries the profit margin is only 10%, but that's an average that includes the tech sector. Industries with a lot of competition stabilize at margins around 3% e.g. supermarkets or airlines.
When we look at AI labs, what do we see? They spend like they are Google, but the market they're in looks nothing like ads or SaaS. Instead they are in vicious competition selling largely undifferentiated products to price sensitive customers who display almost no brand loyalty. Quite surprisingly to those of us who got used to the FAANG and SaaS era, AI firms look more like airlines than tech firms.
In normal companies the rules and culture are totally different. Decisions are driven by people with MBAs who use terms like "core competency", "outsourcing" and "margin compression". In-sourcing is rare because it's expensive and risky. If your margin is only 3% then you don't compete with your own supply chain unless it's absolutely required. Doing that without a shit ton of spreadsheets showing it's necessary is the sort of decision that will get you fired, pronto. You use things off the shelf. If nothing is available for free then you're expected to spend months in vendor negotiations trying to squeeze another 5% off the contract, and this is the kind of work valued in such a culture which is why MBAs rise to the top.
Historically the software industry has avoided this because of ads, the ZIRP era, lock-in, the explosive growth of VC up to 2017, and low growth in other sectors like pharma. But that era could never last forever and seems to be ending now. A strong signal is the growth of Bending Spoons, whose business model can be summed up as buying SaaS companies run by people who can't psychologically handle a high interest rate environment and then just running them like a normal business.
I realize acutely that the reason I don't run a top tier AI lab is because of this mindset, but if I were running Anthropic I'd be trying to find ways to cut costs dramatically. For a decade+ you never heard the words "tech startup" and "bankruptcy" together in a sentence, partly because VC firms forced firms to acquihire startups that weren't working out, making failure look like success. But this week Character.AI went into Chapter 11. None of us know the true figures but it looks like AI firms don't have the margins to justify acting like a Google or a successful SaaS. Corporate cultures imported from those companies risk blowing everything up.
Also IIRC from the original 2011 Perforce admins paper before Piper, they said they had seriously overspecced the RAM and could use a lot less. There's an annotated copy here: https://www.anantjain.xyz/posts/perforce
Gives me a vaguely creepy feeling, but it's reasonable.
For how the Linux Kernel, a highly decentralized, completely open source project, with a single monotonic product, tyrannically lead (a good thing) - is developed.
Permissions, Partial Checkouts, Sub Repos, Vendoring, pub/sub... How many companies bought into git and then had to buy more tooling (like artifactory) because their 3rd party dependency's needed a home. Thats a feature that git does not support because git, as designed, did not need it.
Think about how much better your life would be if every time you typed npm,pip,goget,cargo it fetched THROUGH your repo. If those updates triggered code reviews of whatever changes you're going to consume. You are code reviewing those updates aren't you (someone is, cause supply chain attacks don't last that long)
Git has some weak points, Google doesn't use it for a reason. Epic created lore for a reason, This exists for a reason. Im not sure that any of them are the next thing that replaces GIT but it's getting to be time. Tools like this are going to keep popping up till someone gets it right.
There are orgs that are moving to not including names at all as the next step on that trajectory.
Does it, though?
What problem do they actually have that requires redesigning a whole new vcs?
Merge conflicts? That's not exactly a vcs problem but a "we changed the same code at the same time" problem. No tooling on earth eliminates the need to verify conflict resolutions if agent A renames int foo to int bar while agent B renames int foo to int baz.
Growing repo size? They should take a long hard look at what they are doing. How is their software architected? Is every single little change and half-baked AI churn commit pushed to the main repository, or does the main repo only receive the deliverable? Is everything under the sun tracked in a mono repo or are projects stored independently?
Do they take any precaution at identifying churn and taking steps to minimize it?
Frankly, this reads like a bullshit PR stunt, much like the bullshit curl vulnerability findings. They have a self-inflicted problem, and their solution is to persist the problem and mitigate the problems by rolling out ad-hoc tooling? It reads like "if all you have is a hammer" kind of situation.
All of this may be slightly inspired by the Militech neural matrix in Cyberpunk 2077
A big motivator of thinking architecture through really well is that although the software may work just fine now, a good architecture will make it easy to add a feature a year from now, whereas a bad architecture will necessitate significant rewriting.
But significant rewriting is cheap nowadays, so in code reviews I put a lot more weight on whether it works today, and a lot less weight on whether we'll need to change it when making a different change.
Entire classes of things like "should this be in this component or split out into its own" just don't matter anymore because no matter how tedious changing that might be once there's a ton of things depending on it, that's the kind of thing AI can do no problem.
I tend in the other direction when I can (for components and microservices - harder for monorepos): I encourage refactors on a continual basis, as it prevents the models from getting bogged down with the way the system used to behave across changes.
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
Give it six months and there'll be have Optimus robots toggling switches to load terrabytes of binary programs directly into memory. Nobody wrote it, nobody will read it, but it will run everything.
that's not quite confirmation that assembly is source code
If you don’t feel that toggling the binary 16-bit words into RAM demonstrates that binary is what actually runs on the hardware, and that assembly is source code for a binary, well ok. I find it persuasive.
machine code is a language.
they are isomorphic, but different. assembly language (source) converts to machine code (destination).
assembly is... source. code.
machine language is not, unless you consider vliw or microcode, useful to think about but doesn't extend the analogy already in place.
I never said otherwise. I think we're talking past each other, so I'm out.
It works, but there is a significant amount of churn. I find when the swarms use git it's less churn and less token usage.
This is how we solve cybersecurity, by making code into constant shifting sand, so a vulnerability never truly lives long enough to be exploited at scale. Sure, a hacker may occasionally pick out a vulnerability like a bear picks salmon out of a river, but it makes larger scale cyberattacks impractical. The code is a firehose, constantly in flux.
This doesn't make everything secure. It turns the universe into what we see in Star Trek, where everything is hackable all the time by everyone just by trying hard enough, or, in the real world, spending enough. A lot of the utility of hacking systems can be obtained in running off with the data so the inability to use a break tomorrow is not necessarily a problem, especially if I just ask my magic code fairy for another break tomorrow and the remote system has, to a first approximation, learned nothing from the previous breach as it busily churns away.
This is not the way. In fact for secure software I'd happily testify in court as an expert witness that this would be criminal negligence.
As for how closely it matches what Anthropic is doing, no comment. I'd need a lot of time to look at that and decide. That they can write code easily I do not question, whether they have 100GB of problems or 2GB of problem and 98GB of garbage that is slowing down even their AIs is not something we can get from a happy PR puff piece. I'm just speaking to the idea of letting code mutate wildly and constantly.
I know you are joking, but the answer is the same. Because the agents need it, not because any human need it. The Agents quite actively use the history. It's more effective than any comments or separate documentation might be for the agents working on the code.
Do you think each and every one of those are going to be "good" commits, commits that still "compile" or function. Possible, but unlikely given the insane velocity.
That is, the deterministic part is the platform that is being built and that non-deterministic agents will use and build it, but that layer is a key part of how they do work and will continue to be.
All the others seem necessary. Maybe you don't need branches at all, of course we do! Things have to have names.
There could be some fun cryptographic ways you could do it directly inside of git.
If your code changes are all over the place, touching half the repo and stepping on everyone's toes, your problem is not the version control system. Your problem is your changes, and the mess you are leaving in your wake.
In fact most of the architecture work I’m building, assumes that there will not be a human reading code, they read (effectively) UML and data diagrams
so most of the interesting parts I’m working on now are how do you make code blazingly fast, formally verified and memory safe if you remove the constraint of human interpretability of machine instruction
I believe you're mostly thinking of version history though. There’s plenty of other reasons to have source control we still need, like guarding against racing writes or continuous integration to ensure correctness. Even some industries require version history for compliance reasons.
I do love how AI is challenging us to rethink our tools though rather than blindly reaching to git for every project.
They can just analyze PRs and diffs without loading the entire 10,000 line file.
Also, anti-AI folks really have a fundamental misunderstanding of how everything works in the LLM world and is probably why all their predictions are wrong
The big baddies of build have worked this way for a long time already. Git surfaces a single Merkle tree id for the whole repo on each commit, but build tools like bazel/blaze, buck, pants, et al look at hashes of subtrees and use cached artifacts for the parts of the tree that didn’t change.
Your CI no longer becomes a binary signal on the code and more of a continuous signal on how much your “this works” pointer has caught up with your “this is the latest code” pointer.
Pulling that off successfully at scale would be pretty incredible to watch.
We built all this tooling because it's useful. If you replace the human with an agent the tooling doesn't stop being useful. If anything it becomes more useful, since AI Agents excell at reading tool output but struggle with maintaining long-term state
The whole thing, including ingestion and review, is not meant to be used by humans.
This is the shared memory / history of a horde of coding agents.
At 00:00:00 utc you take a snapshot of the code deployed to all your machines (if different code was deployed in different machines then it is part of this capture).
That is the unambiguous code state of your production system.
There is no conflict in a prod state by definition. If there was it cannot be prod.
There's lots to improve in how we handle code, and losing the agreement on what the codebase even is sounds like an awful regression.
From the post (especially the end) it seems that they are essentially letting software be fluid.
So "no direct human supervision" seems like a misunderstanding. Article even bigs up the purpose of engineers with a hilarious (to me at least) line about auto adversial reviews resulting in over engineered designs.
I'm not saying that Anthropic here close their eyes and tell Claude to ship and that in the future there will be no code review at all, but that if we have 30K merges/day, human review seems to need to be done at some higher level, likely AI assisted like it seems to be done with their Antique code review tool mentioned in the post.