Hacker News Reader

Hacker News

Comment by usrme | Hacker News Reader

Parent Full thread

usrme

By my count, this OpenAI post is the fifth major article on the topic of scaling, with the previous four being:

- Cursor: https://cursor.com/blog/git-at-any-scale

- GitHub: https://github.blog/engineering/architecture-optimization/bu...

- Anthropic: https://www.anthropic.com/research/redesigning-source-contro...

- GitHub: https://github.blog/engineering/architecture-optimization/th...

Maybe also tangentially East River Source Control's too: https://ersc.io/blog/what-comes-after-git.

I'm wondering how much cross-pollination of ideas and subsequent implementation details there are between people working at these companies. It does seem like all of this could potentially converge.

View on HN

Aren’t the concepts mostly already in place for a decade or two at Meta and Google?

Which would support your argument, but I don’t think this is particularly new.

At least for Anthropic, their article states they've vastly overgrown Meta's and Google's pace.

I think their article reads a bit differently, they haven't optimised git, they've ditched it in favour of a new ~~CVS~~ VCS.

> At least for Anthropic, their article states they've vastly overgrown Meta's and Google's pace.

I don't think so, their article says they were seeing 30,000 commits a day in September, when I was at Meta six years ago they were easily doing that.

Size of the commit matters just as much as the total number so there's a missing half not being presented.

I would disagree. The size doesn't matter that much. The main metric is just number of commits, because that is what drives the flow. Not saying size doesn't have an effect, just saying number of commits is the primary signal.

Average number of files changed per commit is going to have a huge impact on performance since that’s a huge part of what the “git push” handshake is trying to figure out regarding what to upload.

Not even close. The rate of code development nowadays far exceeds what Meta _and_ Google combined could output during the early-to-mid 2010s. I wouldn't be surprised if OpenAI is committing over a billion loc a day.

> fifth major article

Seventh, at least, all the major code forges are on it: https://about.gitlab.com/blog/scaling-git-for-the-agentic-so...

Entire also seems focused on this: https://entire.com/news/entire-launches-distributed-git-netw...

Ideas are getting cheaper with the day, so convergence is less likely every day too.

Cheaper and worse!

Don't forget that Claude just got 3x faster by... using gzip!

What is this story? Do you have any references on what happened? What did Claude code start to use gzip for?

https://x.com/claudedevs/status/2111209208012210409

Supposedly they started gzipping the conversations.

I'm amazed that they're not ashamed that they weren't compressing their requests all this time.

Their Claude models had not suggested that, so far

Aren’t the packets already compressed?

I can only assume the whole announcement is about enabling http compression.

This is not my wheelhouse so i could be mistaken, but i thought all the traffic was compressed at a lower layer and has been since the 90s

At least they weren’t using brotli.

Wait till they find out about zstd...

On one hand, we speak of code its authors never read, never share, and never accept patches. So the entire point of git is moot.

On the other hand, some lower-level tools have to be simple, reliable and efficient. Here, sharing is critical. Even the authors of the post shared some incremental improvements to git. Here, the scalability story is moot.

I think git is becoming a protocol. One product can no longer cover these cases.

  "the entire point of git is moot"

agents benefit from the same abstractions as humans do and they also benefit from a version system where the history can be tracked back where merch conflicts become obvious and so on

What got us here won't get us there perhaps? Can you tell why open source started to exist in first place? Wasn't that a way to crowdsource manual labour, and this reason is not valid anymore, as cost of this spiecif labour is near zero?

What if I told you that I could add ten million pages to the US tax code right now at zero cost. Would you be excited?

Depends on the problem those million pages solved. If it meant the government sent me a prepopulated tax form that I could just sign if I agreed with the numbers and be done - yes, I would be extremely excited.

... and if you don't agree with the numbers?

Then you can do the whole file process. For most people the government gets all the data. The filing process is purposely keeping tax companies in business today.

The point of my response is that the analogy was broken without saying what adding those pages could provide.

Right, but now there are a million extra pages in the filing process. Many more companies would be kept in business to deal with the extra garbage!

The analogy isn't broken. Your objection didn't really make sense, because adding pages to the tax code makes the tax code strictly more complicated. Your response just ignored the case where that complexity actually matters.

Tax law is a 100% human slop

I... Unfortunately agree.

I am watching OpenAI's output, and while a lot of it is absurdly impressive, I can not even begin to imagine what kind of activity of theirs justifies a through-output of 5 petabytes of -GIT COMMITS-. That's not their data sets, that's not their models, that's not their math library. That's their code.

They're rengineering Git for scale, but like, why is there a demand for scale at all? If i saw an employee flipping the same number on an excel file over and over, I'd fire them, and yet they're perfectly happy running agents that do that and setting up meetings and writing blogposts about how much they improved their spreadsheet number randomization factory.

They have an unlimited token budget.

Reply on news.ycombinator.com