Hacker News Reader

Comment by trnvll | Hacker News Reader

> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.

I feel like the advantage of having these models in house can't be understated. Companies with access to inferior models, trying to compete, will be left in the dust when it comes to general development.

View on HN

The question of what to build rather than how is still central to development. If the article does point to success it's not as much a technical one but of product. That said I'd take these articles with a lot of grains of salt, knowing first-person how corporations spin their successes, including internally.

That's been my thinking lately. The SOTA models are good enough that they "solve" coding but still need directing. The product is still important. Products are human too, and need human attention (still).

Maybe that'll change in the future, but while AI can make awesome images without a good "director" they're pretty cookie-cutter and bland.

Yes, I think we'll find that technical limitations were never the bottleneck. I work for a large corporation and that has been the case here so far. Yes, a lot of things are faster, but issues with product vision, roadmap and cross-team friction aren't gone. They are actually maybe even harder to get right.

Yep, agreed. Like seemingly is the general opinion, taste and direction I think still matter a lot. Efficiency in it of itself is kind of ”solved”, but effectiveness, doing the right things, is not.

Worth mentioning that it’s much easier to point an LLM at something that already exists and optimize it than to create something from scratch.

I’m not saying Claude wouldn’t eventually have done a good job with a from-scratch implementation, but it would have required a lot more hand-holding.

This is quite insightful: even the best of software engineers (presumably) already are unable to even loosely keep up with these models.

Just a reminder to simply embrace their superior capabilities instead of feeling threatened.

Or they suffer from skill atrophy by overreliance on LLMs.

Others can confirm, but I suspect a lot of inefficiencies in large company programming projects are due to multiple people working on the same code - where you ultimately have to let both above and below average programmers work, and fixing bugs or improving existing implementations is more difficult because of company politics or interactions with others.

> even the best of software engineers are unable to keep up with these models.

It’s more likely they used an earlier version of Claude to write v1, then a later version of Claude rewrote it.

Unless Anthropic engineers were writing code by hand prior to Mythos

There actually was lot more hand-writing of the code back then! Or, at least, only moderately assisted by Claude with a lot of human-still-typing-things-in-an-editor. The ability to delegate larger chunks of code to Claude had a noticeable, step function increase with Mythos Preview.

I don't think anything points to Anthropic having the best software engineers, at least their products surely don't!

Was it a first version in TypeScript then rewrite to rust situation?

Software is not sold, used, or replaced on speed and memory efficiency. Case in point, most software today (including written by LLMs).

Absolutes matter. If 1x is good enough, making it 20x literally does not matter.

This is true for consumer software. I would imagine for developer tooling and especially internal tools these metrics are rather different?

A 10 minute CI pipeline is great and fine, but everyone absolutely cares if you can bring it down to 30 seconds. This software, by its very nature, is measured by speed.

I used to think this as well, but until it turns out to be a problem for the business worth solving (eg impacting $ machine), it will not get solved. There has to be a why now, otherwise plenty of "problems" get shoved under rug.

Internal tools, IME, are almost always short budget. unless it is in path of revenue, then there is $ and performance matters a great deal *if* more performance means more $.

Does it really matter since It's one-time import pipeline.

Ripe for optimization against maintainability, of course, but yeah it would seem to matter if that generalizes.

We ran the repo ingestion in a loop for several months while we were split between Scion and git to keep things in sync, so this perf win was actually impactful for us!

Are you already operating with multiple levels of orchestrators and agents? i.e. one human talks to one or more orchestrators who then talks to orchestrators who handles teams of agents?

Yes, this is a common pattern. There is still an opportunity to improve how these agents write code without needlessly conflicting between themselves and we are exploring some ideas.

Reply on news.ycombinator.com