Comment by mirmor23 | Hacker News Reader

Comment by mirmor23 | Hacker News Reader

> As an experiment, one Anthropic engineer gave Fable 5.2 the task of designing a prototype operating system. Starting from an empty folder, the model coordinated over 1,000 subagents, working around the clock for eight days writing, testing, and reviewing code. It produced a working operating system that boots on a laptop, with its own core that runs the machine, drivers for the disk, keyboard, and network, permanent file storage, windows, a login screen, an installer, and built-in Claude functionality (see the video below).

As impressive as this sounds, whether it is tech teams or agent farms, the progress slows down considerably as the codebase grows. The real pain was seldom about shipping v1, but was more about how the v8 scales and how easy it is to get there, how robust the system remains between versions, and how constraints are held.

A huge part of the challenge is to pick/choose the design and implementation strategy that can only come from experience. That is one of the primary reasons why it's an absurd advice to me to let the llm do the goalseek and freestyle. This may be a no big deal on microservice or any managed runtime software that is an update away, but on an embedded or a high availability system where the field upgrade and hardware replacement costs could bankrupt a business.

if llm designed a circuit, will you bankroll it?

There's been some discussion around Shopify (and others) move from React Native to purely native and that leading some people to conclude that LLMs have effectively killed RN. Which has always seemed silly to me for 2 reasons: First, Shopify almost certainly has on staff (or access to) actual native devs to steer that ship that a small/medium-sized company may not have. But more importantly, it's not the "Translate this RN app into native" that's the trouble. It's the hundreds and thousands of additional iterations you'll have to do on top to maintain and build out that product.

It's hard keeping separate code bases with separate languages in tandem, and while LLMs definitely aid in that area, I can't imagine it's a free lunch.

They also seem to be ignoring the realities of modern app store reviews. Apple and Google are only getting slower as vibe-coded apps flood their marketplaces with submissions. Maybe Shopify has a special relationship that gets them faster reviews, I don't know. But if not, the ability to put out updates, rollbacks, and hotfixes without being gatekept is a critical feature. Especially with today's security landscape. React Native now seems more compelling than ever before.

From what I heard, updates for many established developers, even individuals, go through within a few hours.

I'd also love the ability to take back control over my rollouts, but with Apple's recent crackdown on remote controlled feature flags in response to fraudulent nudification apps, I'd be very careful about this.

Rumors say Apple now uses heuristics to detect feature flags in the binary. While that's not directly relevant to remotely delivered JS, I fear that bypassing App Review in this way could become a quick ticket to account termination.

It’s probably only a year or two out that Apple will inform us that “for security reasons,” they’ll only accept App Store submissions in the form of an XCode project, i.e. as source only. (What are developers going to do? NOT be on the iPhone?)

I'd have original Slopus 5 rename my variables and replace all comments with inline 'docs' for my existing 500k LOC code base.

Good luck with parsing that one, Apple.

I'm joking, but in seriousness, I'm not sure they would gain much from that which they couldn't already analyze from the binary.

Yeah, I have an app with ~150 users, and Play Store updates slowed down to about a week for a moment, but lately they have been going out within the hour.

I've heard those reports, but it's absolutely insane. Having feature flags is table stakes on mobile. And being able to put out security or bugfix updates rapidly is the killer feature of React Native. The current ToS say you need store review for feature changes, and that's fine. But OTA updates and feature flag changes for bugfixes, phased rollouts, A/B testing, and security patches are critical.

Just last week we saw a critical CVE in BetterAuth integrations for mobile app sessions. Being able to put out a patch without being gatekept on review is just so important in a world where crazy security vulnerabilities are being disclosed daily.

What makes you think you can circumvent the App Store to update React Native apps?

How is RN speeding up your app store review time?

You can set up your app to do over-the-air updates. This can be as simple as shipping an updated JS bundle or delivering just patch diffs for it to apply. So long as your code changes are JS-only and don't involve any native code, you can push an update.

People are already used to being able to do things like phased rollouts, automatic rollbacks, and hotfix deployments for webapps and API services. OTA updates let you do something similar for mobile.

It's against the ToS from the stores to do this for notable feature changes that impact what the user sees, but for things like fixing crashes, bugs, or applying security updates, it's a game-changer.

We moved one of our apps away from RN to SwiftUI and Android (Jetpack Compose). The binaries are way smaller, and the apps work beautifully.

So I don't see any reason to stick to RN, when we can just use any OS native APIs to our heart's content and our apps automatically feel more native.

> So I don't see any reason to stick to RN

For the exact same reasons you chose in it the first place: experience with React to build a single codebase for different platforms.

If AI allows you to build your native apps and keep 2 different, growing code bases in parity without needing native dev experience, then use AI. But my point of the original comment is that those decisions are different at smaller companies than Shopify. There is no free lunch.

You're still stuck in this loop of humans reviewing and maintaining code. It's over. They will go native and abstractions like RN will die.

This is it. Most people (including myself occasionally) are having trouble letting go of the old world. It's not about writing code that's easy for you, the engineer, to maintain anymore. Our context window is small. Things need to be easy to read and well organized for us to work with. LLMs will write the code in a way that's efficient for them. If it's easy for an agent to jump in and make a change without doing a ton of investigation first, it's maintainable code.

Abstractions are for humans.

> Abstractions are for humans.

I disagree with this. The mistake rate goes up rapidly as context grows. Current frontier models get noticeably more stupid at context sizes of only 100k tokens or so, and inference becomes slower as the context grows, leading to more rework and more token usage.

I foresee a trend of having _smaller_ code units, more modularity, more abstraction, because agent work on smaller units leads to smaller contexts and a better chance of the agent getting the change right first time. Or a higher likelihood of a lower skill, lower cost model getting the change right.

I agree completely. That doesn't require abstractions though. All that can be done with native code. In fact, the abstractions often grow the context window if you need to debug any mildly complex issue.

> Current frontier models get noticeably more stupid at context sizes of only 100k tokens or so

It may be domain/task/prompting dependent etc, but my experience is different. I haven't subjectively felt any degradation up to like 500-600k tokens with current Opus/Fable models. After which I usually do a manual hand-off to a fresh session anyway. I don't use context compaction though.

> Our context window is small.

It's tiny - 7 items or so, if you are talking short term memory, especially when you compare it to an LLM's context window.

But humans transfer important items to long term memory, which is huge. We have over 100 trillion synapses, compared to an LLMs 10 trillion or so weights. LLMs can't reprogram their weights.

Which means after a human has worked on a system for a few years he will know far more about it than today's LLMs can ever know, because they won't be trained on bespoke systems.

But almost certainly not more then a AI trained on the system could know - and probably a very small AI at that. That was demonstrated by AlphaGoZero. It is tiny by today's standards, but knew more about playing Go than any human.

It's seem likely that today's cloud LLMs will, like AphaGoZero, know more about writing good programs that any human. But they will never be able to build a big system as well as a human.

I dunno what happens tomorrow. Do we train in-house AI's to maintain our in-house computer systems, run the company, or even the country? Maybe that is the end game. We would probably start with the country, because it's easier to justify the huge expense. But I suspect if we do, they will be different from today's LLMs. You will be able to train them incrementally, from a single example. Maybe they will call the training dreaming.

That's called reality, anyone that throws their product to LLMs to design, code and review will end up with a pile of slop garbage pretty quickly

While this unattended example is impressive, this is why you need to guide things more closely and make sure the design is itself scalable and maintainable. Otherwise you can get wad-o-slop.

Same thing happens in many cases with junior or mid-tier programmers slung at a project without supervision.

> While this unattended example is impressive, this is why you need to guide things more closely and make sure the design is itself scalable and maintainable. Otherwise you can get wad-o-slop.

Indeed, I'm curious how much better/worse one human with "one agent" could do that same functionality in a OS from scratch. Last time I did this exercise, I took Cursor's shitty attempt from "millions of lines of source code" to about ~20K (https://emsh.cat/en/one-human-one-agent-one-browser/), and seemingly one of my conclusions still hold true ~1 year later:

> One human using one agent seems far more effective than one human using thousands of agents

I thought the recommendation was to not plan for scalability but rewrite every time your user base grows tenfold. I'm only half-joking here :-)

This makes sense in a way; why design for potential future growth in an environment of fast-paced technological and business development, bringing greater risk?

It seems the problem is the solution, and vice versa.

We had DRY for a long time. Now it's sloppy wet

Data correctness. You can purge and re-prompt your codebase any time. You can't your data.

This is something I've been thinking about more and more. Since Opus 5.5 came out, I feel we've crested the hill and the answer for "who's going to maintain this slop going forward" is obvious - the LLMs will. Whether or not its slop, is extensible, maintainable, or forward looking almost doesn't matter anymore. Does it solve the problem today? If so ship it, and use the agents to solve the future problems. I am not quite sure this is a good idea, but assuming LLM progress doesn't stop means its probably the bet I would take.

I watch a YouTuber who is homesteading and builds everything. Often times I see him doing things in ways I know will break or fail eventually, but it doesn't matter because he is there and will fix it when it does. You could argue this is a waste of energy, and for a human it might be. For an LLM however I do not think it is.

Why would it not be a waste of energy for an LLM? It must still be a waste, but perhaps the cost of the waste is so low that it’s much easier to justify

Yes, the cost of fixing it when the actual problem arises is IMO lower than the cost of spending energy and human time up front to make it maintainable and extensible. I know it sounds crazy when I write it out like that, but this is something I've been thinking a lot about recently. Agentic coding is a paradigm shift and we have to rethink a lot about what use to make sense.

I know it’s kind of a hot take and it might also be hazardous to my career if I’m right, but I kind of agree. If you create a code base that has some significant flaws, and a bug or change request arises later, it is definitely possible that “Super Fable” or GPT-7 Multiverse or whatever can, for $500 of tokens or whatever, perform a refactor of all the related modules, and create and thoroughly test all the necessary data migrations, as a prerequisite to fixing the bug.

I also think that if we’re being completely fair here, we have to acknowledge that most codebases that are more than a couple of years old are giant piles of tech debt already even with 100% human code. At least in all of my jobs, none of which were FAANG, nor run by extremely elite engineers. YMMV.

I (slightly joking) think this is amenable to zero-one-many. The first time your user base grows tenfold, rewrite. The second time, you need to rewrite again to plan for arbitrary scaling!

This is just correct engineering practice.

"Oh no! We're making so much money off the massive number of customers we have that we need to redesign our system!"

Most companies never make it to that stage.

Get there to that point, don't fall victim to second system syndrome and spend 3 years on the rewrite, and you'll be fine.

Reply on news.ycombinator.com