enraged_camel

Specifically, it's because the verification loops are much longer in applied sciences. In biology you need to wait for wet lab experiments, clinical trials, regulatory steps, etc. before you can call something "solved".

Since sequential steps are slow, I think we're going to see massive increases in parallelization. I'm not sure how the regulatory frameworks will keep up honestly.

>> If I have a function add_nums that takes 2 numbers and adds them, there are like infinite inputs that can be tested.

That's what formal verification is for: you generate math statements from the code, and then prove them.

Testing is for covering the most common scenarios and likely edge cases. 100% coverage means every line of code in the app is executed when your test suite runs. It is useful as a bare minimum, but by itself does not guarantee that there are no bugs.

...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.

People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.

>> You people drunk on LLM kool aid forget that some of us are old enough to remember those years. I do.

>> It didn't require hundreds of billions of capex. It required someone to set up shop with a PC, a serial card and a couple modems.

This is a bad faith comparison. Those small business owners who gained access to the internet benefited from decades of research by military and research institutions, as well as massive amounts of infrastructure spending by telcos. That spending really started to pick up near the end of 1995. Between 1996 and 2001, North America collectively spent $1 trillion on the Internet buildout.

>> How do you determine which one's ahead in what?

We are talking about discoveries the labs have publicly showcased.

>> They're trying to catch up to OpenAI which has been doing these a lot

Sorry, what? OpenAI is "ahead" in math. Anthropic is "ahead" in life sciences, and even have their own wet lab.

I think it goes beyond individual functions. AI is capable of building complex features in a coherent manner. But it has trouble realizing those features exist as part of a larger system and need to fit upstream and downstream workflows. IMO that's where human engineers come in.

>> AI rarely seems to write better code than humans.

This assertion is at the crux of a lot of these debates. Fundamentally it depends on what is meant by "better code", does it not?

Code that is bug free? Frontier models are inarguably better at that, because they can think of far more edge cases and test each one much faster than you can. You can argue that its code and surrounding tests are frequently overkill, but that's a dial you can tweak until you're happy.

Code that is performant? Frontier models are inarguably better at that too. They can create a benchmark harness and run it over and over to optimize the code they wrote while you're still poring over the docs on which method of iterating over a list is faster.

Code that is secure? Frontier models are also better at that, for the above two reasons: they can create complex security vulnerability plans and run swarms of agents to audit the code and attack the system that runs that code until it's like a god damn fortress.

Code that is easily understood by humans and fits coherently with the rest? I think humans are better at this still, because humans are definitionally the best at understanding what is easily understood by humans. I posit that this definition of "better code" is quickly going out of fashion though.

I don't care either way, but I do wonder how those stats look if you focus on powerful women, rather than all women.

>> In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.

This happens if the model does not have good context regarding what is being built, why and for whom.

For one of my side projects, my CLAUDE.md has a line that says "the users of this product will be non-technical people who find technology scary." And you know what? Claude always steers me away from complexity when working on that project.

Sorry, I'm not following. It's badly engineered because it changes frequently, or because it does not control spacecraft?

Well, the goal is not to impress HN user bigstrat2003. The goal is to have a government and country that continue to function. A fiscal deficit by itself is not a bad thing, and I suspect you would have an extraordinarily difficult time coming up with coherent arguments to the contrary.

I really like the the new ability for agents to start and manage other top-level agents. Makes it super easy to have one agent as the orchestrator for a feature and have it launch new sessions, answer questions from those agents, review their work and build merge trains. This was possible before, but required a fair amount of manual work. Now the experience feels first-class and it's really nice.

Claude Code is not in-flight software though.

Software is very different than other engineering disciplines. It is meant to be malleable. I think you need a better analogy.

The UI has improved substantially over the past six months. New capabilities are prominently featured and less-used ones are de-emphasized or removed. Every product goes through this type of evolution. The difference with Claude Code is that at the speed they iterate, it happens a lot faster and is therefore more noticeable.

>> My best guess is that the uptick in code changes corresponds to the availability of Mythos/Fable-class models inside of Anthropic.

Yeah. OpenAI basically said the same thing with regards to having Astra available to their team internally over the summer.

How does this one instance refute the argument you quoted?

>> I've been mostly using GLM-5.3 on low reasoning, with great results.

Bro come on.

They pushed back the IPO because their finances are a mess and they don't have any models that can give them a decisive edge. Astra was touted as AGI and got crushed by Opus 5.5 two weeks later. That's not the type of thing that fills investors with confidence.

Fable 5.5 is around the corner, and 'Bel' will be out by end of year as well. We don't know how good they are, but if the improvements from Opus 5 to 5.5 are any indication, I think Meta will be left in the dust again pretty soon.

>> Here’s a third one. In some of the code reviews I’ve encountered that AI gives a lot of feedback, it’s just providing noise.

This is completely normal if you haven't written your own custom skill with instructions on what types of issues the AI should emphasize, which ones would be considered nits and which ones aren't a problem at all.

In our repos we use a classification system: blocker, should-fix and nit. Each one has specific definitions, criteria and examples encoded in the skill file. When the time comes to review a PR, agents invoke the skill, and frankly do a stellar job. A human then reads each finding, asks the AI follow-up questions and makes the final decision in terms of whether the finding goes in the PR review.

The reason I know this works is that we have one guy on the team who does not use this skill, and blindly throws his agents at PRs. And the results are exactly as you describe.

>> He catches stuff in code review that I didn’t catch, and claude/copilot didn’t catch.

This, I find completely unbelievable. Because we had (emphasis on had) seasoned engineers on the team who similarly eschewed AI tools and insisted on doing everything by hand in the manner you describe, well after the rest of the team adopted AI. Yet when it came to code reviews, even in parts of the codebase they were familiar with, the bugs they found often came down to nits, bike-shedding and opinion-based feedback (that they usually tried to frame as objective fact). They would also disagree with almost every AI finding, arguing that it was an unrealistic scenario or an edge case not worth worrying about.

Fundamentally, I don't think humans are going to be capable of providing high quality feedback on PRs authored by AI agents unless those PRs are fairly small in lines of code and volume. It's just way too much information and context for one person to keep in their head. I read a statistic that said the average lines of code a senior engineer can read and provide good feedback on is about 400 per hour, and that number goes down the more time they spend doing code reviews. So, to anyone who insists on trying to keep up with AI, I say: good luck.

In law, intent matters. You should look up the difference between murder and manslaughter for example. Each has multiple degrees as well.

Got it. Thanks for your input I guess.

>> that just sounds like openai/anthropic cope/propaganda, based on absolutely nothing objective lol

Nah. There are benchmarks. They are free to look at. And they paint a very clear picture.

Not really. We use Haiku 4.5 to turn users' natural language queries and requests into fairly complex structured specs for interior design and construction. It has near perfect accuracy.

>> Not really, you have to fiddle with generating api keys and setting environment variables.

That's 5-15 minutes of work at most. Not exactly the type of lock-in the parent is implying.

What does "tangled in their api" mean? Switching is pretty easy.