Alternative interpretation of the numbers; there's only a subset of productive work that LLM's are 'capable' of doing at this point. That is, work that's verifiable by a machine or doesn't require verification. Thus, the cleavage points are along that axis and-- naturally-- most human beings don't have that kind of work.
That's where I'm at on the capabilities. LLMs have demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself. Stuff that's more subjective they can produce good work, but with a significant error rate which makes it hard to let them loose on important client work without checking. Or rather, a number of companies are going through the cycle of trying that, having an expensive error, and reconsidering.
Or sometimes a company can simply pass the error cost onto consumers, who are gradually arming themselves with LLMs in response. We're going to see a productivity-sapping boom in disputes.
They are proving useful for the parts of those things that have that verification, but in neither case is that the whole of the field. In maths, for example, we have seen LLMs find answers to some interesting problems/conjectures, but haven't yet seen them pose any similarly interesting problems/conjectures themselves because the criteria for a "good conjecture" are unclear whereas the criteria for a resolution of an existing conjecture are usually supplied by the conjecture itself.
It was asked to do a specific thing, and then its brain turned off/was reallocated to something else entirely (e.g. CRUD programming or answering celebrity trivia). It doesn't just idly think in its downtime. If we want it to do something else, we need to ask.
Given how... motivated the major LLM manufacturers are for good press? I've zero doubt that they have.
Only now that their job (the elites) has been automated was it really about explain results and theory building all this, despite those actions getting explicitly rejected from all the top journals (sans maybe math journals intended for a undergrad/high-school readership). People who weren't in the club and genuinely enjoy math are having the time of their life proving results. There is just a lot of groupthink and silent majority effects due to said elites running departments.
I use "theory-building" separately to Gowers since what he's talking about is field creation that solves tons of problems at once (Grothendieck, FTC, etc (I'll add AI itself here as a meta theory)) not making connections after the fact for the benefit of the lesser humans' understanding.
I’m sorry, are you seriously referring to mathematicians as elites?
> People who weren't in the club and genuinely enjoy math are having the time of their life proving results
If they couldn’t do so before and they can now, it sounds like they don’t understand what their LLMs are producing anyways. That doesn’t sound very fun to me. If they genuinely enjoyed math, maybe they would want to understand it instead of prompting an LLM to do stuff for them?
We're all over here using agents to code projects that used to take weeks and feeling pretty good about it, and they're using agents to build the next generation of frontier models — an order of magnitude more complex problem...
Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
Depending on the type of output, the value immediately (?) goes to the electricity cost of the LLMs and the cost of materials. So: more “efficient”, but without a big margin, right? So its cheaper for the workers that remain working, other workers are displaced, and capital owners now have a super slim margin business which can be instantly replicated at a place where material cost are lower (?).
Thus, it almost seems like “value add” by humans would in this simplified view be vastly diminished, and all that remains is resource wealth. Pop the materials in on one end and let the AI manufacture the product. Great news for the third world assuming the USA doesn't invade them.
Also, whose the consumer? Or are we going back to a strange medieval time where consumers are not of concern?
I don’t know what to think
We don’t see electricity companies among the biggest companies in the world, except in China where they are a state owned monopoly… maybe a good analogy after all.
All over the world they're either state owned or very tightly controlled, precisely because they would be among the biggest companies in the world and even worse than that, they would be essential. If Google Search goes down for 2 days, meh, you can handle it. If the national grid goes down for 2 days your aggressive neighboring country can invade.
Final vindication of the Georgists. All value flows back as "rent", which in the 21st century includes IP licensing.
> Great news for the third world
Not sure about this in a world where there isn't demand for human labour. Especially in the post-consumer world. Very difficult future to see. Possible that it will deliver enough capability and expertise to drastically improve things; possible that without skilled human review that fails catastrophically.
I'm reminded of the Houthis asking Claude to build them a missile guidance system. We're going to see the Toyota Hilux + .50 cal "technical" combo get some weird targeting system upgrades.
It depends; can you send pics first?
Most peoples jobs do not involve hacking other services in large scale fuzzing/whatnot attacks.
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
It does not seem to me that tech industry or its investors care about consumers even a little bit. Consumer preferences dont matter once you have enough or raw economic power, I guess.
Isn't this a massive alarm bell? That the capability hasn't done much for consumers yet that isn't basically toys? Although I suppose the pitch was always B2B "you can automate away your expensive white collar workers" and it's not clear how widespread that is actually happening.
The businesses pay for tokens and try to deliver their usual enshittified experience, and nothing interesting gets created, because it cannot - enshittification extinguishes all power of AI. Using AI to full extent, in contrast, obviates most need for the business in the first place. So they're all struggling to design an AI powered product, in a world where those very products are already becoming subsumed by AI.
So normies wait for middleware companies to get their shit together or (much more likely) finally die, while people with software experience and eagerness to use AI just don't subscribe, because their favorite LLM handles that work just fine.
I also see no reason for that to not generalize into every field of human endeavor.
You don't see a monument to vertical because its construction is bottlenecked on the time it takes for companies to form around it and start executing. So it's few months out still.
How would you quantify this?
My proxy metric is observing the growing ratio of "AI suggested, I haven't even thought of that" to "AI nonsense I had to correct" (where it didn't turn out later AI was actually right).
Some of the differences were:
- Mythos recommended salvaging damaged labels. I pushed back: data whose structure is broken shouldn’t be trusted
- My simpler more readable test caught a regression that Mythos’s more elaborate test missed
- Mythos allowed invalid values, citing real-world data that I could not reproduce, while I decided to follow the spec
- Mythos kept the old error types. I made every kind of malformed input raise the same error, and accepted a small, documented breaking change for anyone who relied on the old ones
Overall, I much preferred the final version of my PR to the initial draft Mythos provided. However, getting there involved quite a bit of time messing around with the code, as well as some back and forth with Opus (which was valuable).
This is a recent example that matches my overall experience working with current SOTA models (the ones mere mortals like me have access to).
TL;DR: I don’t think AI already has better judgment than (most) senior people. Obviously, that might change with future models. But that’s a hypothetical scenario. It might as well not change anytime soon.
You don't know what good judgement calls the AI made along the way. Or rather you do but not in detail because those are the part you looked at that said, okay, this is good code and I should ignore it. Your judgment is biased because you're looking at the bad parts and not the good parts.
I'll add they also seem to make 'bad' judgement calls around which layer to implement something, often preferring downstream fixes/hacks/workarounds when upstream fixes would be better.
They also duplicate code lots rather than finding and reusing existing.
Both of these patterns lead to degradation and time wasting in complex code bases. Part of the behavioural adaptation I've made in using these tools is heading these off as much as possible at prompt-time
My prompts are often verbose to several paragraphs
It sometimes feels like the model is too lazy to do the actual reasonable fix and implements a _hacky_ fix downstream. And to be honest: I feel that!
For human-maintained code you manage duplication and locality of changes in the codebase because you want concepts to fit inside a human’s working memory. Keeping method bodies short is so they fit in one code window on the screen.
LLMs’ working memory works in very different ways; they see codebases not as windows of code but as blips of grep results. It might be that it’s easier for an LLM to just ensure five pieces of code are all modified in sync with one another than to safely implement a DRY refactoring.
For some greenfield project I'm developing, 99% of the time, fixing architecture and going as far upstream as possible is the right choice.
For the codebase I'm working on professionally, doing the hacky thing is the right choice most of the time, because there's enough people changing it frequently enough, with little coordination, that architecture is fluid, and hacks only survive if they work well enough (otherwise someone will come around and fix it).
That was true pre-LLM, too, and is simply a function of scale. And ironically, it's not laziness but pragmatism - at this scale, you just can't assume code you worked on last month will look today like anything you remember. Code review process + extensive linting and testing and AI-assisted analysis filters out most of the junior-style mistakes and flag ugly hacks, so you get a moment to pause and double-check, but if it's good enough, you merge it in and go do another useful thing.
One thing they are definitely not doing is making decisions based on how easily and quickly they will be able to maintain those decisions in the future, or considering that duplicating code rather than creating abstractions will be easier for them to keep track of because of their "very different working memories."
They're just doing a bad job. It's not some sophisticated work avoidance/efficiency in defiance of feeble human abilities to maintain what they have written. 1) LLMs are not lazy, and 2) if writing maintainable code is part of the remit, they're consistently failing. You can't convince me simultaneously that LLMs can handle everything, and also that they can't figure out how to factor their code in a way that would be a pleasure to review.
It simply doesn't make sense. If they're so good, they can make it easy to read and well-factored. If they can't make it easy to read and well-factored, they ain't so good. Saying that they're so good that the code has to be this bad is a very weird rationalization. One that apparently involves speculating about their internal states in a strange way that sounds a lot like the projection of fantasies.
It has better judgement in math and coding, which have infinitely generatable training data; if you ask it a domain-specific question about something like sales or marketing, its answer is often horrendous.
The suggestion from frontier models was to have our service check for this flag and use that to decide whether to trigger this behavior, even though the flag had nothing to do with this behavior semantically. I'd expect this from a junior engineer. Correct now, but almost surely to cause a bug in the future as the codebase evolves.
These tools are incredible and can achieve things I would never have imagined five years ago. I have yet to see their judgement exceed mine.
No offense, but if you noticed it has better judgment than you, consider that you might not be a "senior people in a tech domain"
Don't we already have this experience and isn't it called "the Google support experience"? And I don't know about you, but I don't like it.
Coding - of course I use LLMs at this point, and as he says in the past year it has gone from “ok nice try robot” to “build me a BI tool”, and being pleased with the outcome.
Outside of coding my major use case is organising the chaos of a complicated life. I currently have it managing (including comms and form filling) the import/export process for the international move of a pet, managing pricing on my Airbnbs (dynamite - massive revenue boost) and giving me a weekly performance report, coordinating and corralling documentation for the humans for said international move, and being an extremely dogged advocate/personal assistant in a commercial dispute, which it has won.
The use cases are honestly exploding. I’m kinda at the point where next year’s multi jurisdictional tax returns are probably going to be “yo machine, file my taxes, let me check your work before I hit submit”. The year after next, without doubt.
And just to underscore the comprehension gap… I’ve been doing this stuff for much of this year now, and two days ago my wife ran to me in a panic saying “your computer has been hacked!”, as she happened to notice one of my agents filing Airbnb documentation with the U.K. government in chrome, and had no idea agents could drive a desktop.
so, scamming your customers?
also, maybe consider branching out from just "do better"[0] and give reasons as to _why_ this might be a neo-feudalism is a good thing. just calling my comment "poor" won't do anything to change my position on this.
[0]: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
They haven’t given you any reason to believe they are colluding illegally. In fact their use of the LLM suggests that they are not. You’re connecting this to known bad behavior of others based on no evidence.
I haven't found anything in parent's post about illegality, only low ethics. These two are often confused which is a pity.
https://www.propublica.org/article/yieldstar-rent-increase-r...
If that’s a scam, sure.
It seems that every time they try to use AI to do some real world task like manage a vending machine, then it falls flat on it's face.
The first time DeepBlue beat Kasparov, or AlphaGo beat Sedol, I think everyone got it - we're got a machine that is superhuman at Chess, or at Go. We didn't need to see it beat 100 other people to get it. Yet people at OpenAI think we are so dumb that we don't get that they have built a DeepBlue for Math, and that they need to show us another 100 examples to hammer that point home.
So, next time, OpenAI or Anthropic want to try to impress us with how smart their models are, forget the math, forget the three.js animations (we get that too), and release a model that can do something more useful and not just "here's DeepBlue beating contestant #100".
So now it's just a question of, do you sit on your advances or release them? Why would sitting on them be the 'right thing'?
AI has made strides in programming, but mainly in web development (which nowadays is the largest market), but there are other more demanding areas like embedded development where they'd find plenty of signal.
The promised "drop-in replacement remote worker" also requires a lot more capability than these models currently have, albeit more qualitative - architecturally based - than quantitative, so out of reach for now.
It seems OpenAI are trying to promote frontier math capability as relevant to the commercial market for AI, which it is not (something like Jev, much less glamorous, is much more relevant), and most businesses not only have no use for anything remotely approaching this level of math ability, but also more generally have no need for expensive frontier models which constitute a small percentage of revenue for the AI companies.
If it's investors rather than customers that OpenAI are trying to impress with math results, then I think they are wasting their time - investors are increasingly worried about AI revenue and the financial viability of the business as currently structured, with OpenAI looking like the least connected to reality.
AI is in some ways still a technology looking for applications, at least applications with a large enough commercial market to justify the valuations these companies are seeking. The answer is not math.
And you seem to be confusing the immediate commercial value of a particular math reuslt with the commercial value of a system capable of producing original math research. Thise are 2 different things, and the latter is worth a lot of money.
>AI is in some ways still a technology looking for applications, at least applications with a large enough commercial market to justify the valuations these companies are seeking.
These companies already have the fundamentals they need to justify current valuations. ~1T is nothing to a company that has over 1.2B+ weekly users (and growing) and is entering the ad space, nevermind everything else. What they are hoping to achieve will give them power akin to nation states. Current Valuations would be nothing compared to that.
I'd agree, although I doubt you'll like my framing: it's a commodity business - a race to the bottom on cost of production and pricing
> commercial value of a system capable of producing original math research
I'm not sure if you're claiming a market for original math research, large enough to be more than a distraction, or perhaps you are imagining AGI will emerge from pursuing the problems that mathematicians have set for themselves?
> 1T is nothing to a company that has over 1.2B+ weekly users
Sounds like they are wishing for a return of the dot com era where companies are valued on eyeballs rather than revenue growth
> and is entering the ad space
Good luck with that. Let's see how much ad revenue OpenAI can take away from Google and Meta. The ad-serving product better be free, since the competition is.
> What they are hoping to achieve will give them power akin to nation states
Sure that's what they are hoping. That customers will be spending trillions on their commodity product, but the question is what is the product? Decision models? Free chatbots? Coding agents competing with free weight models unencumbered by development cost? Does OpenAI envisage a trillion dollar market for Terrance Tao automation?
>Coding agents competing with free weight models unencumbered by development cost?
Prices falling doesn't mean a market dissapears. Compute, Bandwidth and Storage requirements have already undergone reductions orders of magnitudes in the last few years and consumption has only exploded. The vast majority of customers, consumer and enterprise alike do not give a shit about open weight models. I have no idea why people here think they do. If they pull ahead significantly enough then they might attract people by value of the amazing service they're providing (nothing to do with open weights).
If they're just going to be playing catchup then they're not gong to gain any signifcant share no matter how free model dev costs become. This is just not how customers work. They're sticky. They value reliability and familiriality and simply having something that works. If you want to unseat google search then you need something so far better people start talking crazy about it. Creating Bing won't do shit even if it's the default search engine on your default browser (you'll just change it).
>I'm not sure if you're claiming a market for original math research, large enough to be more than a distraction, or perhaps you are imagining AGI will emerge from pursuing the problems that mathematicians have set for themselves?
The economic output of math research is very often something other than mathematics. It's not about the market for math papers.
When you see stuff like this
>> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
Do you imagine the gaining advances in math competence had nothing to do with it ? I'm telling they'd be well justified even if their only customer was themselves.
>Good luck with that. Let's see how much ad revenue OpenAI can take away from Google and Meta. The ad-serving product better be free, since the competition is.
They hit $1B in annualized ad revenue run rate ~200 days after launch. They'll be just fine. That you think they might have any trouble taking away or generating their own ad revenue is quite bizarre. If the metrics are good businesses will pay hand over fist, and it seems to me that OpenAI should have a much more complete profile of their median user than Facebook or Google.
>Coding agents competing with free weight models unencumbered by development cost?
The utility of frintier agets isn't remotely limited to software developers. Whoever cracks marketing for agents towars non software professionals will make a lot of money.
Yes - I would expect that the models' coding competence came from being trained to code, not being trained to do math.
There is very little cross-domain generalization from RLVR training, which is why they need thousands of simulation environments. The models are memorizing how to reason in specific domains/situations.
The general public has heard of this, tried it on free tier AI, and uses it as an example of how dumb AI is.
Anthropics cheapest tier paid AI, on lowest effort, gets this right.
Today I gave a free tier AI a photo of a burning ship posted on Twitter, and asked it to confirm it was an oil tanker. It said no, it was a bulk grain carrier, and gave a specific ID. I pointed out that the bulk grain carrier it had identified was painted black, not red (like most liquid carriers) like the one in the photo (as well as other obvious differences).
That is just one example. The free tier models continue to make very dumb mistakes all the time. Maybe the top secret models, too dangerous for the public, are able to distinguish a black ship from a red one. Who knows.
Before fable, the top tier models didn't pass it. Now even Haiku at low effort does.
Where I'd push back a little: a lot of "non-verifiable" work isn't. It just hasn't been turned into a checklist yet. That conversion is most of my job, and it's mostly sitting with the person who does it by hand today.