"I'm on a war footing": Terry Tao and the crisis in math
nytimes.com
nytimes.com
Capitalism with AI would be exactly the same thing, but zero compassion (unless the AI has been trained for it).
The government does not have an incentive to run anything efficiently as they cannot go bankrupt. Imagine if AI was run like the DMV.
Putting monopoly control of it in the hands of government also makes it extremely likely that whoever is in power will use it to manipulate public opinion + elections as well as encoding their political values into it.
What do you think a Trump-controlled AI would say if I asked about the war in Iran? We’ve already seen examples of companies encoding their beliefs into the system (see Gemini saying nuclear war is preferable to misgendering Caitlyn Jenner), imagine how much worse it would be if politicians were given control instead.
I went to the DMV in Oakland and it was the most efficient DMV I'd ever been to.
Personally I remember waiting for hours in a line wrapped around the building for the chance to take my drivers license test.
Another example: have you ever tried to interact with the IRS? The most nightmarish bureaucratic system I’ve ever had the displeasure of interacting with. And guess what my recourse is for them treating me like garbage? Nothing. A company, if they treat me poorly, I can take my business elsewhere. The IRS? The DMV? Not so much.
Like Chevron, or Verizon, or Microsoft.
Chevron: Exxon, ConocoPhillips, BP. From a customer perspective there are tons of competitors which make your life better ranging from QT, AMPM, to Buc-ees (a favorite which people literally LOVE to visit).
Verizon: Starlink, TMobile, ATT, Comcast, etc
Microsoft: Apple, Google, Cloudflare, Amazon (differentiates itself literally on quality of customer service)
The IRS or DMV? No competitor. Unhappy with their service? Go fuck yourself.
The IRS is treating you bureaucratically and poorly because of lobbying from companies and politicians being bribed by said lobbyists.
My point being that you are beholden to a poorly functioning IRS because of capitalism; where companies have lobbied so hard against tax reform and simple filing that it more or less forces you to use their software.
How is this somehow the fault of “capitalism”? Regardless of how the system is structured, the government has a near-infinite budget, they could theoretically staff the agency with enough people to answer my call in a timely manner. They could also train their staff to treat me with human dignity. They don’t do this, and the reason is that they have no incentive to do so. If I am upset by being treated poorly what recourse do I have? Nothing. In fact if I ignore their system long enough, armed goons will show up with guns and arrest me.
There are many examples of government agencies that have this reputation, not only in the USA but the entire world.
“Show me the incentive and I’ll show you the outcome” -Charlie Munger
Also the only reason we have to pay for Turbotax (etc) is due to lobbying. We had IRS Direct File for two years and it was great before being gutted by the current administration.
I dunno what reputation it has in your area, but my actual experience with it was indeed fast, efficient, and enjoyable.
They had to pass legislation to intentionally fuck the books to make them inefficient to fit their preferred narrative because government could be held accountable for inefficiency in ways private corps cannot.
Oh come on.
Here's another thing to get scared about: AI and other kinds of advanced automation has the potential to completely undermine the attributes that made capitalism work for the vast majority of people. Rehashing dated mid-century anti-Communist propaganda tropes isn't going to address that. You're gonna need some fresh ideas.
Also people who want to seriously recreate the Soviet Union don't practically exist (outside of some irrelevant weirdos), except in the minds of some really indoctrinated libertarians.
And the micromanaging from a distance produces catastrophic results.
I think this has historically been true, but it is worth revisiting now with AI and general tech advancement. Might be more feasible now.
> Socialism CAN not work due to human nature. The sociopaths hijack the centralized assets while exterminating via reallocation all opposition.
You could also argue that capitalism CAN not work due to human nature. Just look at the state of climate change.
The socialism that did exist didn't take environmental preservation seriously and rather tried to silence reports about the damage.
Am I missing the sarcasm here? The difference from the current system would be...?
Becoming a billionaire does not magically make you evil believe it or not.
Or did you mean "human nature" (I'm using huge air quotes there) as you have experienced it in late 20th century / early 21st century United States?
The current arrangement does not compute (or work), and gets us straight to doom, "taking into account the nature of human beings".
Historically socialism and central planning have been inefficient. With recent technological advancements it may be that the purported inefficiencies are no longer there, and I think a democratic, socialist government will be better at having the interest of the people than private for-profit monopolies. A society where everything is automated and you don't have to work could be pretty nice to live in.
What part of it wouldn't work?
Curiously, capitalism will work the moment capitalists understand options markets, but in their case they need to also understand put options, which is to say that capitalism is impossible.
The attempt never materializes because some kindly clerk deliberately buries the letter from the enthusiastic engineer, who clearly does not understand that writing Stalin to tell him you figured out a way to build communism better than him, would be suicidal.
It's a fun counterfactual to think about, given that the vast bureaucracy needed for the command economy was one of its major limiting factors. Anyway, interesting book if you are into economic policy or quirky ways that systems can go haywire.
This is the self-defeating contradiction at the heart of progressivism: it is a cult of good intentions. It doesn't matter if the promise of socialism has already been proved, on a global scale, to be illusory. It's the intentions that matter, and in their minds no amount of failure will ever tarnish good intentions.
I'm convinced that susceptibility to such pre-enlightenment "thinking" is a Great Filter.
Where they working on "slop"?
Scorpio!
He'll sting you with his dreams of power and wealth.
Beware of Scorpio!
His twisted twin obsessions are his plot to rule the world
And his employees' health.
He'll welcome you into his lair
Like the nobleman welcomes his guest
With free dental care and a stock plan that helps you invest!
But beware of his generous pensions
Plus three weeks paid vacation each year
And on Fridays, the lunchroom serves hot dogs and burgers and beer!
He loves German beer!It's a beautiful story, more and more relevant.
I'm firmly in the "programming is fun" camp, I enjoy thinking about the problems and solutions, but if I can get a LLM to deal with boilerplate then that's all the better for me.
1. AI agents aren't just typists. They'll fill in the blanks and implement something that works from extremely vague instructions and probably get it right too, because you and your users are predictable. You could be strict and throw out any changes that weren't explicitly what you asked for, but it becomes tiring and it will shape the way you think anyway. It becomes extremely tempting to increase the boundary of what you consider "boilerplate",
2. You probably have colleagues who don't like programming but please the managers because, of course, everything needs to get done right now, as soon as possible. Maybe one day we'll get organisations where enjoyment and understanding matters, but right now we've got tons of people who got in it for the money, just like to "build stuff" and don't care one iota about understanding (and they're oddly proud of this fact),
3. I'm starting to think the typing was a necessary limitation on output speed. I just don't think anyone can think as fast as LLMs can churn out code. You can easily get addicted to the speed and forgo understanding. Like point 1 this is insidious. If you're not really careful you just won't be able to keep up and before long you too will have a bunch of code you don't understand.
The broken economic system is the lack of perceived individual choice, and the priority of business decisions and business profit over individual satisfaction or autonomy.
If we lived in an economic system where maximising individual enjoyment and autonomy was priority, we could give AI to individuals to do with what they want. No sensible human/border collie would choose to give away the fun sheep herding part of their job if there weren't other forces and incentives at work.
But instead, AI is sold to businesses to profit, rather than given to individuals to do the right thing for everyone.
That's why I mentioned autonomy too. People like working and being valuable and choosing to add value. The issue is the current economic system doesn't consider things valuable if they don't directly generate a lot of money or profit for a business. An individual developer gets a lot of value from having a job they enjoy where they produce useful code that solves problems. AI removes some of that value for the individual developer whilst promising more profit and value for the business.
> Why would anyone choose to work with sewage? Or why would most developers choose to align with rest of the team or with user needs, when programming is most enjoyable?
Why do people choose to work with sewage now? They're not paid a lot. Programming is most enjoyable when it's useful. Writing code which is going to get thrown out because it doesn't align with your team is not valuable or fun, so why you or any other individual developer choose to do that given the choice?
Also, some things simply are not valuable within a system, regardless of the economic structures at play. Individual happiness is one of those things that has no value under any economic system, specifically because it is a moving target and follows Goodhart’s law.
It's not about giving it up, it's about it being taken away. You can't round the sheep up if they never leave the pen.
If something doesn't need doing then it's not work, it's a game. Many people enjoy decorating, but you wouldn't trash a room just to give yourself the excuse to decorate it. A few people might like programming just for the sake of it, but for most it was because we liked doing cool stuff that was hard. It's really important that it was hard.
If things aren't hard then it's just like doing the dishes: it's just time consuming and mildly unpleasant. There's no room for creativity or pride in doing the dishes. Nobody steps back, wipes the sweat from the brow and thinks, wow, that was hard, but I'm really proud of the result. Programming is becoming like doing the dishes.
Kennedy famously said we do things not because they are easy, but because they are hard. We need the challenge. We need it to be hard. If all we're left with is boring work like the dishes, or games, then we'll be like border collies left at home with endless treats and a ball. They can and do get depressed and we will too.
And not just implied: it has been a repeatedly openly made promise, from the start of the industrial age to today, repeated again and again, by scientists, journalists, technologists, futorologists, businessmen, even by sci-fi authors.
But the promise of less work, and more free time, has been there from the beggining.
In the early/mid 20th century they even had big discussions and books about what people would do with all the "leisure" time that was coming.
Practically, technological progress does not implies working less, it implies higher productivity. Which implies more wealth for "someone". Who that someone is depends on political system.
That's on you. Implied doesn't mean "caused".
>Also, what sci-fi authors and futurologists write is utterly irrelevant here. We should distinguish fantasy and reality
A useless distinction, since what sci-fi authors were saying was taken to be a foretelling of things to come, adopted as such by many voices in the wider cultural space.
And of course futurologists were people who made (or attempted to make) informed predictions and even policy suggestions of things to come, not mere "fantasy" peddlers.
>Practically, technological progress does not implies working less, it implies higher productivity
That's your concept of it. Hasn't been historically the concept people (from technologists and journalists to politicians and sociologists) had of it. And even if it matches how things went, it's hardly the only way things could have gone (or should go).
> That's on you. Implied doesn't mean "caused".
It can mean both, actually (and usually someone insinuates something, not something, so I didn't find that this meaning of imply was a fit here).
A whole lot of people. Anyone who got into mathematics for love of solving problems (which I'm betting is most of them, as it's not like the pay is awesome) will be unhappy if they have the joy of solving problems taken away from them. Your argument here is analogous to saying "who doesn't want to have more children in the world without having to have sex to do it?".
This is evidenced by the scant few people who get into math just so they can rediscover group theory or differential calculus on their own with no spoilers.
The argument was about mathematicians, and "anyone who got into mathematics for love of solving problems".
We're not talking about doing it as a hobby.
Or did you think the argment was that they'll be forbidden by force to do math, and come to refute that?
There's a whole industry of people on LinkedIn, preaching that I don't exist, that I'm obselete or worthless or deluded.
You get mad fast at the torrent of lies and "come to jesus" thinking.
Then they help themselves to your software.
good ol' TINA is back in town...
The reason most problems aren't fixed is because you and I rather take our family to Disney Land than feed starving kids. We're don't vote for a political system which taxes us significantly more to try to stop people starving around the world. The problems Tao is worried about already exist for a large part of humanity.
You have two problems here:
1. You're not going to change the human incentive to care far more about their own family and friends and resist the government taking their assets to help the needy they feel no responsibility for.
2. In a world where most people are economically useless people ultimately have no political power. If you're not funding the government or contributing to it's maintenance then government ultimately doesn't need to care about what you want. You could go on strike or tell them you're very unhappy, but so long as the trillion-dollar AI companies are funding them and AI is running the government it makes no difference to them what people think. In fact, from their perspective, all you're doing is consuming resources. AI will likely create a new kind of resource-curse where wealth concentration will cause political corruption.
That's typically not what happens - you either get bloody revolution or people just ignore the rules that aren't giving them a fair shot ( ie turn to crime - ie state disapproved activities ).
This is what has traditionally held back the elite from their worse excesses - they don't want to live in a world where either of those things happen.
ie what's the point of being rich if you are constantly worried about yourself or a member of your family is going to be violently kidnapped etc.
As to your first point - it's a bit like the first - if there is no social safety net, then it you could happen to you - you could get ill, lose your job etc. Even if you believe the system is created by total self-interest - then the past has shown that the equilibrium involves looking after everyone to some extent as:
1. It could be you one day.
2. Having lots of desperate people, even if it isn't you, is in no-ones self interest.
Good luck violently kidnapping the family members of a AI CEO in a world of flock cameras and humanoid robot patrol units.
And good luck withholding your labour when 80% of people can't even find paid employment.
Also, arguably if the unemployed masses are going to try to overthrow those provide them with at least the basics to survive and part take in a revolution, then those involved in the revolution shouldn't be surprised if what little they have is withdrawn. Strategically this sounds a lot of like punching your boss and demanding he pays you more or you'll punch him again. This isn't a reasonable negotiation tactic unless you hold some of the cards. The only reason threatening to quit a job can sometimes work is when it's a real threat of economic harm, absent that, it's just annoying and they'll probably be happy to see you off the payroll.
In fact the rise of things like cheap killer drones or robots makes that truth more real, not less.
> And good luck withholding your labour when 80% of people can't even find paid employment.
So those 80% are either being supported somehow to live a life at a level where they don't resort to violence or crime or they aren't. ie you keep them happy at some level or you fight. Note if you let them die/kill them, then you will likely lose the support of the 20% as they can see which way it's going.
I think this all calls into question your basic premise that "politics is downstream of what we humans want", ie that the state is an emanation of the collective will of the people, which it serves. The facts fit equally well, if not better, an interpretation in which the state is independent of society and seeks to extract resources from it. This is also consistent with the Weberian state conception in which the state starts as the "stationary bandit" that collects taxes from a village to protect it from other bandits.
Your conception of "economically useless people" is also incoherent. Economic science is about how to distribute rare resources to maximize utility. These people are no less able to work and no less able to benefit from utility and are therefore not "useless" from an economic standpoint. I think you would benefit from just using words like "rich & powerful" and "poor & weak" which if I understand correctly is what you actually mean.
It only questions "the government is downstream of what we humans want". politics != government.
What you're saying in this comment sounds _very much_ like politics, at least modern politics. There's been a few studies that suggest that people's position on the political spectrum is very correlated to their capacity for empathy for others (and in particular, strangers). First example I found on google: https://pmc.ncbi.nlm.nih.gov/articles/PMC10281241/ (note: I do not vouch for scientific rigorousness, I'm saying "some studies suggest that" not "it's proven that").
So:
> you and I rather take our family to Disney Land than feed starving kids
Speaking for myself, it's more true than false (although not that clear-cut). However:
> We don't vote for a political system which taxes us significantly more to try to stop people starving around the world
I sure do vote for that, as do a very significant portion of people. It's absolutely what I want, I am aware of my own biases (and other people's) to prioritise myself, and taxes are a good way to force this redistribution of wealth to happen despite these individual biases. That's kind of the whole point of the left, which is a very strong power in many countries (note: I'm not american and the "american left" is very different from other countries').
So "what humans want" is very much a political question: a very-much-reducted observation of the political spectrum is that it seems that right-leaning people want the best for themselves and their immediate social circle, whereas left-leaning people want the best for everyone.
You are obviously right in some sense that they are therefore political "problems", but politics is in the business of giving what people who have political power what they want.
Is extreme poverty in Mozambique a top 10 issue in any wealthy country? No, obviously not. Because politics barely registers this as a problem. If it did it could solve it, but like you I'm more concerned about the cost of eggs and being able to afford to take my family on holiday.
This is what I mean when I say it's not a political problem.
I of course would agree all of the negative social and economic impacts of AI could be solved with political solutions, but I also think most negative social and economic problems could be solved with communism.
Humans are kind and compassionate for those who share some sense of communion with, but struggle with even the most basic level of empathy for the people and animals for which they do not.
I don't trust people to implement communism and I don't trust trillionaire AI CEOs to redistribute their wealth. The people who put forward these ideas in my opinion are promoting a dangerous idea. At a minimum those advocating for these ideas should at least be implementing the empathy they require of their political preferences in their own lives. Most people pushing these ideas probably don't even acknowledge the existence of those begging them for cash in their own city streets, yet for some reason they think trillionaires will take sympathy on their own economically pointless existence.
Not just the pay, but motivation doesn't work on math insight alone. If you look from a distance it's clearly a mix of status, money etc. If what you say was true, many of them would be recluses up in the mountains, but none of them are.
LE. Think about it this way: if someone truly only cares about his ability to do "non-AI-contaminated" mathematics, nothing stops them from ignoring all the AI-related papers and just carrying on on his own. The thing that they are passionate about is not taken away by the field being more rapidly developed. What's being potentially taken away is the impact of their potential contribution.
Except, for instance, Wiles and Perelman? And I’m pretty sure that mathematicians are rather more introverted and solitary on average than most people.
In any case, that just doesn’t follow. Mathematicians realize that collaboration is important in math just as it is in most other fields. Humans are also social animals and many of us enjoy collaboration and social interaction for its own sake. And yes, that means social status is a thing that motivates many humans. But nobody decides to study math in order to gain status! Any status stuff is entirely contextual to and conditional on having already decided to study math.
Humans are built to focus on some objectives and not others at any given time.
Humans naturally prioritize objectives differently. Drives have exclusionary impacts on each other. In both real time and lifetime frames.
Reinforcing experiences further differentiate objective prioritization.
This leads people's behavior to be driven by entirely different net objective directions.
--
This diversity would be inevitable due to genetic diversity. But it is also a survival advantage.
Lots of studies show that populations maintain stable diversity of genes for entirely different life strategies, because operating differently from the majority is an advantage.
--
We don't actually have a "status drive".
We have drives that tend to generate status seeking behavior but can be subsumed by many different status-like pursuits, such as self-status of solving problems or completing single-person video games, being good at a hobby nobody else cares about, pleasing certain people even to the point of harming any recognizable definition of social status, acting to please (raise status with) internally generated or deceased personalities, a need to be contrarian or difficult to the point of low-social status but high internal status, etc.
--
Your assumption that same basic nervous system -> same net drives/behaviors, fails in much simpler species.
--
So no, mathematicians are no more motivation clones than they are genetic clones.
And mathematics is not a field that is attractive to people who give social status high priority, because it impedes social status.
It's interesting to recall Turing's own position: "There would be great opposition [to AI] from the intellectuals who were afraid of being put out of a job. It is probable though that the intellectuals would be mistaken about this. There would be plenty to do, i.e. in trying to keep one’s intelligence up to the standard set by the machines, for it seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits."
On the other hand, Thomas Bernhard's The Loser is a horrifying account of the opposite standpoint---of an essentially second best piano student who cannot stand existing in a world with Glenn Gould.
Mathematics represents the rational skeleton that upholds and stabilizes our vague, formless but far more common belief-based thinking.
Without it, losing it from neglect, we would collapse back into a pre-rational state of fantastical, wishful thinking.
Pretty much exemplified by large parts of US society already, but there is no bottom otherwise keeping society from collapse.
> What would such a world look like?
Major religions, minor religions, cults will have a field day. Or more like a field century or a millennium or twenty, who knows.
Up to a few years ago, I would see or read some science fiction that was medieval times with spaceships (ala Dune) and I would roll my eyes and think “ffs, can’t people move on and invent something more original?”. Now I seriously believe that’s one of the most benign futures waiting for our species[^1]. Not that there aren’t lots of lights in this tunnel, it’s just that its shadows are very black.
They have nothing to fear if that is their motivation. New players get into chess every year despite computer vastly outperform us.
Now that all of the trite and boring intellectual work is automated, I can look forward to having it bottle-fed back to me at night. Now get me a gramme in time...
~ unnamed Gamma BNW elevator technician.
That what it cannot do is to invent new drugs, cannot figure out cheap energy sources, find something better than Standard Model in physics, and anything that would contribute to overall human progress.
The result is that it can replace a lot of jobs, it will efficiently killed any research done and the level of master/PhDs, because majority of those can be done now with the help of AI (we already see this, the burst in new scientific articles - nothing groundbreaking, but publishable).
As a results we will have soon a gap, as only very determined people would go into science, analytical jobs, etc.
In one way, this is OK, we had too much science research that was done only for the sake of producing more "papers", we also had a lot of people in the companies doing "bullshit jobs".
But on the other hand, we might face soon situation in which there will be almost no people in Academia, so who will teach people, in addition, science worked often by doing wrong, unreasonable, crazy things, testing vague ideas and come up with something new and great. This required a lot of people doing science without any good results coming out, a lot of seemingly wasted effort just to hit several great ideas (a bit like Venture Capitals work).
This might also lead to decreasing number of white collar jobs. We can have a new "rust belt", the one for blue collar fall happened because of jobs offshoring, now we are "offshoring" white collar jobs to a few companies who offer AI models.
This all will affect badly society without any huge, visible benefits for it as a whole.
That has not been my experience.
This was a video I saw that covered this idea well: https://www.youtube.com/watch?v=KqiDqJMxg6o
> But closer examination reveals that OpenAI’s proof builds its unnatural force in a totally distinct way. The proof’s similarities to Buckmaster and Alpöge’s progress are superficial...
https://www.scientificamerican.com/article/openais-blockbust...
So I think more time needs to pass before a call can be made.
An interesting thought experiment is this: if we were to go back in time 150 years and tell our ancestors that Newton is just an approximation and give them a 5,000 page book about Special and General Relativity, QM and QFT, would we be robbing them? Initially the whole thing would be incomprehensible, but eventually they would chew through it and make sense of it. I don't really know the answer (and perhaps the analogy is not great since we're quite attached to our physics heroes from the last 150 years).
there is a difference between "knowing" and "knowing about". "knowing" is having it deeply ingrained into one's intuition by working at a solution over years, often without progress in the immediate sense, and often without success in the absolute sense either.
Maybe post-LLMs mathematicians will work on problems/topics that even ChatGPT can't solve? Like, P=NP remains unsolved, we can work on that. Sure, it's possible that it will get solved tomorrow by an AI or somebody else, but that possibility was always there.. Or, to be more practical, one of the claimed OpenAI results is that integer multiplication is sub n×log(n), so assuming it stands, now there's a new question: "What is the actual lower bound for integer multiplication?"
I have no proof, but I believe (i) LLM progress will converge at some point, (ii) that point will not be so far off from human intelligence, since they're trained on human intelligence, so.. not all problems will be solved/solvable by LLMs. But they'll be fast(er), like calculators are faster than us.
This is just what is happening. Techniques discovered by AI have already been used to solve other problems, or to extend the AI-discovered solutions.
I find this argument difficult to understand. As a math student in college with no ambitions (or realistic opportunity or aptitude) to pursue original contributions to the field, everything I learned was by definition already solved. Nevertheless I was able to - required to, in fact - work through each solution in order to develop understanding. And I did in fact learn by this method, as countless other students did before and after me.
So when I hear this argument from mathematicians, all I can hear is that they are frustrated at being out back into the position of being students, where the end result of the hard work of understanding is just the understanding itself without the prize of a new discovery. If they want to find the solution without reading it, they are still very much able to do so, just as their students do every day.
Difficult problem -> Developing new branches of mathematics to solve said problem -> solving problem, or failing to do so, it doesn't matter -> aforementioned new branches of mathematics are used to solve practical, real-world problems that benefit industry and economy.
The point of solving the problem is not the prize. The point of solving the problem is not even solving the problem. It is developing new methods and deepening understanding to drive useful applications.
Knowing that a solution exists is not an impediment to deepening your understanding of a problem.
For example: Knowing that there is a proof of Pythagoras’ theorem is not an impediment to thousands (millions?) of geometry students developing the intuition that comes from drawing the famous diagram where the number of squares visually add up to one another.
What about the existence of an AI-generated proof prevents a mathematician from working through the problem on their own to deepen their understanding, as if the proof didn’t exist in the first place?
That’s what is hard to understand.
I'm baffled that you find the mechanics of human motivation hard to understand, (presumably ;) being a human yourself.
Studies have shown that people that know a certain piece of knowledge is stored somewhere, are less likely to remember said knowledge.
The sense (thrill?) of exploration and discovery can in fact be part of a person's motivation. The popularity of open-world games (often derided as 'walking simulators') are a testament to that. Do you really not see how that could be undermined by knowing that a problem that interests you is already solved?
Basically all natural sciences work that way. Knowing the truth of an apple falling down a table when rolled over the edge is the beginning of discovery, not the end of it.
Mathematical discovery also worked more like theoretical physics 200 years ago. The modern way of formalization in maths might be dead now in some ways, but it's also only 100 years old. There's still many ways to do modern maths and have a lot of fun with it that are not taken away by AI.
it is qualitatively different
> If a set E in R^k has one of the following three properties, then it has the other two:
(a) E is closed and bounded.
(b) E is compact.
(c) Every infinite subset of E has a limit point in E.
Likewise, when given an exercise of the form "Prove P" or even "Prove or disprove P," you can be pretty certain 1. it is doable and 2. it is doable using techniques/theorems discussed in the course/book you're working through. But people are perfectly happy to do this for several years.
But that’s exactly it! The mathematicians are framing this as being bad for human understanding of math, but it’s obviously really about their own motivations for developing that understanding! Which, again, goes back to the obvious conclusion that what they have lost is the extrinsic motivation for being first, not the intrinsic motivation of developing understanding.
That is totally fine! I wouldn’t be motivated to do my job if I wasn’t extrinsically rewarded for doing so. But I am also not feeding anyone a line of BS about how it’s actually some other higher-minded motivation driving my work.
I’m empathetic to the instability of the extrinsic rewards for doing math research as a profession in the current moment. I am not at all empathetic to an attempt to manipulate me into thinking that the real problem is something other than that.
It’s like an entrepreneur complaining that they can’t build a business that would better humanity because of high taxes. That’s fine - but the motivation lacking there is obviously making money, not bettering humanity.
When you're learning/internalizing existing knowledge, you tread a defined path skipping unnecessary stops and diversion and heading instead directly to the destination. The destination, not the journey, is the point.
When you're discovering new knowledge in the form of proving/disproving, there's no defined path to follow, so there can be quite a bit of backtracking and diversions and detours into other areas you weren't planning on going initially and while there might be some well-defined paths to follow, you might have to cut your own path through uncharted desert or jungle.
All those diversions and detours make the journey just as if not more important than the destination because they create new destinations of their own for future travel that would not have been spotted or noticed for a while longer (if ever) otherwise.
So to bring it back to math, transitioning from having to develop proofs for theorems to validating/learning extant proofs completely changes the scope of future discovery. Everyone will tread the exact same path to a proof/theorem and then stand in the same place to look for future destinations and then use AI to build them a path to that new destination and then everyone will follow the same path to the new destination and repeat the process, so the chances of finding something curious along the way plummets. And something curious could be an entirely new field of math, or maybe just a new problem that isn't relevant to the intended destination but gives a view of many new never before seen destinations for future trips.
Demis Hassabis has talked about this a bit in regards to theoretical physics and it's an idea that I think I agree on. Every once in awhile we randomly get a super genius who's brain is seemingly able to conceptualise or visualise beyond the state of the art. But at some point I think we have to accept that we are just animals and inherently limited by our brains (and means of observing the physical world).
The scary part isn't that AI is solving the unsolved problems that will force people to act like students again. The scary part is that we may not be able to understand the work needed to get to the solution at all.
Obviously we're not quite to that level yet but it's entirely possible we do get there. And on the optimistic side ~ maybe that's what we need: better tools.
I see no reason whatsoever to expect that humans have any kind of limit to what we can understand as a species. We've had more unthinkable and incomprehensible discoveries than anyone cares to count. They become understood, normalized, and folded into our scientific canon. We went from inventing airplanes to landing on the moon in a matter of decades. We went from banging rocks together to quantum physics and string theory. I think we'll be alright.
But there's no guarantee that such "low IQ" intermediate steps exist. We do well with many classes of problems because of divide-and-conquer. But not all problems can be solved through divide-and-conquer.
For a rough analogy, see how AI plays chess and intermediate board positions often don't make sense to us. Imagine now an enormous chess-like game played in an enormous chess board where a match lasts longer than a lifetime and none of the intermediate steps make sense to any human.
I remember a recent "100 hundred mathematicians on AI" post recently. Some of them appeared truly excited even if the majority wasn't. Who's right? Who's biased? I don't know.
Right now you're doing the equivalent of telling athletes that you've solved sports, and they need to sit back and stop playing so that we can enjoy progress in goal scoring.
Thanks.
It's just a law of nature man. Don't take it so personally.
If OpenAI had never made a push to get good at maths, we might still have a sizeable group of peopne stuck at that first line. I say that because I still hear in discussions the naive rebuttal "But we'll still need _someone_ to {one-step-up-in-hierarchy}" . Although you have to be curious now how long that sentiment can last.
The good news for those doing abstract maths might be that, now that OpenAI have proven that their AI can do the job in principle, there's probably not much left in it for them to keep going. Mathematicians should know better than anyone the lack of a direct profit motive in doing the work that they do.
(Edited for clarity)
This is a disingenuous framing. I've "built" more in academia than in some jobs I had, and some VC-backed startups are renowned not for building but for their lack of ethics.
"Building" is not the discriminating factor.
Earning money is not curelated with usefullness these days. It is somewhat correlated with lack of ethics.
Grigiri Perelman, who was awarded the Fields Medal, notoriously refused the prise because he stated no other recognition was needed.
There's a reason that so many open source builders and contributors are unemployed people in social-welfare European states. There's a reason that nearly every great work in the English canon was written by someone who was on the dole at the time, from Nietzsche to Harry Potter.
if you think that's the takeaway, you're not listening to any of the people affected by AI
By snooping on work math researchers were doing on their platform and then overtaking them when sufficiently close to completion? isn't that what happened i'm not sure
Finally, the list of hundreds of solved problems after that completely defeated any argument in that direction.
Using LLMs doesn't mean some easy mode where humans don't do difficult work
not saying it's not possible ai will continue to expand to more fields, but i don't think it's a coincidence that both math and programming are easily automatically verifiable. in fact for programming their effectiveness drops substantially once you remove any sort of testing or verification loop, and i imagine it doubly-so with math. try getting an agent to build something without any tests or verification, you'll see it go back to near-2024 levels. half of the work i do with agents is setting up a harness that can take and analyse screenshots, send direct input to the application, and compare results. this is something that should come standard with every harness.
if we can build automatic verification for more fields is a different and perhaps more relevant question.
If that is true, then only because "most people" have no understanding of what differentiates programming and math from "computation". Computers (as the term suggests) have always been extremely good at the latter, and useless at the former. It's only 2–3 years ago that computers, by way of LLMs, have started being good at the former.
If the same agent writes the tests and the code, it will happily weaken an assertion, add a skip, or special-case the fixture to get to green. I keep test files read-only for the implementing agent, or have a separate pass write tests from the spec before any code exists.
Mutation testing is a cheap check on whether your tests constrain anything at all. If you can flip a comparison and the suite still passes, the agent's "all tests pass" means very little.
Property-based tests and differential checks (compare against a reference implementation or the old version) hold up much better than example-based tests the model can pattern-match against.
On the screenshot harness: pixel diffs get noisy fast. What worked better for me was dumping the accessibility tree or DOM state alongside the screenshot and asserting on that, with the image as a secondary signal for the model. Fixed viewport, fonts and animation disabled cut flakiness a lot.
Also, verification only covers what you thought to check. Veracode's 2026 update on LLM-generated code had syntax pass rates above 95%, but only about 55% of tasks produced secure code. "It compiles and the tests pass" was true, and the security property just wasn't in the loop. Math has an edge here: a Lean proof that checks really is a proof of the stated theorem, so the remaining risk sits in formalizing the statement. Programming has no equivalent unless you specify everything.
On your last question, I suspect many fields will get partial verifiers (simulators, unit checks on physical quantities, citation checks, reproducible pipelines) rather than a full oracle. Then the human work moves to deciding what the verifier misses, which is roughly what code review is turning into already.
Now I'm forced to think about it much more carefully. I don't believe that current LLMs can find every possible proof on their own from now on. I also agree that the PR returns have fully diminished. Given how hard it's been in recent years to fund e.g. a poorly paid postdoc, I doubt there's a ton of money lying around for getting the best models to work on mathematics problems. I think there's a chance we're heading into a really good era. I was able to make so much progress back when Google search was a useful research tool...
Over all, I'm not quite sure whether I see these deep cuts occurring less and less with progressing AI. It might just be, that I'm giving them progressively harder tasks as well. But the fact of the matter still is that they aren't there yet and it needs proof they will get there ("there" being able to do competent work that is not strongly limited in their scale by human review and correction). So I'm still not convinced about the point you want to make, even though I see AI disrupting about every part of society, and am hoping for the benevolent super AI to lead us to utopia (and afraid of the cliff of human dictatorship on one side and extinction on the other).
[1] I've started to use these moment to reflect on whether a human in a comparable situation would fare better. Having only a few text inputs and small glimpses into a code base and knowing next to nothing about you or your project personally. Quite a few times I've concluded: No, they would not. But more than 50% they would, and also, these limitations are technological limitations of LLM, so you should probably not excuse them for that.
So the pain points you and I frequently run into, simply don't exist for a well connected individual(Sam Altman).
Considering that this is where the mathematician excels over machine wouldn’t that only offer a new view from such a mathematician that “OpenAI has rapaciously stolen from mathematics that last dignity: of choosing worthy problems”? Perhaps with sufficient effort the community of mathematicians may yet forestall such an effort.
Don't invent things to get mad about, please.
This will be interesting to watch: Where will AI creating and solving problems in a loop lead to.
This leads to the comment I've seen many mathematicians make that humans should refocus on understanding not just proofs and mechanics.
So far, a lot of the hullabaloo has been around solving legacy problems, but more interesting and more important for actually progressing mathematics is the generation of new insight and new problems. Someone had to pose all those problems OAI is solving in the first place, in some cases, ages ago.
The collective letter response to navier strokes, sort of touched on this, but still didn't really emphasize the epistemic value of problem generation explicitly. I find it sort of strange that this has been so underemphasized.
So I'm not sure if it's something we should expect from a machine, just because we don't expect it from mathematics either. Maybe the machines will interrupt that process because we've changed how we go about looking, but perhaps not because old problems are solved.
From the very top of https://www.claymath.org/millennium-problems
In order to celebrate mathematics in the new millennium, The Clay Mathematics Institute of Cambridge, Massachusetts (CMI) established seven Prize Problems. The Prizes were conceived to record some of the most difficult problems with which mathematicians were grappling at the turn of the second millennium; to elevate in the consciousness of the general public the fact that in mathematics, the frontier is still open and abounds in important unsolved problems; to emphasize the importance of working towards a solution of the deepest, most difficult problems; and to recognize achievement in mathematics of historical magnitude.
It then goes into details in each of the chosen problems.
The only thing that will matter is capital, which will translate directly into all the talking and doing needed to make more money.
Interestingly, the dynamics strongly resembles the industrial revolution when from highly skilled manufactures society moved to factories with power machines. And it all came down who owns means of production. Now AI is a new means of production for all white collars jobs and if the industrial revolution taught is the way to go, nothing good will come out of it.
I don't particularly like AI, but if AI replaces developers and other knowledge workers, not everyone is worse off, and I don't mean just capital owners.
would someone who runs faster 100m also somehow drive better?
I use LLMs to great advantage, solving problems in my domain. I no longer make for loops.
my point was that the new future where running faster doesn't matter any more might come, because everyone is driving (and potentially can't run much at all).
See the beauty of analogies is you can make one to support any point.
He described it at 'strip mining', 'rude', and said that many grad students, who are given problems like these as a part of their training, had the rug pulled out from under them (because its customary to let people know you are working on something to prevent this kind of thing).
And your whole analysis is that Tao is worried about his 'identity', status, and 'being a differentiator'.
This is projection on your part. People have values that bear no resemblance to yours and it would behoove you to learn to recognize that.
AI is coming for all of us knowledge workers, nobody is immune.
Disclosure: PhD, early 1990s. In my case, replicating my project would have required a sizable investment in equipment, much of it was purpose-built, among other things.
Interestingly we're a few months away from a data point on what happens to grad school applications.
In math, proofs are "the coin of the realm," and could be devalued to zero. This leaves no answer as to how someone will earn a graduate degree. For instance, what kind of work will be accepted by a thesis committee in five years? The professoriat can only say "we will need to change some things." That's not good enough when the deadline for grad school applications is a few months away.
Strip mining what, platonic reality? Mathematicians don't have a monopoly on mathematical truth. It doesn't exist for their benefit.
It's not about improving understanding, as you just pointed out.
So why? What the purpose? Is it just pure misanthropy?
A small subset of humans who lost a bunch of comparative advantage hate it. Most people who care about math in the abstract probably think it's really cool!
The thing I liked about humanity was its ability to think. Making that valueless is not my idea of progress.
I’ve seen multiple conference submissions that read like English, but take hours of effort to disentangle and figure out what they mean.
There is a limited number of known interesting problems that are worthwhile to engage in. New ones are found and understood, but it is a slow process and it is dependent on understanding and verification, not generation of content.
Saying AI will create new ones, strengthens the strip mining analogy. The idea that AI will find other problems parallels the idea that firms can cut down a forest and plant them elsewhere.
The conversation on HN is driven by non mathematicians, to whom math is a subject, like “nature”. It’s not their core business. The original post by Tao is a better source of information on what he means.
No one has the right to prevent others from working on a math problem. If you're working on a problem the best way to not get scooped is transparency: publish whatever you have as soon as possible.
Tao's behavior is most consistent with any other worker resisting automation for selfish reasons.
> It's time for the social fabric [of math] to be ripped up
This is the attitude of a social arsonist. You can, and should, integrate changes without maximizing destruction for its (or Altman's) own sake.
> an industry being disrupted.
This is the SV euphemism for law-breaking and monopolist practices in a country that has forgotten how to prosecute those with the most power and privilege.
Quite the opposite. The mathematicians are the monopolists who are upset that new-comers are able to do their job better than them, and even worse, they are giving it away to the world for free!
The horror! People giving knowledge to the world for anyone to use freely! Won't someone think of the oligopoly that was coordinating to make sure each of the gatekeepers get their slice of the pie?
Mathematics is collaboratively and publicly produced, it is vetted for free by peers, and results are publicly available. There is no 'market' and no 'price'.
These results, like the invention of LLMs, are often used by private companies for the generation of private profit.
"Open" AI used a private, unreleased, for-profit model to generate those papers.
What, precisely, did the mathematics 'oligopoly' gatekeep you, or anyone, from?
> their slice of the pie
There's very little money in academia, and even then, its a struggle to get any funding at all. What are you even talking about?
Final point, 'knowledge' is socially produced as well. Until the papers are accepted, they are machine artifacts that might be knowledge, but that same system you disdain will be what incorporates and validates those insights.
How wonderful then! You should be happy to know that the AI companies are releasing information for free for all to collaborate on then!
> What, precisely, did the mathematics 'oligopoly' gatekeep
Well what they are trying to do is convince the AI companies to stop releasing all of their findings. Because they are upset that other people are able to do their job better than they are. Its the classic gatekeeper move.
> There's very little money
Oh there is much more than money to silly games like these. Instead, clout and the admiration of their peers, in this tiny insular group for sure, is what most of these people are so desperate to acquire.
Fame (well... relative.... in their subgroup) and admiration of their peers is a much stronger motivation factor for humans than mere money.
> but that same system you disdain
I don't distain anyone who is willing to get with the times. My distain is held out only for those who want to prevent the rest of the world from using the latest tools to achieve real results. If a mathmatician simply doesn't care and isn't wineing about all this AI stuff, I have no issue with them.
There are already several of these "proofs" that have been retracted because they did not make sense.
I would have no problem accepting this whole ordeal if the humans behind each proof took the time to re-read, understand and synthesize the result in a readable paper.
There is ample evidence of how damaging fake information can be, and paper retraction never fully undoes the damage (see the Andrew Wakefield and his antivax "paper" for instance).
So there's that.
This is what Terence Tao's ChatGPT chats look like: https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed.... Can a layperson prompt the LLM like that?
"Garbage in, garbage out" is a polite way of saying the LLM's output is a reflection of the prompter's own intelligence.
At the same time it should be considered that with physical labor workforces lost their income source numerous times. Ironically they are currently being considered a safe haven.
We know it's a filter, and we know it's an idiot parroting Claude.
The last(-ish) chapter does have a section on how AI has robbed the "cognitive elite" of our identity.
I think it's always better to publish preliminary results early (as long as you clearly say so, and they did), and let mathematicians decide which of those are worth focusing on.
No? How much did OpenAI spend on this and how many trials failed? I can’t imagine anyone not working for them can pay the absurd token costs!
Can you elaborate what does that look like?
And that multiply those few people by number of different problems.
And I honestly have no idea what Tao wanted to happen. How could you ensure that the AI "empowers" scientists and doesn't replace them. Intelligence is intelligence. If it's powerful enough, it will replace other intelligence.
Unfortunately, though, it is what you said. And more or less what you
> If one wants to argue something, it's that the Nexus should not have been built.
Or that it is being built in a way that is causing forseeable problems, while making no effort to anticipate and solve or mitigate those problems.
Don't we as tech ppl understand "purpose"? You might not feel purpose in their work, but surely you see how when it comes for your own, you might revisit why it is such an imperative that we give AI all tasks for want of speed and efficiency?
For math, just because we know a thing is "TRUE" doesn't mean the investigation is done. The folks that love math are driven by solving puzzles, and the puzzle is still there, only changed: instead of being "is this true?", the puzzle is now "why is this true?" Building theories, finding nicer proofs, etc: there is still plenty of room for creativity.
But in the meantime, it really is tragic that so many young minds have had their work stripped away from them.
And the economy puts no such large incentives on much of the remainder of economically valuable work being purely human, quite the opposite actually. And instead of regulating the AI industry (which produces the "cheating tools"), they get trillions of investment. So this seems to be going in the opposite direction where it is actively worked against humans staying economically relevant.
The societal rewards of the mathematical community's pursuit of answers to problems is not primarily in the answers themselves. It is in the practical problems that are eventually solved not by knowing the solution to the initial problem but by means of the new branches of mathematics that were invented to solve it.
Are you really unfamiliar with the concept that "the journey is the destination"? Read a book.
OpenAI is Doctor Octopus who just buzzed the climbers. They've proved there is a route, but so far can't explain it in a way that is intelligible leaving the climbers to catch up. Or is the AI making leaps of logic too complex or out of reach for the climbers a la the four-color-theorem[2]?
This observation is so important. There is unintendedly something being taken away from us that cannot be given back. And it's a pure loss, there is nothing returned to us on an experiental level. It already existed before that "somebody else" would solve or discover something. Now this "somebody" is just just an alien nobody can relate to. And that "something" is everything.
Even if AI solves ageing, and climate change and whatever else; as humans we'll have lost something beautiful and precious. There will just be less of this sense of wonder and curiosity.
But we still have no self driving cars. Seems like the automation came by in the opposite direction. Perhaps driving will actually be the final thing automated.
While I agree with your statement overall, as someone who drives everyday in LA alongside Waymos without anyone inside of them I must disagree with this aspect of it.
Edit: nevermind. https://www.cbc.ca/news/business/waymo-toronto-winter-self-d...
Because if it became fashionable for pedestrians to wear large green dresses with red balls, and everybody would wear that, we have no idea what the AI would do.
Something I'm noticing a lot is that automation finds easy footholds in unregulated industries because there's no basis to push back on it. And there's a lot more tolerance for poor quality work in unregulated industries too.
A truck or a car that has a 99% success rate would be the top threat to society and quickly rise to the leading cause of death under 30 if widely deployed.
You have to look at the benefit of something when balancing against a cost.
Wait, aren't human-driven cars already the leading cause of death under 30?
I agree that LLMs have and will continue to increase the number of activities that people can DIY. But I believe many people will still choose to pay for others to do things on their behalf.
This is beside the point of the analogy though.
99% successful web development where the 1% missing is security that and the company leaks its entire repository of user data every few months is not good.
99% successful car driving where 99% of 5-year periods have zero accidents per car is an order of magnitude safer than the average driver. (And gut feeling, I suspect approaching the tail-end of the safest drivers on the road.)
It feels like it's happening every Thursday and nobody cares. They just give people credit watch coupons or what's it called, and move on with their business. Worst case, a few pennies in fines.
But if CVS leaks your purchases on a weekly basis, it's probably going to affect people's willingness to shop there. (Or at least to use non-cash.)
In 2025 it was occasionally driving weird. Today it's a much better driver than me.
That just makes you less prepared when you do have to take over. That's the danger of almost-perfect self driving.
In the case of Tesla, daring to call such a system "Autopilot" is a breathtaking level of irresponsibility. You are responsible for the vehicle you are driving, practically, morally and legally, even if a companies' marketing and the system itself does its very best to divest you from that simple and obvious truth.
Pilots still are required to be able to take over immediately for an autopilot system and are responsible and accountable for all those items you list and more.
Calling it autopilot is precisely what it is, calling it self driving would be false marketing.
We are talking about cars, not planes. Driven by regular people, not pilots.
It's a term chosen specifically to evoke a certain connotation among the majority of people, and you know it.
Autonomous driving is something entirely different in my mind. That would be a car good enough to work without driver controls or a remote operator. Nothing on the market meets this definition today.
Most people expect other things from something marketed as flying, but specialists know that this is exactly how most things fly. And lawyers know to read the fine print.
/s and argumentum ad absurdum, sorry for eristic tricks.
Hasn't been my experience for the past year or so, give it a try sometime. Incredibly relaxing and frees up my mind to focus on potential hazards further than just one car length in front of me. Sometimes (don't hate me) frees up my hands/eyes to handle other things that need some temporary attention. Traffic jams, getting cut off by impatient drivers, being stuck behind slow drivers, none of it bothers me. I'm chilling out and talking to my kids and enjoying the scenery.
The current FSD system has no relation to the old Autopilot, is enormously better, and very pleasant to use. A year ago FSD was nice, but I had to take over occasionally. This year it's much better, and I've yet to see a weakness, aside from some parking issues.
It's definitely a better driver than me!
All past inventions have put some of the most miserable humans out of a job. Farming by hand is hard, so is copying books and don't even get me started on the dullness of making clothes by hand.
Driving all day isn't all that far off, though still much better than the above and definitely better than riding horses all day.
Telegraph operators? Hell no. Dude sitting in an elevator all day? No thanks.
It's always been from the other end. This time around, there's gonna be some 10-20 years of robot manufacturing, then a universal lifetime of robots fighting misaligned robots until enough paperclips have been manufactured.
Or you know, we block out the sun and live in caves or whatever you fancy.
Do you live under a rock?
The same isn't true for elephants.
Developers are guilty of hubris and they are getting slapped back to reality.
It was too easy to assume that an AI would never approach human intelligence and hence would only be used for “boring” jobs like driving.
I’ve long disliked the hype around AI but now see that human intelligence is perhaps slightly overrated, at least in terms of the loftier positions seemingly requiring such.
Claude Opus puts to shame any highly caffeinated “10x” software developer. Not saying it’s perfect, but damn good. It can knock out difficult code analysis/reverse engineering more than 20-40x faster than I can. There was even a time long before AI when people thought the same of me…
Hell, at this point, I’d rather deal with Claude than a low-cost offshore team that puts up crappy implementations with endless go-backs and rewrites. I say this from actual experience, not xenophobia. There likely are some high performing offshore teams, I just wasn’t lucky to have the chance of working with one. Have had a couple of local colleagues that were worse… Claude’s probably much cheaper too.
Unfortunately, the “knowledge worker” jobs historically tended to be higher paid.
On one hand, AI is a major asset to experienced professionals with access.
On the other hand, it robs junior engineers hard-fought experience - much like watching a movie instead of reading the book.
And longer term, I think the entire economic structure of society is going to change. Doctors, lawyers, software developers, engineers — none of these are safe from AI reducing the need for headcount.
Which role is that, and why you think so ?
On other hand, in a completely fair market, i think AI can make profit. A lot of traders just stay at home and monitor news.
Intellectual pursuits are, by comparison, relatively new. Part of the reason they might seem so impressive to us is simply because we can't rely as much on eons of biological evolution, so even though what we are doing most of the time is really simple it seems "smart and sophisticated" compared to the far more complex mental operations that we are lucky are just automatic.
Think about it, you have an entire complicated as hell biological interior system that your brain manages continuously without so much as a conscious thought.
This is also why this hubris around the idea that LLMs rival human intelligence is quite laughable. If anything one of the major reasons they are so "great" is because they don't have to deal with a host of issues themselves (their computation is not responsible for keeping their bodies alive and hearts beating 24/7, data center ops takes care of that, unlike our computational system which, in addition to solving all these intellectual problems, also needs to regulate the body continuously)
There is more than one way to skin a cat.
Funny when it’s someone else eh?
Just learn a new trade Terry.
(I am kidding if it wasn't obvious)
> Of course you get a lot of right-wing people saying "oh these Poles; they come over here taking our jobs". I think if a guy from Poland arrives and he doesn’t speak the language, he doesn’t have any money and he doesn’t know anyone, and he takes your job on the first day – you’re shit, fuck off."
https://www.youtube.com/watch?v=t2ICmOYP8gs
Somehow I think "taking our jobs" concerns are going to be a lot more palatable.
If the pole is willing to work for a third of the price the local(I'm assuming Englishman), of course it is detrimental to UK labour.
Pretty good for the capital owning class, though.
I suspect their internal harness, models, and thinking budget far outstrip anything us peasants have access to (never mind the compute). So saying that they achieve all these headline results with 3 hours of GPT Pro is disingenuous. If that were so, hundreds of math phd students would’ve already recognized it.
I’m coming to the realization that my monthly subscription is only helping subsidize their real internal AGI, and my interactions are a small amount of human feedback and creativity which they can strip mine should they choose to.
It's hard to prove, but I'm personally extremely convinced of this. The point of AI (from the perspective of these companies) is to solidify and centralize their control of the economy and the world.
Therefore the #1 focus of their research and efforts and expenditure will be something that they don't want any of us to get access to. I think many other behaviors like model nerfing follow from that. It helps explain these frontier research products which we ourselves cannot get access to.
I have faith that when AI settles, it too will open more doors than it has closed.
Real grief for mathematicians, coders, artists etc (not waved away as "he is just in stage 2 bla bla").
But also: yes of course, the challenge now is to use these supposed super powers for good.
Certainly, it’s like being made aware of cheat codes to video games: One doesn’t enjoy the game as much after but surely knowing something is true means you can find many other things that come from the pathway to it now that you know it’s true. Lots of science and tech is like this where someone shows something is doable and then people find better ways to do it but until that person showed it was doable there was no progress.
> I am horrified by the many person-years” of research, particularly by young mathematicians, “being casually disrupted or destroyed,” he wrote.
That’s what technology does. And he might be upset but we’re the ones paying for it and frankly, I’m happier to have more questions answered than for fewer answered but in an artisanal fashion.
Most notable is the fact that the released facts are like “treated waste”. But if the question was worth answering and the value is only in a good answer and not simply a correct answer, then the value has simply not been realized.
But that feels strange then that the value is in the good answer, not just a right answer and yet one finds oneself unable to go get the good answer because a correct answer exists.
The fact that any answers are made public at all is useful to us all. That’s free knowledge. Would it have been better if OpenAI had discovered this very achievable fact and not said anything? Surely the papers, post discovery would have spoken of the “hoarding” of such “essential human knowledge”.
Overall, I find the thing unconvincing wholly.
Would you prefer if graph theory didn't exist but we could answer if you can pass all bridges in Königsberg, without going through same bridge twice?
Or if we could answer any question without knowing why? What about answering Hilbert without theory of Godel's Incompletness theorem? Or knowing Nth digit of Pi without calculus.
Because that's essentially what you're asking for. If you gave humans a Djinn that could answer perfectly any question, it would kill most of our endeavors. Because it's not a problem that was unsolved for years "It's just the thing Djinn already answered so why bother?" Even if one is extremely curious, and devote a chunk of their life answering it, others would not see value in that curiosity. Because a Djinn can answer it in a few minutes.
Now math isn't my wheelhouse, but every LLM I've seen provides far more than just the final answer and is quite willing to answer questions.
Imagine you are Euler and magical Djinn's exist (that possess all mathematical knowledge written up to that point exist and function identical to LLMs). You ask it to tell you answer to the bridge riddle and it says that's not possible and even gives some plausibly sounding examples even. Then it throws a 11 pages of gibberish proofs at you, which are all true but very convoluted.
And what if you start digging into it? Won't other people say - "Hey Euler go back to ploughing the fields? We can have Djinn's answer all the math questions but this field isn't going to plough itself."
Today's, the Djinni-Euler's equivalent of someone looking at LLM's Navier Stokes equation proof.
Perhaps they would discover new Liquid Set theory that would turn Liquid dynamics from a field fraught with problems into a beautiful mathematically simple meadow.
Seems like an overly pessimistic take to me.
If you have a magical Djinn happy to answer questions, why wouldn't you badger it a bit for the details? I can't imagine Euler of all people just wanted the answer.
Remove the human theorizing and the LLM won't be able to fill in that gap. All it can do is remix. It cannot formulate and, equally importantly socialize new mathematical theory. Without social acceptance and promulgation theory is meaningless. I might have a private "sort of graph theory but not" studying a slew of new mathematical objects called quizzgams, but if I cannot actually articulate this theory to anyone and get an active participatory body of people around it, it's pointless. Any "theory" an LLM can create de novo is such a theory. Because, again, it doesn't actually produce novel thought, just novel byte sequences. A thought is more than just its encoding, it is processual and social, which all the nerds and vc bros seem to not understand.
If I were a Claude instance, rather than a software developer fielding these questions as a small component of my job, could I take a step back and instruct them from first principles until they do understand? Maybe. But maybe there’s just too big of a gap to be crossed in one step.
But the value in seeking mathematical answers was never in the answers themselves. It was in the things one discovers along the way, the questions asked, the tools developed. The artisanal fashion is the part that benefits non-mathematicians.
These AI proofs only serve as an I-told-you-so. They're not creating surrounding theory which can later turn out to have useful applications elsewhere.
Given that this is pure mathematics we're talking about, for the most part: Why?
Perhaps you are a mathematician and so these questions are relevant to you, but for most of us, these questions and their answers will not make any tangible difference in our lives or society.
It used to be that Pure Maths got to affect the real world via Physics (and that odd thing where last decade's interesting mathematical exploration turned out to be the solution to this decade's Physics discovery). But t'internet consumes Pure Maths pretty much raw.
When mathematicians discover new results, they automatically become experts during the process, and then go around giving talks to explain their results, methods, and intuitions to other mathematicians. After years or sometimes decades of refinement and a healthy dose of serendipity, some of the mental frameworks percolate to other fields.
No one from OpenAI is willing to give a talk on any of their 300+ results because their machines have decoupled formal proofs from actual knowledge.
edit: lest this be misconstrued, it can become their problem if it is wildly unpopular and undesireable behavior is regulated away.
I advocate for this path.
"Not our problem lmao, our IPO valuation went up"
1. If it costs $6-22 million to produce a paper that the brightest minds in maths can't understand and use, this trend will stop soon enough on its own.
2. Why are these minds complaining that there are no authors to discuss the paper with? There are OpenAI researchers to talk to. If they refuse, because they themselves don't understand, then don't bother with it? You're responsible for your time. Write a response on what's unclear, and move on. What's the point of whining about others writing things you don't understand? See point 1.
3. If LLMs can be used to solve problems, that leaves more time for mathematicians to come up with follow-ups. That seems a great way to speed up the field, no? Just be sure you understand the solution before expecting anyone else to spend time on reading it.
Given the trajectory of LLMs to date I see them helping construct proofs for conjectures but not devising new conjectures of any material substance any time soon. The challenge is, as Tao has said elsewhere, part of being able to formulate a conjecture is by exploring the landscape around it by doing proof research. Seeking answers finds useful questions. The fear is the tools will take away the exploration of the landscape. But that’s silly IMO. It takes a special type of lazy to let a tool take away your understanding that’s important for what you’re goal is, and a special type of passion to learn with a powerful tool. The passionate will adapt and progress will accelerate, not slow down.
For instance Tao has posted some of his interrogations of proofs with AI assisted reasoning and it didn’t seem to diminish his ability to establish insight rapidly, but to shut down avenues rapidly and to pivot and expand his understanding rapidly. The insight was all his own, as was the path of inquiry. The understanding of the output was his as well. He carries it into tomorrow and uses it to formulate new more interesting conjectures the next day. The AI ceased to “exist” as soon as the inference turn was over.
The hand wringing is pointless and lame. John Henry moves aren’t necessary or useful, and when I see a tunnel boring project now I see tons of people well employed there, just the holes are bigger and better, and they’re engineers instead of strong backed hammer slingers.
Because it comes across as "ooh, you're just going through the five stages of grief, dummy! Look I've taught you psychology 101. Now that you know about grief, you might as well skip to the next cycle already. Support me on Patreon. Like, comment, subscribe"
Pointing out that someone is "just" grieving is not helpful.
I'm seeing this a lot in the past months of AI-for-math discussions. Had to get this off my chest.
It's not at all clear whether the kind of math created by LLMs will retain the usefulness of the old math. LLMs are good at rigor but bad at intuition. It's not even really clear if the rigor was ever all that important. Theoretical physics is full of people using math in ways that don't meet the bar of rigor that mathematicians set for themselves, but they have enough physical intuition for what they're doing, so it doesn't matter. It's an open question whether post-LLM math will continue to supply scientists with the kinds of ideas that they need from it.
Experts in fields are still required to validate the claims of AI. For mathematicians like Terry, people like him are still required to read through what the AI has come up with and confirm or create more experiments.
We don't yet live in a world with super AI hooked into everything and everything so it's not like AI can actually do anything with its new knowledge yet anyway. You cannot just hand those AI maths papers to anybody and expect them to understand what's going on. Nor can you expect the AI to find a revolution in fusion technology, send plans to its factory and have robots build a new type of fusion reactor because we do not have that level of integration yet.
But neither do we have UBI and we are still hurtling towards the hard ground of capitalism. This may resolve itself though because I'd imagine once we get proper AGI, the "meths" won't be able to keep it under wraps forever, eventually weights will leak, hardware will improve to where individuals can run them. Or an intelligent enough AI may decide to be benevolent and buy and run skyscraper greenhouses for those who are most in need.
We've already had this solved for a long time, but nobody has implemented it yet. Automating math isn't going to get us closer.
It's telling that isn't his response.
I'm not sure this article is a good way to figure out his stance on the issue, I'd recommend reading his blog posts or mastodon threads.
"the way OpenAI is doing it" - unfortunately while mathematicians have had an internal culture of not stepping on each other's toes, their ingroup is still susceptible to the same disruption as every other field as AI shows up to eat their lunch.
You can either get with the program and discover this is the greatest opportunity in the history of mankind to accelerate knowledge, or, you can cry over your lost prestige. Terry has chosen the latter.
Aand it's gone.
"Oh wow, that opened the door to a whole new area of..."
Aand it's gone.
That's a lot less than i expected. The value he gave back to them should be at least 2 magnitudes higher.
Can you help me understand the difference in your analogy?
Is the cost prohibitive , or a proprietary model?
It turns out it doesn't matter, because outside of a few narrow bands like Claude Code, no one cares what documentation an LLM writes for your project, because no one is ever going to go looking for someone else's project or library to solve their problem anymore.
So either an LLM decides to pull in your project as a library for reasons, in which the LLM can understand the documentation, or else no one besides the creator is using it anyway. So it doesn't matter.
Why would anyone use someone else's AI-coded project if they can get their own version that better fits their own needs in less time?
Terrence Tao was very supportive of AI and formalizing math with lean when it looked like it was giving ‘generalists’ like him an edge over mathematicians too deep into one particular niche field.
But now that it’s letting everyone, including mathematicians who are perhaps too specialized, explore ideas and patterns from related math fields, he’s suddenly ‘on a war footing’.
basically both sides now dislike that he flip flopped
its either you are anti-AI all the way or flexible and adaptive without making strong opinions that can be put on record.
most people are in the latter and its growing. math is solved.
No, the dumb thing is the idea that there are "sides".
> basically both sides now dislike that he flip flopped
Literally the only people with this opinion of Tao are internet forum warriors who have no expertise in the field and who's opinion of Tao don't matter in the slightest.
Similarly, Tao being ‘on the warpath’ does not matter in the slightest when it comes to formalizing math and finding proofs to important unsolved problems with AI.
Do you think that this is a quote from the article you just read?
I swear half the comments here are just from people reading dunks on r/singularity and x and getting real mad at strawmen.
So not remotely the same thing.
I find it amusing that you would be the one saying it here.
Eh... That “everyone” is a multi billion (trillion?) $ company paying top mathematicians specifically hired for generating math proofs.
We’ve yet to see the democratizing fantasies in math field to materialize.
What privilege? Did you miss the part where it doesn’t pay well?
Funnily enough, the understanding of modern economy is achieved through mathematics, developed by mathematicians and economists (using tools provided in math)
No, they are not.
Consider any man at his job, fighting the good fight. Your boss says "I've hired this man to assist you" and you are happy to finally have some help with your difficult task. Then one morning you come into work and see that the coming day’s work was done overnight by this other fellow. And the realization sinks in, he isn't there to help you, he is there to replace you.
In this view his reaction seems totally expected. I expect many professionals, e.g. doctors, lawyers, accountants, etc. will make this realization sooner or later. The companies are saying these tools will assist professionals, but in reality they are aiming to replace them.
lol. lmao even.
I guess there is a first time for everything.
Regulated fields - another clueless poster lol.
I know many of you really hate whats happening but don't try and project it on other professions where the environment is different.
Society largely never cared about maths.
You are, of course, right that doctors, lawyers and accountants have more control over the environment they work in compared to mathematicians. But it doesn't change the underlying argument Tao is making. Your observation that those professions will react differently is likely to be true and only time will tell how effective their response will be.
But AI "assistants" is a marketing approach being made to all professionals at this time. Tao is implying this is a Trojan-horse strategy. If you let the assistants in without reasonable protections, you may lose control over the environment you are trying to protect.
Out of sheer, idle curiosity: were you actually laughing when you typed that out?
If anyone doubts this, consider that Anthropic considers its TAM to exceed $30 trillion which is nearly the entire GDP of the US and 25% of the world GDP.
Only makes sense if they believe they will massively replace labor. Which, to their credit, they used to admit openly and publicly and only stopped because some crisis PR people told them "hey maybe you shouldn't say the quiet part out loud while the hoi polloi still have the power to stop this from happening".
You're not this naive are you?
Amazing, truly.
They have practically infinite investor money to throw at salaries (lost in the noise of how much they are spending on compute) so why not use it for now to reach their goal as fast as possible?
My thesis isn't that they have already achieved the ability to replace massive amounts of the labor pool, just that that has to be their goal given their stated TAM.
> If they want to flex the power of their models to automate human jobs, why not lead by example?
They don't want to flex this. They realized (after a false start) that it is better for them to avoid talking about this in public.
That doesn't mean their goals changed, they just realized that flexing before the goal is achieved could endanger the goal being achieved through political backlash.
Wouldn't throwing that money into GPU be better because AI is smarter and faster ? So they would reach their goal even faster ?
As one of the smartest people in the world, he is so naive to believe that those capitalists will stop at that.
They also had a rich starting point from open source code (which we somehow gave up without realising it). All of this won't happen as efficiently for these other fields, but it will happen, much in the way you've described. We (swes) turned out to be a rather dumb group of people, which is hilarious since we think we're clever.
The only moat we have is the human to human element, we need to talk to other humans in the business. Customers, etc. But once they're replaced by machines or automation, then machine to machine communication will be the preferred mode of communication and work and the legacy humans will be increasingly cut out of the loop. Human in the loop is a death sentence we just don't realise it.
> The companies are saying these tools will assist professionals, but in reality they are aiming to replace them.
They are openly saying that the goal is to replace humans and make humans obsolete. Like, literally openly. I am not saying I agree with that goal or find it realistic, but they do think so.
“AI won't replace humans. But humans who use AI will replace those who don't.”
vs
"Some areas in the job market will be 'just like totally, totally gone' as they're replaced by AI agents.”
and
“AI will probably replace most of the jobs people do today,”
The most recent position I could find says he has revised his earlier position on a total replacement due to the lagging adoption of AI in many industries. But I don't think there is a single position we can point to that they are "openly saying". It is fair to grant even the CEOs of these companies a change in position as they gain more information about capabilities and adoption, rather than insist that they hold, have held and will continue to hold fixed opinions.
The point is that Tao's understanding has changed. He may have been naive before, he may have heard in their words something he wanted to believe. Perhaps he was misled by people who he trusted that worked in the company. But now that he has more evidence, in the form of the maths drops and the way they have been handled, his suspension of disbelief no longer holds. The companies could have done this in a way that included mathematicians, so they chose not to. And that choice reveals their aims unambiguously in a way that a media-trained CEO or careful PR department can avoid.
That suggests that Tao is not being "self-serving" as others claim. He is saying that his optimism in the best-case scenario turned out to be incorrect.
Terrence Tao is famous because of solving problems better that most mathematicians because he has an uncanny ability to look across different math fields. AI gets everyone this ability for 20 to 200$ per month, eventually free.
Most mathematicians are happy to see interesting problems in their field being solved, and are looking to build on these to find more interesting problems.
The point is to make the field progress by discovering new things!
Who gets credit for what and how credibility is assigned is going to be muddy for a while but that’s the case whenever a bunch of new science happens in close sequence. History is full of math/science discoveries having debated and dubious provenance.
It provides deep insights into logic, computation, provides tools for us to understand the universe, establishes mind blowing relationships between seemingly unrelated things.
For example “how much information can a black hole contain?”
To whom? If there are no humans in the field, there is no one to get the insight. Maybe you are right, and there is just a new type of a mathematician being born, but as an outsider I just can't see what they could look like, what will be left for humans to contribute
AI solves math problems to
> If there are no humans in the field
Realistically, math will be done by well funded teams of people instead of cult of personality mathematicians.
Also, insights from scientific discoveries are for everyone. You do not have to understand the complex multi dimensional algebraic geometry of general relativity to appreciate that gravity bends time as well as space. You don't need to understand fifty years of linear programming research to use a modern solver.
Most math is already being done by meagrely funded mathematicians. The entire US federal budget for mathematics is like $100M. One college football coach can already make 10 percent of that. No mathematician is making bank.
Is the implication of all this that mathematics research should get a cash injection to pay for high-powered computational tools?
Math will, sure. Mathematicians, doubtful. After all, most high end physics research with billions in funding is now done by postdocs making a lower middle class salary.
It has been proved a hundred years ago that some problems can not be proven. So there are problems AI will not solve.
I just wish I was young enough to live closer to its peak.
Humans have been guilty of burying things we don't understand for millenia. Will we continue to do so "because AI"?
I think this sort of progress, compounded over the next N years will serve to cleanly split useful math from the ego driven math you speak of.
Certainly there are enormously impactful pure math results yet to be found, but we have no idea really which directions to pursue.
I wonder if this is going to look like the chemistry-alchemy split in the rear view mirror.
There are plenty people in every discipline that prioritize their ego over measurable progress towards a stated goal - even less so a community focused goal.
Humans aren't that original. We just reinvent stuff we used to do to suit the present day, with our own twist.
In Pure maths, people are not generally solving problems because of some pressing practical need, it's purely out of intellectual interest, and if we're lucky there's some immediate practical application. More often, the knowledge sits in pure form for a while and years down the road some new invention, or practical consideration, or other field finds a useful application.
Just like literature and visual art, it serves an important epistemic and social function even in spite of its immediate lack of raw practicality.
You are right, we should get rid of their profession
But to be honest the reaction from the ivory tower here is seemingly quite telling to a layman watching from afar.
It looks far more like this was some extremely gatekept community where you were only allowed to work on sanctioned things if so deemed worthy. Better not step one someone’s toes lest you get exiled! How much progress was lost due to newcomers never working on a problem due to told not to?
It’s starting to make those old timey Usenet kooks look a whole lot less crazy to me. They seemingly were right about this part.
Mathematics was funded and supported by a whole bunch of janitors and blue collar workers busting their asses to let entire departments of thinking men and women work on hard problems for the betterment of society. If it was just a super special friends club the whole time that was gatekeeping what amounts to a hobby, it’s going to me much harder to justify such a social contract.
Honestly not even sure what to think at this point. Extremely disappointing to me the more I read from these people.
Admittedly it's also just really hard to come up with something better than the current track, but doing better has to start with being eager to explore different directions. Even if the new directions don't quite work just yet. And that eagerness just isn't there in the community, new directions are threats to the careers of sensitive people
What are you even talking about here? Mathematics was funded by janitors?
How do you like them apples?
Case in point: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
The information in the article suggests that he regrets his prior hyping of AI.
OAI first paper on the prize was a pretty shitty thing to do. But the rest of this looks like they are just mining the corpus and taking a brute force approach to do it.
Im going to say this again, the "pure math" academic types need to get a grip. Plenty of your peers got PHD's and happily made a fortune in finance, while screwing the world (See 2008 housing bubble) and none of you said a peep about that.
They call it the most populist post of all time.