Mathematics Without Mathematicians
borretti.me
borretti.me
Well, that's just where we started — and then we developed science and mathematics to understand it. Why would it be any different if the demons live in our technological progeny instead of the natural world?
(I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.)
My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells.
Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in.
--
The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part.
This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms.
--
Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves.
That is the biggest reason wrapping our heads around this change earlier than latter is important.
We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded.
The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk.
If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I believe our lives depend on it.
It seems far more likely that AI will be working in the service of a government to oppress change and violate any human rights that are standing in the way.
Keep what’s happening today in your mind as you consider the future.
I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself.
But AI in an unethical society is suicide for humanity.
As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker.
We all need to put more steam into the pushback against corruption. And help others realize the threat is real.
99.99...9% of humanity has a common interest, if they can recognize it.
The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization.
I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.
Celestial mechanics suggests it can hardly be economical to move large amounts of matter around a solar system for economic goals considered in the current capitalist societies. Nor the risk, nor the energy needed are in line with the benefits.
A fully fueled ship in Earth orbit can go to Mars, land, and return. Same with most other round trips.
Not to mention, fuel can be generated in the belt outside any noticeable gravity well.
Tell that to the Europeans who were so afraid of naming the bear that its original name is still lost to us :) The Luddites also had a lot to say about the mechanical demons that were eating their lunches.
> Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Yes-but, time scales are relevant only relative to other time scales (or the Planck time, if we get there). If the whole universe were to start running at 1,000,000× speed, you wouldn't notice. Speed is important to us humans currently because of our individual perspective and associated limited lifespan, but not in a universal sense (modulo some big hurdles like star lifetime and heat death). There are already self-replicating organisms with much faster iteration cycles than humans or human technology, like bacteria.
> Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
Just like it was in the 1980s, or the 1800s? I don't claim that these things _won't_ happen in the next ten years, but looking at it empirically people thinking that they will coïncides with every new technology and is a poor predictor.
> If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I don't disagree, but it's not like we haven't been trying. Attempts to codify ethics go back about as far as human civilization, and it's not obvious to me that there's any faster road, and certainly not one fast enough to outpace our current rate of technological development (before you even start talking about AGI!).
["First time?" meme]
Illustrations: https://matt.might.net/articles/phd-school-in-pictures/
I am hopelessly ill qualified to judge.
Edit: Looking around at context, this has major crank warning signs (claims to also prove the conjecture, author has claimed solutions to both the Riemann Hypothesis and the Navier-Stokes problem and is involved in "alternative propulsion" research). I doubt OpenAI would have published without thorough verification.
Mathematicians are not going to be well-served by cowardice-based thinking on the situation confronting them.
I would gently suggest that you may be assigning the most uncharitable interpretation to people's honest attempts at having an intellectual discussion around these topics.
There is, but having that discussion without a discussion and understanding of the velocity with which those strengths are improving and those weaknesses are disappearing is pointless.
Worse, much of the discussion is already outdated by where the latest models (with proper agentic harnesses) are.
> The AIs will exceed humans in taste and intuition...
> The AIs will be better teachers than the humans. In any case, there won’t be a human audience for expository work of frontier math.
> No human needs to understand any of it...
It seems like the person has no idea what they are talking about, or just too sarcastic to the point of idiotism.
My only cope among those was, using the author's words, "We will teach the mathematics AI discovers." This was back in 2018, before GPT-2. I abandoned it when I realized AIs will become better at explaining math before they become better at doing math.
I still have some hope in the last one "mathematicians and the mathematically-inclined will still do math on their own". Maybe some limited communities of this sort will form.
I am not so sure. I am talking about math with AI and I feel as a math teacher, it is terrible. It introduces new notation and concepts faster than I can ask for definitions. It doesn't provide many examples. Sometimes it realizes it made a mistake in a guess and the confusion compounds.
Maybe it's just my (in)experience - but I would welcome a practical advice on how it could explain math to me better.
Instead I am convinced that if modern LLM teachers were given the same luxuries as human teachers, much if not most of this performance drop would disappear, and probably result in higher explanation skills in LLM's.
Which luxuries? a human teacher at a certain school / university and at a certain level has the luxury to have an understanding of what prerequisite knowledge the students already have. The educational system has already sorted them by "level" and "topic" at the start of the first session of a lecture series. Classic texts would be used for teaching materials, or the local text "inherited" from teacher to teacher, and those would already have been impedance matched to the preceding and following courses in the local educational system, such teaching aids would effectively have continuous notation conventions etc. Also compared with long-term human 1-on-1 teaching, I believe that if the educational chronology of the pupil were continuously added to its training corpus for additional post-training, so that it doesn't need to maintain ridiculous context lengths, then I don't see why an LLM wouldn't be capable of systematically introducing the conventional names of symbols it introduces, when referring for them for the first time, or providing definitions instead of referring to certified unknown-to-pupil concepts (which today an LLM can not magically guess, you have to drill into it what you know and what you don't yet).
Everyone has experienced LLM's explaining things and introducing acronyms without their expansion, and everyone has observed that such phenomena can be addressed by stating such demands in system prompts, or repairingly simply demand them in the conversation upon hitting this annoyance.
Imagine future math students (if any motivation still exists to become one) the education software making use of LLM's could train the student during the day, and itself do post-training on all its observations, trying to predict latents that describe which parts of mathematics are ready knowledge and which parts aren't for the student.
To point at a privilege vis-a-vis LLM's and then maintain a raised level of self-confidence does read like cope to me.
If one really wanted to taunt the performance of LLM's by gathering lists of such things it is incapable of, it means one can turn this around and start with collecting lists of priviliges a normal human teacher (or other job position) has vis-a-vis a naive LLM implementation of a human teacher (or other profession). For example a human teacher typically has the privilege of repeatedly teaching the same insights year after year, if different knowledge aware LLM's were connected or offered by the same service, it could build statistics of which concepts stick better and why, where things are hand-waived too much, etc... But the only result of publishing such cope-lists is providing gratis business models to whomever feels like implementing it, destroying the validity of the cope-curtain.
"Guys, I think I see zombies outside of this window!"
"Oh, just close the curtains of this window then if the McKenzie's are acting weird too... what a pity, they were the ones who showed me how closing the curtains is sufficient to keep the Zombies at bay!"
Explaining in the sense of getting someone who doesn't get math to actually get math, giving illustrative examples and so on, maybe AIs are not good at, but anyway it's not something I enjoy much.
I am not sure what writeup you refer to. I have looked at proofs of 10 problems from Open AI written in Lean, I would not call a megabyte of Lean with no comments a good exposition.
I think it’s pretty likely. Contrary to all the negative things said about mathematicians in this thread, in my experience they’re mostly not profit-driven (for the ones that are tend to filter out of academia during grad school, it’s a high-effort, low-reward environment).
Tao does not seem particularly threatened by AIs doing math, and I trust his judgment.
One of the effects of this entire debate is it's exposed how many people claiming to be pure mechanistic rationalists actually never were. At least those of an openly religious inclination have been sticking to their position consistently.
Imagine being able to only throwing money and patenting everything, even if it existed before. The legal behinds this will be so mess that no one will be able to produce anything
It’s akin to the subset of cryptocurrency proponents who fervently argue that people who are against cryptocurrencies have that opinion because they’re jealous they didn’t become rich. It shows a profound lack of understanding—and, more importantly, an unwillingness to understand—the other side of the argument.
This is made evident by the very first “argument” of moving the goal posts. It’s dismissed off-hand, without even explaining what those entail. That’s the same as saying “if a person makes a fallacious argument, they’re wrong and you can ignore them”¹. But crucially, sometimes what we ourselves think is a fallacious argument, isn’t. “Moving the goal posts” is probably the most misused accusation I see in LLM discussions. It’s basically a talking point by now.
¹ Which is itself a fallacy: https://en.wikipedia.org/wiki/Argument_from_fallacy
Taking arguments seriously doesn't necessitate agreement.
I agree. That’s what I’m saying, so I don’t think I understand what you mean. Mind clarifying the point you’re making?
To try to clarify, what I do mean that is that disagreement does necessitate at least some measure of taking arguments seriously. Some arguments maybe don’t deserve to be taken seriously, but you need to first be open to taking them seriously so you can decide.
In other words, if you go into an argument with the initial mindset of taking it seriously, by the end you’ll be able to decide if it should be taken seriously or not; but if your initial mindset is to not take it seriously, most likely you won’t be open to understand the nuance of what is being said. Thus, the former leads to productive discussion while the latter does not.
I suppose I have also interpreted downplaying of AI's impact as largely coping too.
The arrivals of machines that can outthink us (or will soon) only 80 years after the invention of the transistor is so clearly (to me) a change incomparable to anything that has happened prior. It is very hard to relate to arguments that downplay the complete upheaval to the order of life on Earth that non-biological cognition and interests herald.
It does not seem surprising to me, in the sense that we humans ourselves, and our technology, are of a different order than anything that came before. We have not stopped accelerating change. AI is (will soon be) building directly on that. But we will no longer be a bottleneck.
My strong impression, is the faster things change, the more people tune out that the change isn't stopping. Some of that reversal-response must involve coping. Perhaps another reason is that faster-than-us change is challenging to imagine.
But your point is valid. False or uncertain assumptions of "cope" motivations don't help anything.
Plus, no, multiple of the problems Astra solved do have real-world applications in computer science, and perhaps even chip design.
Perhaps already you are not the intended audience. Slop-raised parrots will lap it up.