Hacker News Reader

Comment by sigmoid10 | Hacker News Reader

This kind of argument is always dangerous, because it essentially resorts to moving goalposts. "Oh, AI can now do X? Sure it is amazing, but it can't do Y yet, so we're totally safe!"

Just because humans can't process the proof or see the advancements, it doesn't mean that it will remain that way in the future or that it will not change the field. If you only define yourself by things AI can't do yet, you're about to have a rude awakening. We've gone from high school, to university math, to Euler problems all the way to Millennium problems in a time frame most people couldn't even do a PhD. If you start any math research now with a horizon beyond the next two years, I'd be terrified of the current rate of progress.

View on HN

Fundamentally, the question is this: what is a large language model, what is it capable of, how does it differ from human cognition, and what can humans do that it cannot do?

A lot of people seem to believe that with more time and training, LLMs will surpass human intelligence. But they are fundamentally not like human intelligence. They do not reason, learn, conceptualize, think creatively or abstractly, even though we have some hacks to mimic these behaviors.

I posit that treating LLMs like GPU-powered brains that will eventually surpass us in most fields is pure science fiction if you know anything about how they work. I think they will remain astronomically powerful in some areas (pertaining to fuzzy deep search and recombining existing knowledge) and hilariously bad in others.

>They do not reason, learn, conceptualize, think creatively or abstractly

Since we lack a definition of all these things or a clear understanding of how they are realized in humans or LLMs, the only thing you can really claim is that LLMs do not look like biological neural networks when it comes to processing information at the fundamental level. Everything else is dogma. But since all this stuff clearly emerges as size of the network grows, we can not ignore the possibility that complexity itself is the only requirement, not architecture (hardware or software). At the most basic levels, we are also just machines that evolved to make more of themselves and learned to interact with the environment to achieve that goal more efficiently. Humans will certainly have to come to terms with the fact that we are not the pinnacle of intelligence in the universe, but that won't stop them from denying it.

The authors are absolutely not saying "we are totally safe". They are not denying that AI will affect math deeply in the short and the long run.

If you ask an AI if "P=NP", it will certainly give an answer. It has to, because the answer is either "yes" or "no".

If you ask it to then prove its assertion, it will then certainly output the tokens that look like a plausible proof. It has to, as it is programmed too.

This "proof" might even be thousands of pages of very technical looking and professional sounding jargon.

It's not a real proof though, and as an artifact it is 100% useless to both the field of mathematics and humanity.

That's why this proof and all the other high profile AI proofs were written in lean, which allows you to verify them in a formal deterministic language. You still might not understand it as a human, but any computer with a simple processor can verify the proof's correctness. And from there you will undoubtedly see other people make sense of the proof's key steps using AI too. And the models might even pick up on further details useable for other proofs that humans didn't see. In the end I'm 100% convinced that abstract math will eventually be primarily done by computers, similar to how linear algebra and numerics have been done exclusively done by computers for a while. Noone would even consider multiplying a 100x100 matrix by hand anymore, if only because it is much more likely that you as a human will make a mistake.

That's assuming that "formal deterministic language" is equivalent to mathematical reality, or that it even reflects it correctly. It's the accepted view nowadays but akshually quite the hot take.

Strongly reminiscent of God of the Gaps [1]

[1]: https://en.wikipedia.org/wiki/God_of_the_gaps

> This kind of argument is always dangerous, because it essentially resorts to moving goalposts.

Nope, introspecting motivation is useful. There have been similar posts made about people using bots in PvP competitive online gaming: some people play to enjoy the game, and hopefully get better until they reach their skill ceiling. Others play (just)to win, and are open to buying aimbot hardware and software

>introspecting motivation is useful

Not if the essence boils down to moving goalpoasts to the next best thing that will likely be flown past as well within months rather than years. Sure mathematicians who do this stuff purely for the love of it will continue to exist. Just like human chess or the retro-programming scene exist for similar reasons. But if you are a professional who expects to earn a living salary on these things, your intellect will suddenly be outcompeted by orders of magnitude. Ignoring this today by saying LLM's can't do this or that will undoubtedly set you up for a rude awakening. For all we know is that there is still no fundamental limit to what they can do in theory and compute may well be the only true limit.

> But if you are a professional who expects to earn a living salary on these things, your intellect will suddenly be outcompeted by orders of magnitude

Which organizations do the most math research? What are their budgets, droves and culture? You may be falling victim to the pre-IPO PR razzle-dazzle by AI companies expending unreasonable resources on problems they know will get them good press and make their models appear superhuman.

A university math department will not spend $20M in one summer to solve 1 problem when it can hire 10 human mathematicians for 10 years.

How is this theoretical university math department funded? Which college sports team or endowment is backing it? What do they want?

> This kind of argument is always dangerous, because it essentially resorts to moving goalposts. "Oh, AI can now do X? Sure it is amazing, but it can't do Y yet, so we're totally safe!"

It's not about moving goalposts. What you're not understanding is that, even if for you is crystal clear that AI will be 100x smarter tomorrow, those in charge cannot simply bet all on that. Right now AI cannot really replace the core sauce of mathematicians (all the "understanting" and "asking the real questions" stuff), so it'd be unwise for, say, countries to start making decisions as if AI is capable of that.

Technology advances, but extrapolating the value of human labour to 0 is equally invalid.

Reply on news.ycombinator.com