Hacker News Reader

Comment by benreesman | Hacker News Reader

With all respect to luminaries: this will not stand up. This will be treated harshly by history.

I’m nobody but I’m going to stand up to Terence Tao and Scott Aarinson: you’re wrong or bought or both.

This is a detour and I want to make clear to history what side I was on.

What's with the beef? In the paper Terence describes how he currently uses some imperfect but useful tools, which will surely change in the future if better tools appear. It does not say "LLM will be smarter than a kid bwahaha world domination".

This paper just namedrops ChatGPT. Previously, we had this:

https://mathstodon.xyz/@tao/113132502735585408

It may only take one or two further iterations of improved capability (and integration with other tools, such as computer algebra packages and proof assistants) until the level of "(static simulation of a) competent graduate student" is reached, at which point I could see this tool being of significant use in research level tasks.

some of the most demonstrably evil people alive at scale are extravagant in their claims of precisely “blah blah world domination”.

I’m calling bullshit on AGI. I’m repudiating as offensive both the ill intentions and the technical failure of these people.

I’m proposing that all sincere hackers denounce OpenAI as a matter of principle.

Did I stutter?

The beef is that many of us have enough grime on our hands. Many of us at the absolute elite frontier of this technology find its application in both technical reality and fundraising fantasy to be beneath any possible ideal.

And there are a lot of us.

What’s your reasoning?

There’s much more honor in being right for the right reason than for a wrong one.

I’m wagering my entire reputation that no LLM, nor any LLM run in a loop, will ever be as intelligent as a precocious child.

The burden rests on OpenAI and the scholars on their payroll to show otherwise.

I have a great many regrets in life but if I died opposing Sam Altman and Fidji Simo and Larry Summers in the newest version of their oppressive lies that would be a good death.

Respect.

This isn't really a meaningful prediction unless you define clearly your idea of what being "as intelligent as a precocious child" is, and how you would assess an LLM or any other system against that metric. Though I suppose you avoid the risk of having to move the goalposts later if you never set them up in the first place.

That's fine, and unrelated to the article in any way.

LLMs are way more useful than a child in many ways, some of which are discussed in the article. They don't need to be as intelligent as a child for anything proposed in the article.

Have you considered that children, and people in general, may be very significantly less intelligent than your baseline assumption?

A flaw in the Turing test is failing to specify which person is making the judgement. We're working with statistical distributions here and I would not bet on the intellect displayed by the LLM models being below that displayed by the human population today, let alone with more improvement to one or degradation to the other.

More concretely, if you sketch some normal distributions on a whiteboard for people vs machine based on how you see things, it should be hard to confidently claim minimal overlap.

Even without a definition of intelligence, this is not what the paper is about, which only mentions LLMs in passing. And LLMs can be useful even if they are wrong, because formal verification (though Lean and such) checks the result.

Are LLMs useful enough? I don't know.

Reply on news.ycombinator.com