Hacker News Reader

Comment by bonoboTP | Hacker News Reader

Ok, where will you shift the goalposts if next year AI will also be superhuman in didactically explaining proofs to humans?

View on HN

AI capabilities keep getting bumpier and bumpier. There's no reason to believe it won't stall out on a front that it is already quite bad at (correctly formalizing in human language)

“LLM progress will stall in some area at some point” — which is my interpretation of what you wrote — feels particularly unfalsifiable

The negated statement --- that LLM progress will never stall in any area --- seems trivially falsifiable, so why is the GP's statement unfalsifiable?

This incorrectly assumes falsifiability survives negation. https://philosophy.stackexchange.com/questions/52015/is-the-...

logic vs wisdom

It's a new technology and we're figuring out important distinctions as we go, nothing necessarily intellectually dishonest about that.

If shifting goalposts makes AI better, why not shift them?

I don't think that is the thing that makes AI better. In the metaphor the idea is that someone scored a goal and then people retroactively shift the goalpost to say it doesn't count as a goal.

Is there any evidence it does? Without that it simply feels like poor debate.

Because making AI better is orthogonal to making human lives better. We are quite literally racing towards artificial superintelligence without any serious plan or preparation for the societal impacts that will have.

Did people prepare for or ever fully foresaw the downstream impact of any other major technological revolution, such as farming or the industrial revolutions or the Internet?

Has there been any other major technological revolution that underwent exponential recursive self improvement?

If AI development stopped right now, we would see a technological revolution at a scale similar to the Internet. AI development has not slowed down, let alone stopped. This technology demands far more planning than we are allocating.

I don't think we are smart enough with enough foresight of consequences to plan this. If we used AI, however...

AI inherently has misaligned incentives (as do the companies developing them). I don't trust them to outline the consequences of something like this and to develop an appropriate plan, and neither should you. I believe in humans, but we need time.

Reply on news.ycombinator.com