Hacker News Reader

Comment by bmau5 | Hacker News Reader

From my understanding of how LLMs work, I'd argue thinking will become the most valuable attribute of humans (as it has been for some time). LLMs will inevitably surpass humans at functions that we know how to do, and likely through cross-domain knowledge and connecting disparate concepts will surpass us at many functions. But LLMs haven't shown they are capable of truly novel thinking, because it's next token prediction, which is inherently based on existing sources.

I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.

View on HN

Quite often I feel like my mind is pretty much doing next token prediction also. Unfortunately.

Same! I do think a vast majority of tasks are essentially next token prediction, but that's why I'm thinking that long, creative thinking periods will be what's actually valuable/rare.

when I was young we used to play lots of poker with my colleague mates, one day one of my friend said "any of yall seeing the cards before you sleep?"

I think that's whats happening with some of yall, seeing tokens every day in the screen then thinking you're just an llm as well.

I struggle to imagine the inner life of someone who would say this.

I expected someone would say this.

That’s how it feels like. ;)

More seriously, the only time where you’re jolted out of your prediction flow state is when you’re surprised, which means something happened counter to what your brain was predicting.

I have a bit of both. Most of my thoughts aren't really originally because I'm not playing around and go wild. This is what LLMs seem to be lacking a bit.

Most of my thoughts are things that other people have come up with as well. Aka it feels like I'm just predicting the next token based on the data that I already have.

It's only when I go to places such as "what if life at the speed of light is possible, what would that look like?" that I get to places I personally find somewhat novel. It's almost a different dimension for these guys of course since a million years can feel like 10 years to them. Yet, they're moving so fast, but time-wise they move so slow, maybe even too slow for us to understand that anything can be alive at that speed (I'm not claiming there's life at light speed by the way, I just think it's fun to imagine that there is).

You sit in the sofa. You want a glass of water. Maybe later.

Suddenly you’re moving to stand up. What made you do it?

Where does our thought originate from? Do you control it?

You did a quick calculation in your head in which getting up to get a glass of water now instead of 1 hour later just made more sense.

You can do chain of thought without verbalizing it. The LLM can't. Now you might broaden the definition of "token" to include whatever nonverbal pre-verbal impulse your brain produces. But then the definition doesn't mean anything. All state machines of any sort are doing "next token prediction".

> which is inherently based on existing sources

Look at how compute share of labs is moving to RL and maybe reconsider this prerequisite to your argument.

Open to learning. How could RL yield novel discoveries? What I mean to describe are things reliant on empirical or real-world testing of human reception. How would that be synthetically derived?

We're seeing the cost of code drop dramatically and are facing the same problem in building new companies across most disciplines of, "what's a problem painful enough that someone will pay us to solve it?".

I suppose if we get to a massive enough scale we can effectively model every atom's interaction and human on earth's reaction to any stimulus but I think we're still some time off from that :)

For a cost-prohibitive example, you could put a humanoid robot whose pretraining contained no chemistry, in a chem lab, and put it in a hypothesis loop, where it generates a random action, guesses the result, and then does it. Every time the lab burns down, you just build it again. Reward the model for producing novel actions and reward the model for correct hypotheses.

In other words, you generate more training data by actually experimenting, just like humans do.

I see your point itll be very jagged but llms are definitely very innovative. See the recent .ath proofs etc.

But i agree thinking will remain important. As my grandpa put it if something breaks who'll go and fix it ?

Trivial to prove false: ask a LLM to invent 10 new words. Then google them, most will not have any result.

> My favorite is Krezmoria - a word for missing a future you haven't experienced.

You had to prompt it to do that before it considered it, though.

depends on what you consider novel thinking.

Not sure what this is supposed to prove. Yes it can generate new words - that’s the application of established creative patterns to come up with plausible-sounding words. A different thing is to come up with a novel idea, which (in this case) is about creating a new pattern itself.

if patterns can be put into words, why wouldn't the same process happen at the word level? do you think it's impossible for a LLM to write even a simple python program that will generate millions of plausible random sentences in the general problem space we aim to generate new concepts for, then filter and rank them semantically?

More generally, do you really think humans randomly come up with new ideas, in any act of creation, such as art, science, etc.? Why do you think art studies and stuff like Oblique Strategies exist? How long should one wait for a specific random human without any formal education in physics to have 50% chance of coming up with a new workable theory of cosmology?

As you say, putting a random sequence of plausible-sounding words together is a trivial task - even a non-LLM algorithm can do it. No, I don't think humans are unique - they also absorb the ideas available to them and re-elaborate on them to produce novel ones.

An LLM might come up with a sequence of words that express a novel idea/thinking pattern, but so can I on the spot right now. The question, though, is: is that idea any good? In the case of humans, maybe the first idea is good, the second ok-ish, but inevitably they get duller.

Most people come up with very few novel (good) ideas in their lifetime. The question, then, is whether a model can, unlike humans, *repeatedly* generate new ideas, continuously.

My guess/assumption is that NO, a model might come up with SOME ideas that are somewhat original [1], mixing the existing sources available to it (similarly to how a human would do it, just with more sources and more efficiently), but it cannot continuously generate novel ideas.

If that's true, I think there will be a space for humans to be thinkers, even if models are super smart, as a different unique perspective and a different reconfiguration of previous ideas will be more unique than raw intelligence.

Please note: I wasn't refusing the notion that LLMs don't produce anything original, just noting some fallacy in your example about word generation. The jury is still out on whether AI models will own thinking in the future - or will simply be partners to humans - and I don't pretend to have the answer.

[1] More should be said about what makes an idea original (even if derived from others), as opposed to a transposition of an existing concept. I admit I'm not a mathematician or anywhere near an expert on the topic, but from what I've read, the latest AI math achievements seem to belong to the latter case.

As Stalin said, quantity has a quality all it's own.

11000 agents explored one of the recent math problems solved by AI (can't remember exactly which one). About 3 of them had crucial insights which moved the problem forward. A 4th one also had one of the key ideas, but it's session got messed up and the idea was lost, and only noticed in retrospect.

Maybe 11000 top mathematicians working on it would have found the same ideas. But certainly not in days.

Even if LLMs generate novel ideas at 0.01 rate of humans, you can deploy vast amounts of them on even trivial problems, that humans would not bother with.

For biological reasons, human intelligence is kind of fixed and only very slowly advances by better compression and synthesis of the school curriculum.

There is no clear present bound on machine intelligence. One might argue that machines won't be able to get smarter than the material they were trained on (human output). I kind of doubt that. And certainly they can be much faster.

Somebody whipped up a website with GPT2 that does that and it was discussed on Hacker News at the time: https://news.ycombinator.com/item?id=23169962 I don't think anyone is going to argue GPT2 is doing thinking of any kind, let alone novel.

You think the thinking of human brains isn't based on existing sources? Unless you believe in dualism and the divine spark of inspiration, it is.

It is based on existing sources, but doesn’t mean that we produce the same output. There was a paper here in HN a couple of months back on how LMs are good in the deductive and inductive thinking processes, but not the abductive. So if that’s (and stays) true, humans will have a place in the job market after all. Time will tell.

This is kind of an aside to the topic, but your comment made me think that one of the advantages of humans is just that there are currently 8 billion different neural nets trained on different experiences with different model configurations running different architectures on different hardware.

yes! I do think that too - and hope I'm (we're) right.

Reply on news.ycombinator.com