Hacker News Reader

Comment by epistasis | Hacker News Reader

I'm not sure that's a fair summary of the situation though. If the machines understand math but humans don't, we're going to be in a bad bad bad spot.

We need humans to help guide all this, need to learn all we can.

I agree with Sanderson that the opportunity seems larger than in the past. The challenges are quite different, but if somebody loves math, loves learning about it, like I do, I see so much opportunity.

There's going to be so many more connections to evaluate, so much work needed for unification and simplification and pedagogy, and above all there's going to be intense demand for mathematical taste.

View on HN

> If the machines understand math but humans don't, we're going to be in a bad bad bad spot.

With potentially plummeting value for human involvement in these skills, I think that’s precisely what many people are worried about.

Well that's the point, there's immense value for human involvement in these skills. More now than in the past. If you can do more with math, additional funding avenues open up.

Jane Street will be using tons of AI in math, but boy are they going to need a lot of humans to try to keep it all under control.

And we're all just supposed to suck up the fact that our chosen career morphed into "Babysit a machine" no matter what it is? What a swell timeline

It's exactly the opposite of babysitting, from my point of view.

No, they won’t.

The entire point of AI is to eliminate human involvement.

> If the machines understand math but humans don't, we're going to be in a bad bad bad spot.

To generalize beyond math... how many bits of working memory do you have? How about a transformer?

I'm not sure I understand the point of your question. Of course any recent LLM has memorized far more than I or another human could.

That is the point.

So what exactly are you going to offer beyond what near future LLMs will be able to do?

Agency and control, the most important. Even if LLMs could somehow assimilate all the knowledge that's not available in their training datasets that's essential to all aspects of our current economy, which they can't they don't know human desires.

Their training data contains nothing about human desires?

Even with the entire knowledge of human desires available on the training data, one must still construct a prompt.

Desires which were trained on are only partially relevant to what I need right now.

> Even with the entire knowledge of human desires available on the training data, one must still construct a prompt.

AI's are already constructing prompts.

Prompts to do what and meet what human desire?

Whatever the AI does, humans are still here, and "math" just without humans controlling the direction will have limited utility.

Unless we are talking about turning all control of our society and economy to AI, which I have assumed is not even on the table because that's the one way we face existential risk.

Human desires cannot be generalized. We're individuals.

Memorization is long-term memory. Working memory (WM) is short term memory.

How much you can hold in your head and work on, at a given moment.

WM is not the same thing as IQ, but IQ and WM are both necessary parts of effective intelligence.

For a transformer, the WM number has several more zeros at the end of it than it does for a human.

It's not a perfect comparison of course, because their "brains" are very different. (And it's not the same thing as the context window size, because of sparse attention.)

(It's not "guy shaped" but it does a reasonable job of emulating a guy.)

But fundamentally, their brains have the opposite shape of ours. Our brain is drinking through a tiny straw. The LLM drinks through a firehose.

Reply on news.ycombinator.com