Hacker News Reader

Comment by logicchains | Hacker News Reader

We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been mathematically proven. LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. Eventually we'll have LLMs trained on visual and audio input too (and be trained to predict what happens next), so they can receive an arbitrarily large amount of training samples. If the models keep growing and being trained on more and more data, we'd expect them to become at least as intelligent (good at modelling the world) as humans.

View on HN

This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous.

Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.

You have to be a lot clearer about what you mean by continuous here. An LLM technically does not produce a continuous function on the reals because its inputs are floating point numbers with finite precision. Any function on discrete inputs like this has an extension to the reals which is continuous, just imagine joining all the discrete points up with lines.

So then your claim wouldn't be about the limits of LLMs themselves, but on the limits of systems that do not take continuous inputs. The question then is do you think that humans take in continuous input?? Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.

It is quantum which isn’t exactly discrete. You receive one photon or two, not 1.5, sure, but the energy of that photon, its frequency is a real. At least I am not aware of a quantum mechanics over Q. I think the math would be hard because you loose all the convergence properties. Maybe there is some Hilbert space over computable reals, Google is not finding it but I no longer find that to be indicative of anything.

The whitepaper operates on mathematical numbers, though. They didn’t calculate it with floating points of n-bits.

Given a fixed level of precision for input and output, and a function on this discrete space, we can construct a continuous extension of this function to the reals. Now using the paper we know that there is a neural network with continuous weights that approximates this continuous extension to arbitrary precision. If we imagine rounding the continuous input and output to the fixed precision specified, then because of continuity of the neural network, and the fact there is a finite number of weights, we can choose a tolerance by which we can change each of the weights such that the output does not change by more than the precision of the output value. Thus we can pick a level of precision for the weights where both the weights and inputs and outputs are discrete.

I mean continuous in the calculus sense, as in it doesn't have discontinuities, not as in continuous VS discrete.

> Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.

This is a misunderstanding of quantum mechanics. Only certain specific quantities come in discrete quanta (spin, charge, certain energy levels, etc). Other physical quantities are very much continuous - notably time and space. In fact, much of the mathematics of QM is not discretizable, it won't work if you try to make time or space discrete.

But continuous depends on the topology of the space you are working over. If the topology is discrete any function is continuous. I've found some papers that argue that simply from an error corrections standpoint the signals in the brain need to be discrete.

This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function

Incorrect. That's what nonlinear activation functions are for.

Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.

There is a leap from "predicting the next word" to "modelling the world" that you are sort of glossing over here.

The idea that LLM's are actually "reasoning" rather than just performing a probability function seems a little bit...pseudo-religious. Like we've created a new life form.

> LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it.

I don't see any evidence that this is true or proven in any way.

Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.

It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction.

[0]https://thegradient.pub/othello/

Predictive models need not at all reflect what actually happens. Sometimes embracing less accurate models is actually need to enhance understanding. Epicycles in the geocentric model could be made very accurate as far as observations in the sky are concerned, but it wasn't a good world model.

That whitepaper operates with mathematical, infinite precision numbers.

It’s just a categorical error to read too much into it - it’s not really a too interesting property to be able to get closer to something forever, without some sane growth function.

Reply on news.ycombinator.com