Hacker News Reader

Comment by fc417fc802 | Hacker News Reader

Of course we don't say that. You can't ask the (traditionally) programmed computer a freeform question and get a sensible answer back. We tried that for going on 50 years and it never really worked. (The highest achievement that comes to mind is answering jeopardy questions.)

You can very carefully construct a query in a dedicated language, debug that query, and get useful results back. But that's clearly just a human using a tool, not a machine exhibiting understanding or general knowledge.

Meanwhile you can ask a multi-billion parameter LLM a freeform question in ~any human language and it can produce a coherent and meaningful response. It can one shot pieces of code. Track down bugs based on compiler error messages. It might not (yet) be human level in many cases but to get hung up on that is to miss the point.

View on HN

>multi-billion parameter LLM

This is equivalent to an `if` statement with multi-billion levels of nesting. It is just a "traditional program", just unimaginably huge.

Just because it is not "traditionally programmed" does not mean that it is not a really huge "traditional program".

Scaling something by many order of magnitude does not put it in a different category. A computer program, no matter how big, is still a computer program.

No, it's not equivalent to nested if statements. If you can mathematically demonstrate that it is I would be interested.

Anyway that's irrelevant. The point is that we use different language when referring to the one because its capabilities appear to be fundamentally different.

Your argument comes down to a claim of human exceptionalism - that a computer program can never "understand" simply by virtue of being a computer program. You haven't actually provided any defense of that claim though. You've just assumed it without justification.

>No, it's not equivalent to nested if statements.

It is. If you control the randomness involved, the output of a model is completely deterministic. Which means that it can be represented by a huge lookup table.

Anything that can be represented by a lookup table can be expressed as an `if then else` statement.

By that logic sin(x) is equivalent to a lookup table. Yeah, you can approximate it that way. But doing so at any reasonable level of precision will quickly become an exercise in the absurd. Neural networks are far worse, consisting of stacks of massive linear combinations fed into nonlinear functions.

Regardless, it remains irrelevant to the subject at hand. You're going off on a tangent rather than admit your initial claim was wrong.

>By that logic sin(x) is equivalent to a lookup table

NO!

`sin(x)` is continuous, so the domain is infinite.

But an LLM model is not a continuous function, and thus the domain of a LLM model is finite (set of all possible tokens). So using a lookup table for a model behavior would be exact and not an approximation. So it can indeed be represented by an if statement of finite size.

Hence proved!

If you don't understand something in what I wrote, I can clarify if you tell me where you have trouble following.

That’s not true in practice thought https://sulbhajain.medium.com/why-llms-arent-truly-determini...

that is why I said

>If you control the randomness involved

Which you essentially cannot do

It is inherently randomized

But isn't the link shared by that comment doing exactly that

https://sulbhajain.medium.com/why-llms-arent-truly-determini...

>The Thinking Machines research team showed it’s possible to fix this. They built batch-invariant kernels for RMSNorm, matrix multiplication, and attention, integrating them into the open-source inference engine vLLM.

>The outcome: 1,000 identical prompts, 1,000 identical outputs. Perfect reproducibility.

Even if it were true he would still be wrong. I wish HN had some mechanic to evict these sorts of troll accounts that attempt to score rhetorical points rather than honestly learning about a subject. The usual vote and flag mechanics don't work here because there's no singular and overt violation of the guidelines taking place.

Reply on news.ycombinator.com