Hacker News Reader

Comment by guappa | Hacker News Reader

You can't regenerate the book from the ISBN. But you can generate the text from the prompt.

You can go and fetch the book from a book store using the information. Fundamentally there's not much difference between that and "fetching" the output from some model using the matching prompt. In both cases there some kind of static store of latent information that can be accessed unambiguously using a (usually) shorter input.

I'm not saying the value of the returned information is equivalent, of course. But being "just a pointer" into a larger store isn't, in itself, the problem to me.

You realise that you can't fetch a new isbn without altering the archive, while this is not the case for every new prompt that you come up with?

I don't understand the distinction. If the book archive is electronic, like many in fact are, why can you not get a copy of the book with a given ISBN without altering anything? Even if it's not electronic, does the acquisition of a book by an individual meaningfully change the overall disposition of available information? If you took the last one in your local Waterstones, I can still get one elsewhere.

> If the book archive is electronic, like many in fact are, why can you not get a copy of the book with a given ISBN without altering anything?

Because new books are written?

It feels to me that you are set on insisting that a prompt and an ISBN are the same, and no amount of logic will move you from there.

Models can be trained more and fine tuned, though, if we're going to stick to the analogy. But in the context of the analogy, the LLM won't be materially updated between two prompts in roughly the way that telling you that the answer you seek is in a book with a specific ISBN isn't materially affected by someone publishing a new book at that moment.

You are quite right that you're not convincing me of your original thesis that that a prompt contains the entire content of the reply in a way that some other reference to an entity in some other pool of information to doesn't. That's not the same as saying "ISBNs and LLM prompts are the same thing", which is a strawman. It's saying that they're both unambiguous (assuming determininism) pointers to information.

Of course no-one is disagreeing that a reply from a deterministic LLM would add no information to the global system (you, an LLM's model, a prompt) than just the prompt would. But I still think the same is true for the content of a book not adding to the system of (you, a book store, an ISBN).

In fact, since random numbers don't contain new information if you know the distribution, one can even extend it to non-deterministic LLMs: the reply still adds no information to the system. The analogy would then be that the book store gives you at random a book from the same Dewey code as the ISBN you asked for. Which still doesn't increase the information in the system.

Can you, though? I thought LLMs just by virtue of how they work, are non-deterministic. Let alone if new data is added to the LLM, further retraining happens, etc.

Is it possible to get the same output, 1:1, from the same prompt, reliably?

They are assuming a lot of things, like the LLM doesn't change, and that you have full control over the randomness . This might be possible if you are running the LLM locally.

Well if the llm change, why not assume the index system doesn't change too?

And yeah I guess if you control the seed an llm would be deterministic.

Reply on news.ycombinator.com