Hacker News Reader

> I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model.

But it doesn't! The distribution doesn't change at all. The only thing that changes is that sampling of that distribution becomes deterministic as per a precomputed seed.

View on HN

You could describe that as taking an input distribution and a sampling procedure and producing an output distribution. This is a difference in sampling procedure that produces a deviation in the output distribution.

(If you don't like calling it a distribution when it's at 100% for the chosen token and 0% for all others, then look at it as an output distribution across all possible prompt inputs, or perhaps just the cluster of prompts that achieve whatever you're trying to accomplish.)

Reply on news.ycombinator.com