Hacker News Reader

Only in the sense that anything is a markov chain with some sufficiently large state size. It tells you nothing about the behavior.

View on HN

That's only true for Markov chains with an (effectively) infinite state space. If the state space is finite, such as with an LLM with a finite set of tokens and a finite context length, it's trivial to tell the difference between the behavior of a specific Markov chain and a more sophisticated model. Once you have a specific Markov chain, the state space is fixed, and you can just ask something that requires a larger state space.

The same basic idea can be found everywhere in mathematics and CS. Once you have chosen the value for your parameter, I can choose the value for my parameter to guarantee the desired outcome.

Not anything is a markov chain. Markov chains are distinguished by probabalistic transitions between states, just like llms

LLM transitions are pseudoprobabilistic.

No, LLM just returns a distribution of states, you can use a real random function to make the transition if you want it has nothing to do with the LLM itself, the pseudoprobability function is just an optimization since you need hardware to get really random numbers.

Reply on news.ycombinator.com