LLMs "think" with tokens - it's the only unit of state they know, so it's really more of an accident of having larger models with more context.