All that happens in reinforcement learning and everything from pre-training to generation, including the last token, at the end is about adjusting the probability of the next token given the sequence.

Anyone who thinks reinforcement learning changes how the generation works hasn’t worked on inference engines and you can dismiss them as inexperienced and misguided.