I found this documentation from openai that supports my hunch: https://platform.openai.com/docs/guides/reasoning/advice-on-...
The reasoning tokens from each step are lost. And there is no indication that they are different tokens than regular tokens.