To be clear these tokenizers use byte-pair encoding (subword tokens) so an individual token index typically corresponds to a piece of a word; this index does not depend on any intermediate decoding of the byte stream as long as the start of the stream is a start of your input. The decoding always works left to right and always starts at the start of the stream. You could write a tokenizer that uses plain bytes and one that uses unicode code points if your tokenizer was trained on unicode and forced to keep unicode codes together (almost all are), and the results would be identical for all practical purposes.