Yeah that is one draw back they mention in the paper. So much of the full test stack is trained on and dependent on English.

A few months ago I went down this same rabbit hole trying to determine if using Chinese has any inherent advantages due to the way more context is encoded into Chinese characters versus what you find in English.

The total token size may not take into account any benefits gained by Chinese characters potentially encoding more context.

I have been meaning to see if any Chinese native tokenizers have been created to account for this feature of the language.

Unfortunately, many Chinese and Japanese characters can have many different meanings depending on how they are used. As a result they do not break down as well to be tokenized as a language like English.

Maybe some kind of hybrid intermediate language would be most optimal.