LLMs can't really spell as the smallest "block" of information they operate on are tokenized whole words. Same issue with e.g. arithmetic.
> [INST] Spell out the following word letter by letter: margorczynski [/INST] m - a - r - g - o - r - c - z - y - n - s - k - i
> So, the word "margorczynski" spelled out letter by letter is: m-a-r-g-o-r-c-z-y-n-s-k-i.
The text between `[INST]` and `[/INST]` is the input. The text after `[/INST]` is the output.
https://twitter.com/karpathy/status/1657949234535211009
I'm not arguing that you can't use single chars just that many of the issues parent discussed are caused by this.