AFAIK, positional encoding is not that important for the performance of causal transformers (e.g. all important modern LLMs). The causal masking itself supplies necessary information implicitly.
> because the Vaswani et al transformer (as it is described here) simply does not work until you add the most important ingredient of all: the positional encoding.
This is factually correct. The Viswani et al transformer DOES NOT WORK without positional encodings. A transformer architecture that involves (e.g.) causal masking is a different architecture. If you want to separately discuss one of those architectures I am happy to do that too.