This has been addressed several times now in the peer comments to this one. I am addressing the original formulation of the problem because the article is about the original formulation. There are other options available now, including causal masking, but in the original formulation positional encodings are THE most important single ingredient in success, because many aspects of semantics cannot be recovered from anything except position. One example is the subject in an English sentence.