My understanding is limited, but I'm guessing it's to do with the algebraic properties of the embedding vectors. I'm familiar with embeddings that you can add and subtract, which may reveal concepts existing as linear directions within the embedding space.

Here, they're talking about multiplying, dividing and permuting vectors. Multiplying combines concepts, adding creates a superposition.

They also mention randomly selecting embeddings for the concepts in mind. So my guess is that instead of one-hot encoding the classifier, they instead use random encodings on the output, and are working to give those encodings desirable properties.

I would also hazard a guess that the random vectors they choose are close to zero in most components.