nanoGPT is a way to embed low-dimensional embeddings into high-dimensional embeddings, not self-attention.