> [which anyone who understands] how a multi headed attention transformer works [would know is impossible]

How is that productive? The author already claimed in their changes that they aren't an ML expert and asked for advice.

It could be written like

multi-headed attention transformers frobnicate all the weights in their QKV matmuls. <supposition ... I would look at IO rates during inference ...>

You have an opportunity to teach rather than admonish.