Hacker News Reader

This adds connections between the parameters of q, k, and v whereas the original doesn't, unless my very tired brain is missing something.

View on HN

Nope, they all depend on x and the same is true in this scenario

It is actually really common practice. It is a single linear layer so there's no connection intranodes. The reason to do this is because it is a bit less computationally intensive.

tldr: linear layers have an associative property

Reply on news.ycombinator.com