Singular Value Decompositions of Transformer Matrices Are Highly Interpretable
colab.research.google.com
colab.research.google.com
"Please go to the colab for interactive viewing and playing with the phenomena. For space reasons, not all results included in the colab are included here so please visit the colab for the full story."
[0] https://www.lesswrong.com/posts/mkbGjzxD8d8XqKHzA/the-singul...