Visualizing matrix multiplication as a linear combination | Hacker News Reader

Visualizing matrix multiplication as a linear combination

eli.thegreenplace.net

77 pointssigna1126 comments

Here's how you can apply this interpretation of matrix multiplication:

Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system.

Carrying this forward, matrix multiplication A * B gives the combined representation of two coordinate transformations.

Another way to look at it: A matrix is a linear transformation, and multiplying a vector by a matrix is how you apply the transformation. But linear transformations are really just changes of basis. How do you change your basis? You find the dot product of a vector with each new basis vector. And that's exactly what matrix multiplication is. When you multiply your column vector by a row in the matrix, you're finding the dot product, doing the projection in your change of basis.

Hmm. As someone who has never understood/visualized/been taught matrix multiplication in any other way than how it's shown on the linked page, could someone explain the alternative? I.e., if this is new, what mental model do you currently have? Presumably that one would be new and useful for me.

Edit: "Typically this visualization isn't taught" -- that's what I"m asking. What visualization were you taught, if not this one? I can't think of any other that makes any sense.

I think this page is meant for someone who has no mental model of matrix multiplication at all and just applies the formula blindly.

I was taught to go element-by-element: element c_ij of the product = the linear combination a_ik*b_kj. What's presented here is the "same" but represents a higher level of abstraction, so that you think in terms of linear combinations of rows or columns, rather than linear combinations of scalars.

Typically, this visualization isn't taught, in my experience. What's taught is the formula for computing each cell of the result matrix (cell i,j being the dot product of row i of the first matrix with column j of the second). While this is, of course correct, and also the most efficient way to compute the multiplication manually, it's not always clear why the formula is correct.

Hopefully the visualization on the linked page makes this formula's origin clear.

Really? Interesting. We spent some time discussing multiplication by block decompositions in my 100-level linear algebra class -- is this really uncommon?

You usually think of the product of a matrix with a vector as the matrix acting on the vector, i.e. every row of the matrix tells you what the corresponding component of the resulting vector is. This turns the thing on its head and describes the result as the vector acting on the matrix. Because you usually use matrices to describe operations, e.g. transformations like translations or rotations in (computer) geometry or observables in quantum physics, it seem more natural to think of the matrix as acting on the vector, i.e. a point of the geometry or a physical state, not the other way round.

Yes, but if you think of the matrix as a collection of basis vectors, then something like a rotation in R3 becomes crystal clear. Your using the same coordinate but moving in arbitrary (orthogonal) directions rather than just right,up,forward. This makes more sense using scalar*vector operations than using dot products.

I was taught by the an other visualization[1]. I thought it was the "standard" method since many textbooks (in Hungary at least) teach it this way.

[1] https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_m...

Unless I'm missing something, this is precisely the kind of visualization that explains how to compute the result, but not why the computation makes sense. I.e. why is matrix multiplication defined this way at all.

> What visualization were you taught, if not this one?

Oftentimes, no visualization or other motiviation is taught at all. Students are simply instructed the steps to follow to produce a correct result.

Matrices represent maps (functions), and matrix multiplication is map (function) composition. This is very useful because you can easily see how linear algebra generalises to functional analysis and you can easily get an intuitive feel for many theorems from functional analysis if you convince yourself that infinite-dimensional bases are not that special.

One alternative that I find interesting (not the one I was taught first though) is to visualize matrix multiplication as the sum of the rank 1 matrices obtained by multiplying each column of the left matrix by the corresponding row of the right one.

If you want more, I can definitely recommend the video lectures from MIT's linear algebra course by Prof Gilbert Strang:

http://web.mit.edu/18.06/www/videos.shtml

I concur on these videos, everybody who teaches the subject should watch him. You get a lot of "aha! it's that simple?" moments watching him.

+1 for the Strang videos. He's an amazing lecturer. e.g. in Lecture 1 explains very clearly matrix multiplication in the "column picture" and the "row picture". More intuition than you can shake a stick at: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

This tutorial is also pretty good for geometrical intuition: http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf

This is the way I "understood" linear algebra. I even encoded it here in a small c++ header file when I used to think it wise to implement these things yourself:

https://github.com/arithma/mat44/blob/master/mat44.h

Used here: https://github.com/arithma/ios3dball

> when I used to think it wise to implement these things yourself

It's a very good thing that you thought it wise at some point. There's nothing like reinventing a few fundamental wheels for a better understanding of how they work.

Sure, in the matrix product

AB = C,

C has the same number of rows as A and the same number of columns as B, and each column j of C is from column j of B acting as coefficients in a linear combination of all the columns of A.

Next: For the set of real numbers R, positive integers m and n, m by n real matrix A, real numbers a and b, and n by 1 x and y, we have that

A(ax + by) = (aA)x + (bA)y

so that A is linear.

If we regard x and y as vectors in the n-dimensional real vector space R^n, then we have that A is a function

A: R^n --> R^m

and a linear operator which can be good to know.

Then the subset of R^n

K = { x | x in R^n and Ax = 0 }

(where 0 is the m by 1 matrix of all zeros) is important to understand. E.g., K = [0] if and only if function A is 1-1. If m = n, then A has an inverse A^(-1) if and only if A is 1-1.

With C = AB, the matrix product is the same as function composition so that

Cx = (AB)x = A(Bx)

which also can be good to know and, of course, uses just the associative law of matrix multiplication.

That matrix multiplication is associative is a biggie -- sometimes says some big things, e.g., is the core of duality theory in linear programming.

And the situation is entirely similar for the complex numbers C in place of the real numbers R.

I don't know why but it seems to make more sense to me if I think of the multiplication in terms of rotating or transposing and moving the vector or matrix so that the columns or rows line up with the ones they are to be multiplied with, the coloured guides are entirely useless to me.

There are 2 types of matrix multiplications which should not be confused with each other.

The first type is a change of basis where the columns of the matrix are the old basis represented in the new basis.

The second type is a linear function that takes in a vector and outputs another vector.

What is the point of this blog post? Matrix multiplication is shown like this, and in many other similar ways, in many introductory linear algebra textbooks.

This article was posted a few weeks ago, helped me understand matrix operations I've been trying to wrap my head around for years: http://betterexplained.com/articles/linear-algebra-guide/

I'm actually studying linear algebra right now, and this site has given me insights my text book (which is actually pretty good!) just hasn't. Thanks for the link :-)

Yeah, but this is so boring and oh-so-run-of-the-mill.

(If the mafiosi could read the following paper before rushing to click down-vote, then I'd be much obliged. http://www.math.rutgers.edu/~zeilberg/mamarimY/DM85.pdf)

Reply on news.ycombinator.com