Google for Goto's "Anatomy of high-performance matrix multiplication", one of my favorite programming texts.
Also the papers underlying the development of the "Elemental" library for distributed dense linear algebra is worth a look.
Google for Goto's "Anatomy of high-performance matrix multiplication", one of my favorite programming texts.
Also the papers underlying the development of the "Elemental" library for distributed dense linear algebra is worth a look.
[1] https://www.cs.utexas.edu/users/pingali/CS378/2008sp/papers/...
Don't invert matrices.
For example, the solution for the least squares (=linear regression) problem is
beta^hat = (X'X)^-1 X'y
and you might think that the best way to compute it is to compute X'X, invert it, do the multiplication, yada yada yada. But it's really better to just ask your library (Matlab, LAPACK, ...) to solve the problem for you, and it'll probably do a QR decomposition. For small problems, it doesn't really matter, but for big problems you'll be both faster and more accurate.You really just want to use LAPACK/BLAS, no? (That's what the Neanderthal library mentioned in the article does, btw, and basically linear algebra libraries for other languages, too. If not, you probably shouldn't use it...)
For instance I have written high performance spherical harmonic transforms, and these techniques apply then.