Gilbert Strang Teaches Linear Algebra
ocw.mit.edu
ocw.mit.edu
I developed a strong dislike for LA, if not outright fear and hatred. This from someone who majored in math at university. I later learned from reading Dr Oakley's book, _A Mind For Numbers_, that this is a perfect recipe for not mastering a subject. I was sabotaging my own learning by approaching it from the point of view of something to be despised. It was many years later after reading Oakley's book that I challenged myself to learn Linear Algebra.
I found the Coursera course Coding the Matrix https://codingthematrix.com/ perfect. Not only did Professor Philip Klein explain the subject well, but it was also a programming course in python. Coding the algorithms helped make them that much more concrete. I'm happy to say I finished the online course with 100% grade.
Unfortunately Coding the Matrix is no longer available on Coursera.
I know this thread is more about Strang being a teaching god, and he is. I wanted to add something on alternative ways to learn linear algebra.
Actually I now think it might also be a cultural thing. If you take a look at the approaches the English Wikipedia takes vs. the German Wikipedia take explaining a vector space you'll know what I mean:
https://en.wikipedia.org/wiki/Vector_space#Introduction_and_...
The book itself is still for sale, and the website has course slides and auto-grading software for some of the exercises: https://codingthematrix.com/
I had to build a somewhat basic SVG editor in React recently and the other devs on the team had to leave a lot of the core to me because I was the only one that remembered enough matrix math to do it.
If you look at how people solve differential equations in practice, it’s all matrix algebra.
Linear algebra is also pervasive throughout pure mathematics.
Edit: I think everyone should also learn calculus though, and differential equations are the heart of calculus. Here’s an introductory calculus book http://www.math.smith.edu/~callahan/intromine.html and here’s an introductory linear algebra book https://web.stanford.edu/~boyd/vmls/ which try to get students closer to the way these tools get used in practice.
http://aix1.uottawa.ca/~jkhoury/app.htm
https://math.stackexchange.com/questions/256682/why-study-li...
Computer software/engineering is the only field of science that can use naive linear algebra directly to solve hard problems in the field.
That's so cool that you had access to a linear algebra class in HS!
I can't express strongly enough how appreciative I am to MIT, and also schools like Stanford and Harvard, and Prof. Sedgewick from Princeton, for making courses like this publicly available.
I don't know about others, but in my case I had to put up with a lot of BS, and "older" colleagues that were envious of my "performance" (annual feedback reviews by students). These few emails (and two letters!!) went a long way for me.
But they kept coming up with excuses about why they couldn't pay me as much as them so I left. Boom. I encourage more professors to leave and more students to check out venues like Coursera and Udemy.
It's what I should have just done straightaway -- dad's an engineer, his dad was an engineer, half his brothers are engineers -- but I didn't for a variety of personal reasons I won't get into here. I was always fascinated by computers and by prog langs in particular -- a deep curiosity about how text could turn into action, although I wouldn't have framed it like that as a kid. But I was the sort of dork who would check out books on C++ as a 10 year old despite not having access to a compiler or any idea what I was doing. This fascination kind of took a backseat until I was out of law school, working, and slowly realizing how miserable I was at the thought of 40+ more years of what I was doing.
So I took a long and circuitous route to get here, but now here I am, and eternally grateful for these sort of resources. Without them, the switch would have been way more difficult. It already took me 3 years of dedicated work; I can't imagine what I would have done without this stuff freely available. Probably would have had to pay to go back to school.
https://ocw.mit.edu/courses/mathematics/18-065-matrix-method...
And this new book:
https://www.amazon.com/Linear-Algebra-Learning-Gilbert-Stran...
Packed with insight that would otherwise take years to pick up piecemeal elsewhere.
His teaching capability is so great, we ended up buying his textbook and using that instead of the one assigned by her college. My wife ended up with the best grades in the class, but not only that, ended up actually learning the material and retaining it many years later. Her field is in no way related to mathematics, but it would not be an exaggeration to say that she benefited greatly from her maths exposure.
Linear algebra, heck yeah! Thank you, Dr. Strang.
He does more with some chalk and a big black board than any modern courses using fancy presentation technology.
[1]we followed Huffman-Kunze https://www.amazon.com/Linear-Algebra-Kunze-Hoffman/dp/93325...
For example, if there's a certain probability of people with cancer getting a positive test result, what's the probability of a positive test result meaning you have cancer? Too many people absolutely cannot approach that problem in any intelligent fashion.
(Side note: More classes should accept "there isn't enough information to give a coherent answer" as a correct result.)
It's a matter of focus: The point isn't to derive statistics, any more than the point of a first semester course on differential calculus is to derive the real numbers and the infinitesimals. Calculus isn't useless, it's just not as useful unless you go into specific fields.
If you don’t know calculus, someone has to tell you the formulas, which will seem like mysterious completely arbitrary magic. Memorizing them will be a pain in the butt. When you try to use statistics you will make small mistakes which will lead to wrong or even entirely incoherent results, but you won’t notice because you’re just following a recipe you don’t understand.
Of course, to do real statistics you need linear algebra as well.
There are also tons of very complicated advanced calculus and differential equations concepts that have direct "discrete" analogues in linear algebra, and this become glaringly obvious when trying to say discretize and solve a partial differential equation.
I had an HP 48G calculator at the time that could do symbolic derivation and all of the matrix decompositions. And if it didn't have a function, it was easy to write your own in any of four (4) programming languages: symbolic user programs (User-RPL), their compiled language (System-RPL), C or assembly. The matrix editor looked exactly like a spreadsheet. What's cool about HP's Erable (CAS) is, just short of being LISP/Scheme, everything was a value. Compiled binaries could be pushed on the stack (RPN). Oh and programs could be copied to other HP 48G/GX's with it's incredibly-powerful serial IR port. Some ingenious soul figured out how to turn the calculator into a learning IR remote control with a library of preset manufacturer codes. (I once turned on all the TVs in the lecture hall from the back of the class (15 m away) with my calculator during lecture... and people freaked out. :D) People modded them with 2-8x as much memory by chip stacking and wiring the enable pin and an inverter to the processor. None of this 64KiB rubbish, 512KiB FTW. And if you couldn't see parens in your 3 line equation, there's a graphical equation editor. Finally, if you got bored of that, there's always the easter eggs with the team's name in a solved crossword puzzle and grayscale pictures of them (not like Tetris on an Agilent oscilloscope, but okay).
I preferred his competitor's book because it made sense to me, Anton's Elementary Linear Algebra, Applications Version
His book is Introduction to Linear Algebra.
I watched every lecture and did most of the homeworks a couple of summers ago in preparation for taking grad ML courses at Carnegie Mellon (10-601 Intro and 10-605 Scalable ML). The foundations that I built from Strang's course were essential to my success later.
For more linear algebra goodness, check out the 3Blue1Brown playlist, which provides a lot of the important geometric intuition: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
Another good book with visuals and manipulatives is http://immersivemath.com/ila/index.html
And this is a short tutorial that I wrote that summarizes some of the main ideas of LA in just four pages: https://minireference.com/static/tutorials/linear_algebra_in... You can print this and enjoy with your weekend coffee tomorrow. For bonus points, you can also learn how to do LA using SymPy here https://github.com/minireference/sympytut_notebooks/blob/mas...
It's also a topic that most people are unlikely to pick up on their own, even if they are somewhat motivated. Working through the exercises in a math textbook doesn't give most people the "feeling of accomplishment" feedback loop that e.g. playing with a new web dev framework does.
For those reasons, I would highly recommend taking it in college, because the structure and commitment mechanism of a college course will help you get yourself through the material.
One major caveat: In my opinion, courses that only cover pencil & paper "theory" exercises are much worse than courses that have a combination of pencil & paper work and problem sets that require the use of software tools like Matlab, Octave, or Python/NumPy. In the real world, you're always going to use software tools for the actual number crunching, so it's extremely valuable to get familiar with actually using them. It can also help you on problem sets in your other courses - using Matlab or Octave on your laptop is much faster and more pleasant than keying numbers into the matrix interface on a TI-84!
If you want to work on adtech, CRUD apps, or uber-but-for-catsitters startups, probably not. I have been working on two out of the three for a decade, and in all this time, I haven't had to use a lick of linear algebra.
From an academic standpoint, it's arguably a more useful course for your general education than calculus (Which is a requirement for just about any degree), but less useful than statistics.
[1] If you ever intend to work for a quant firm, you should probably take a lot more math courses than just linear algebra.
The overwhelming majority of adtech is like any other enterprise business - building CRUD apps.
In comparison, I'd say Strang's class is more formal and feels more like work, and those videos will be more fun while fullfilling the same purpose, to give you a lasting understanding of what linear algebra is and where it might come in handy.
Any other suggestions from the peanut gallery?
Page 3 of that paper lays out matrix multiplication (e.g.: applying a "transformation matrix" in the spatial parlance of 3blue1brown's videos) as a traversal of a directed graph.
This isn't super readable (it's from my notes to self), but my intuition is:
Matrix multiplication is a graph traversal:
* Each i,j in transform matrix (on left) pulls N (the value at that index) times the j-th value in the input vec (on right) into the i-th output vec index
* This is equivalent to an N weighted vertex from the j-th node to i-th node, so i takes N of j's previous value
* The input (right side) is a matrix? It's just multiple column vectors next to each other, repeat the above process and make a new output vector for each iteration
Just wanted to say thanks for this paper. It really does help with intuition and it's not something I would've expected to get through.
https://www.youtube.com/watch?v=YiqIkSHSmyc&list=PLUl4u3cNGP...
Most higher-level math will be easier for you if you have familiarity with vector calculus and linear algebra, so if you're the kind of person who benefits from a structured class when learning new/weird ideas, it's worth studying in school. (I'd also generally recommend a course in probability and statistics)
Now imagine the inputs are vectors and the outputs are vectors too, in other words we would need vector-functions. Many problems in math, science, computing, economics, biology — you name it — are of this multidimensional nature, so the more you know about vector functions the better models you can build.
As you can imagine describing arbitrary input-output relationships between high-dimensional spaces is quite complicated. Think of all the possible ways that a vector input vec{x} = [x1,x2,...xn] can affect the coefficients of a vector output vec(y) = [y1,y2,...ym]. If you allow arbitrary functions like quadratics, sine, cosine, exp, etc. there would be way too many different ways to map vec(x) to vec(y). Too many ways...
This is where the "linear" part of linear algebra comes in. Instead of arbitrary input output transformations vec(x) --> vec(y), what if we restrict oursleves only to relationships where the output yi is some linear combination of the inputs x1 x2 .. xn, where linear combination means
y1 = m11*x1 + m12*x2 + ... + m1n*xn
in other words, take the outputs are equal to the inputs times some proportionality coefficients mij.
The other output coefficients y2...ym are computed in the same way with different coefficients y2 = m21*x1 + m22*x2 + ... + m2n*xn
...
ym = mm1*x1 + mm2*x2 + ... + mmn*xn
thus the overall map vec(x) --> vec(y) has a total of m*n coefficients mij.This map between inputs vec(x) and outputs vec(y) is the vector-equivalent of the single-variable function y=mx (proportionality relationship, or line with slope m passing through the origin if your prefer to think of it geometrically). We call these maps "linear transformations" and they have really nice properties. If you study linear algebra you'll learn all about them. Computationally (usefully), geometrically (Whoooa moments), and theoretically (knowledge buzz). And that's just the math part! Once you start to look into applications, it goes really far... Others in this thread have already mentioned numerous areas of applications in CS areas, and it's the same in biology, chemistry, economics, etc. TL;DR: linear algebra is good stuff!
PS: Here is a short video where I explain more about why linearity [normally written as f(ax1+bx2) = af(x1)+bf(x2)] is so important: https://github.com/minireference/noBSLAnotebooks#ch2-lineari...
My LA teacher in college told us that he visualized the left matrix in a matrix multiplication flipping over and crashing down on top of the right matrix. (or is that backwards?) The only way I learned things like vector spaces and orthogonality was by visualizing them in my head, Bret-Victor style.
In my totally unqualified opinion, there is a huge opportunity for a software product that allows people to play with and explore mathematical concepts in a visual and fun way.
Both ways work. A row from the left matrix rotated 90° clockwise to line up with the rows of the right matrix gives you the coefficients to use to combine those rows into a row of the result matrix. Conversely, a column from the right matrix rotated 90° anticlockwise to line up with the columns of the left matrix gives you the coefficients to use to combine those columns into a column of the result matrix.
I didn't fully understand that until I read Strang's book.
Before I dropped out of FSU, I managed to rope myself into two math classes (Complex Variables and Game Theory), both of which had a strict requirement of Linear Algebra, which I hadn't taken.
I was a bit panicked because it was a bit too late to sign up for anything else, and so I binged through this MIT course, and learned just enough to get permission from both the professors to still let me take the classes. It definitely would have been better to have taken linear algebra directly, but having this as a resource was super valuable.
He has a great sense of humour and keeps slipping jokes in unexpected places.
These are by far the best lectures on Math/CS I've ever seen. Helped me A LOT.
Strang is absolutely enlightening but I found it incredibly moreso after watching 3blue1browns series on visualizing linear algebrahttps://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
This is actually pretty good advice. A lot of the other MIT Open Courseware sites were ... not very good. At all. Some were only a syllabus. Some of the video lectures were utterly unwatchable.
In this case, however, Dr. Strang's reputation is well deserved. Absolutely, don't just watch these because they're from a 'good' school. But do watch them because they are fantastic, if you have an interest in his topics.
Don't watch these just because it's from MIT, watch them just because of Strang!
I don't mean to sound negative about it though. For example, the notions of abstraction and interface are very important in both maths and software.