Comment by dragontamer | Hacker News Reader

Comment by dragontamer | Hacker News Reader

Why are you trying to learn linear algebra?

This is highly important. Linear algebra is applicable to so many fields, but learning linear algebra for say... Graphics Programmers, is a completely different feel from learning linear algebra for an Electrical Engineer Signals-and-systems engineer.

Graphics programmers largely need to learn "how to use" matricies. Emphasis on associative properties. Emphasis on non-communitive operations.

In contrast, Electrical Engineers / Signals-and-systems want to learn linear-algebra as a stepping stone to differential calculus. In this case, you're going to be focusing more on eigen-values, spring-mass systems / resonant frequencies, applicability to calculus and other tidbits (how linear algebra relates to the Fourier Transform).

----------

The graphics programmer (probably) doesn't need to learn eigenvalues. So any textbook written as "linear algebra for graphics programmers" can safely skip over that.

The electrical engineer however needs all of this other stuff as "part" of the linear algebra class.

I'm sure other fields (statistics, error-correction codes/galois fields, abstract algebra, etc. etc.) have "their own ways" of teaching linear algebra that is most applicable to them.

Yes, "linear algebra" is broadly applicable. But instead of trying to "learn all of it", you should instead focus on the "bits of linear algebra that is most applicable to the problems you face". That shrinks down the field, increases the "pragmatism" of your studies.

Later, when you're more familiar with "some bits" of linear algebra, you can then take the next step of generalizing off of your "seed knowledge".

--------

I personally never was able to learn linear algebra from a linear algebra book.

Instead, I relearned linear algebra 4 or 5 times as the "basis" of other maths I've learned. I learned it for differential calculus. I relearned linear algebra for signals. I relearned linear algebra for Galois fields/CRC-codes/Reed Solomon. I relearned linear algebra for graphics.

Yes, it seems inefficient, but I think my "focus" isn't strong enough to just study it in the abstract. I needed to see the "applicable" practice to encourage myself to learn. Besides, each time you "relearn" linear algebra, its a lot faster than the last time.

Thank you, this is a great point! I am in the category of someone who needs linear algebra in order to apply it for day-to-day stuff, hands on not blue sky. Currently my primary use case is image filtering but a bit down the line signal processing will come up.

> Currently my primary use case is image filtering but a bit down the line signal processing will come up.

Image filtering _is_ signal processing, two-dimensional signal processing to be precise.

Traditionally, a college would take you through linear algebra -> differential equations -> signals and systems, to approach this subject.

I found it easier to go through the reverse: start at signals-and-systems (to see what you have to learn), then work your way back down to linear algebra, and then work your way back up to signals and systems.

---------------

From a "signals and systems" point of view, your image filtering functions are 99% going to just be a "kernel" applied to an image.

https://en.wikipedia.org/wiki/Kernel_(image_processing)

IMO, its easier to start with a 1-dimensional version, where you perform kernels upon sound and/or RADAR signals rather than 2-dimensional images.

https://en.wikipedia.org/wiki/Convolution#Visual_explanation

You can see that the 1-dimensional version of the convolution applied between (data x kernel) is extremely simple and "obvious" to think about, given this GIF: https://upload.wikimedia.org/wikipedia/commons/6/6a/Convolut...

Where blue-box is the original signal, and red-box is the convolution-kernel, and the black-line is the output of blue convolve with red.

From there, you generalize the 1-dimensional convolution, into a 2-dimensional convolution. To do so, you need to study linear algebra and matricies. But now that you're "focused" upon the convolution idea, as well as the idea of a "kernel", everything should be "more obvious" to you as you go through your studies.

You can see that a "Matrix", in your specific field of study, represents a kernel to a discrete system. The image you want to manipulate is a 2-dimensional signal. A "matrix" is many different things to many different mathematicians / engineers. "Focusing" upon your particular application is key to learning as quickly as possible. (You can generalize later after you've mastered your particular field).

Still, the study of signals / systems is a very generalized and large field. Mechanical engineers study this, because it turns out that an "impulse" that is "convoluted" with a "kernel" is descriptive of how a speed-bump affects your car's suspension system (!!!!). (EDIT: A youtube video demonstrating the same math for earthquakes vs buildings: https://www.youtube.com/watch?v=f1U4SAgy60c)

So studying signals-and-systems is still a very abstract goal of yours. It sounds like you need to focus upon the image-processing portions of signals-and-systems.

---------

IMO, you'll find that there's probably very little linear algebra you actually need to learn for your particular path.

I largely agree with your comments in this thread -- I'd been thinking about trying to express the same thing myself. I'd been guessing the motivation would be ML, where I feel that most people substantially overestimate how much they'd need.

Signals processing, though, is one of the places where I actually think a decent understanding of some of the higher-level concepts in linear algebra is really helpful. Linearity itself comes to mind, and maybe it just reflects my physicist's education, but it's hard for me to imagine having a working understanding of Fourier transforms without getting the idea of changing bases. I feel like you're about halfway through a first linear algebra course before you'd get there.

EDIT: that said, a good signals processing book probably covers a lot of this in sufficient depth if you can figure it out. The other catch-all comment I'd make is that linear algebra from a math class can look somewhat different from practical linear algebra on a computer. (That it's often a bad idea to directly invert a matrix for many computing applications is non-obvious from math class.) A book like Trefethen and Bau is great on that latter subject but is not a good starting point for OP.

> I personally never was able to learn linear algebra from a linear algebra book.

> Instead, I relearned linear algebra 4 or 5 times as the "basis" of other maths I've learned. I learned it for differential calculus. I relearned linear algebra for signals. I relearned linear algebra for Galois fields/CRC-codes/Reed Solomon. I relearned linear algebra for graphics.

If I were way better at websites and at advanced mathematics than I actually am, I'd make a site for learning math in a top down manner where you start with some result or application that interests you and then are taught just enough more elementary math to support that result or application.

The site would have a list of results and applications, and for each tell what math is necessary to understand it. You pick a result or application that interests you, either because it is interesting to you itself or because you see that it depends on some more elementary math that you wish to learn.

Once you pick, the site would show you a proof of the result or development of the application, at a level that one would find in a journal aimed at professionals in the relevant field. This of course will most likely be largely incomprehensible at this point.

You can select any part of the proof or development and ask the site for more information. There are two kinds of additional information you can ask for.

One is to ask for smaller steps. You use this when there is some step A -> B where you are comfortable with A and B but just don't see how it jumps from A to B. You understand what A means, what B means, just not why A -> B. The site fills in the intermediate steps.

The other is to ask what something means. This is for when the proof uses something you have not yet studies. For example if the proof uses integration and you have not yet studied it calculus that would be a great place to use a "what does this mean?" request. The site would then give you a short explanation of integration.

A key feature of the site would be that this is all recursive. If you use a "what does this mean?" request on an integral and get the short explanation of integration, you could use "smaller steps" requests and "what does this mean?" requests in that explanation.

Using "what does this mean?" requests recursively should let you go all the way down to things that can be explained with only high school algebra and precalculus.

Note that if you've never studied anything past high school algebra and precalculus and then use the site to learn something like say an analytic proof of the prime number theorem you will learn much elementary calculus but not all. You will learn just what is needed for the prime number theorem.

But there would be other interesting theorems and applications that use different parts of elementary calculus, so doing those would fill in more of your elementary calculus.

The site should have a planner that lets you pick areas of undergraduate or masters level math that you would like to learn and then shows you lists of interesting theorems and applications it has that will cover those areas.

I think this would be an interesting and effective way to learn. At all points everything you are learning goes directly toward supporting the top level proof you have chosen to learn, and you have an idea of why it is useful because you are there because you've already encountered something where you need it.

I think that for many people this will provide better motivation. In the conventional approach, where you do say a whole class in calculus or abstract algebra, then do a more advanced class that uses those results, and so on, a lot of time you are learning stuff with no idea of why it is useful.

Reply on news.ycombinator.com