from Hacker News

A 2020 Vision of Linear Algebra

by organicfigs on 5/12/20, 4:55 AM with 200 comments

• by katzgrau on 5/12/20, 12:13 PM

  I had very intelligent linear algebra professor in college but he was, in my opinion, a very poor communicator. I paid attention to lectures and stared at the text, but couldn't really understand the material. For the first part of a linear algebra course, students who don't mind blindly following mechanical processes for solving problems can do very well.

  Unfortunately I'm one of those people who tends to reject the process until I understand why it works.

  If it wasn't for Strang's thoughtful and sometimes even entertaining lectures via OCW, I probably would have failed the course. Instead, as the material became considerably more abstract and actually required understanding, I had my strongest exam scores. I didn't even pay attention in class. I finished with an A. Although my first exam was a 70/100, below the class average, the fact that I got an A overall suggests how poorly the rest of the class must have done on the latter material, where I felt my strongest thanks to the videos.

  So anyway, thank you Gilbert Strang.

• by pengaru on 5/12/20, 11:57 AM

  It's ridiculous how much random college-level linear algebra textbook material I stared at before things clicked in the course of just jumping in and exploring 3D graphics and writing my own 3D vector, matrix multiplication and 3D transform headers and using them in making some games in plain C.

  At some point it's like "Wait, is linear algebra really just about heaps of multiplication and addition? Like every dimension gets multiplied by values for every dimension, and values 0 and 1 are way more interesting than I previously appreciated. That funny identity matrix with the diagonal 1s in a sea of 0s, that's just an orthonormal basis where each corresponding dimension's axis is getting 100% of the multiplication like a noop. This is ridiculously simple yet unlocks an entire new world of understanding, why the hell couldn't my textbooks explain it in these terms on page 1? FML"

  I'm still a noob when it comes to linear algebra and 3D stuff, but it feels like all the textbooks in the world couldn't have taught me what some hands-on 3D graphics programming impressed upon me rather quickly. Maybe my understanding is all wrong, feel free to correct me, as my understanding on this subject is entirely self-taught.

• by knzhou on 5/12/20, 6:31 AM

  Gilbert Strang's linear algebra course blew my mind back in high school, and I still use insights from it every day. Strang has a particular lecturing style where he approaches every topic several times, often beginning many lectures before the main treatment. At first I thought it was a bit confusing, but later I realized it helped build fluency, just like a language class.

  I'm really thankful to MIT OCW for putting his lectures out for free -- in fact, I think I'll go donate to them now.

• by auggierose on 5/12/20, 8:20 AM

  Have not watched the videos yet, but that seems to me more like an 1820 vision of linear algebra :-)

  If you look at the order of topics in his book "An Introduction to Linear Algebra", you will find the topic "Linear Transformation" way back in chapter 8! Even after the chapters eigenvalue decomposition and singular value decomposition. But understanding that a matrix is just the representation of a linear transformation in a particular basis is probably the most important and first thing you should learn about matrices ...

• by roenxi on 5/12/20, 7:31 AM

  It is interesting to compare this with 3Blue1Brown's linear algebra introduction on YouTube. He seems to have been the only mathematician who has actually mastered the medium; linear algebra lends itself very well to animations. The mathematicians don't understand how badly they need to animate some of these concepts.

• by enitihas on 5/12/20, 9:26 AM

  Another good Linear Algebra book is "Linear Algebra Done Right", which Springer is giving for free right now.

  Link: https://link.springer.com/book/10.1007/978-3-319-11080-6

• by crdrost on 5/12/20, 8:16 AM

  So I like this outline. It is very MIT-ish where there is a sense of teaching someone to solve practical engineering problems with matrices.

  But, I do foresee some difficulties. One thing that I find really difficult, for example, is that I take undergrads who have had linear algebra and ask "what is the determinant?" and seldom get back the "best" conceptual answer, "the determinant is the product of the eigenvalues." Like, this is math, the best answer should not be the only one, but it should be ideally the most popular. We would consider it a failure in my mind if the most popular explanation of the fundamental theorem of calculus was not some variation of "integrals undo derivatives and vice versa". I don't see this approach solving that. Furthermore there is a lot of focus from day one on this CR decomposition which serves to say that a linear transform from R^m to R^n might map to a subspace of R^n with smaller dimension r < min(m,n) and while in some sense this is true it is itself quite "unphysical"—if a matrix contains noisy entries then it will generally only be degenerate in this way with probability zero. (You need perfect noise cancelation to get degeneracies, which amounts to a sort of neglected underlying conserved quantity which is pushing back on you and demanding to be conserved.) In that sense the CR decomposition is kind of pointless and is just working around some "perfect little counterexamples". So it seems weird to see someone say "hold this up as the most important thing!!"

• by brmgb on 5/12/20, 9:27 AM

  After watching this and having read the comments, I am quite puzzled by the approach American seem to take to linear algebra. Are matrices viewed as the core of the subject in the USA ?

  My country curriculum introduces linear algebra through group theory and vector spaces. Matrices come later.

• by balls187 on 5/12/20, 4:36 PM

  Admittedly, I never fully groked linear algebra.

  Some of the concepts made sense, especially solving for linear systems of equations.

  Recently, I decided to brush up on my math skills via Youtube videos, and came across this series: https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw

  It explains Linear Algebra concepts using 2D and 3D vector manipulation, and the animations help me visualize the underlying maths.

• by srean on 5/12/20, 8:26 AM

  I am familiar with the material of linear algebra but haven't read his books. Could someone who has absorbed linear algebra from different sources and familiar with Strang's books comment on what's good and bad and unique about them.

  In my time I had picked LA from Ben Noble, Halmos and Axler and the computation side of things from Golub & van Loan.

• by irl_zebra on 5/12/20, 1:03 PM

  I've been wanting to learn linear algebra. I had some exposure in college along with my calc classes, but never really understood it fundamentally. Like it was mentioned, I mostly did matrix transforms but didn't realize fundamentally grasp.

  I started doing LA on Khan academy, and checked out Linear Algebra Done Right. LADR was a little too much into the deep end for me. KA seemed to be good. One nice thing about KA is that when I didn't quite remember something (i.e. how exactly to multiply a matrix) I could just go to an earlier pre-LA lesson, pick it up, and then go back to LA where I left off. I'm a few lessons in.

  What do you all recommend for someone like me?

• by praptak on 5/12/20, 8:48 AM

  I'm currently trying to grok the finite element method. Gilbert Strang's explanation of the transition from the Galerkin method to FEM did more for me in terms of connecting the dots than anything else I could find on the web. And it wasn't even a lecture, just a kind of an interview. I think it's this one: youtube.com/watch?v=WwgrAH-IMOk

• by tomahunt on 5/12/20, 7:51 AM

  A 2018 paper by Strang about this approach:

  https://www.tandfonline.com/doi/abs/10.1080/00029890.2018.14...

• by cashsterling on 5/14/20, 5:25 AM

  I like all Strang's books... at least the ones I have. I don't have his Learning from Data book, yet... however.

  I also really like the applied linear algebra book by Boyd Vandenberghe: https://web.stanford.edu/~boyd/vmls/ Free PDF is available on their website. There is Julia and Python code companions for the book and lecture slides from both Profs their websites. Also check out their other books, many of which have free PDF's available.

  I can also recommend Data-Driven science and engineering by Brunton and Kutz. http://databookuw.com/ There used to be a free preprint PDF of the book but I can't find it now. Book is totally worth picking up... MATLAB and Python code available. Steve Brunton's lectures on YouTube are pretty damn good and compliment the book well: https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/fea...

  Another really cool book is Algorithms for Optimization by Mykel Kochenderfer and Tim Wheeler: https://mitpress.mit.edu/books/algorithms-optimization. Julia code used in book.

• by frequentnapper on 5/12/20, 7:30 AM

  Back in uni (2005), we used Dr. Strang's text for linear algebra. When reading the text, I felt like some down-to-earth professor was trying to explain these difficult topics as simply as possible. I remember discovering mit.edu back then and finding precious video lectures that went along with the book after the course. One of the very few times I was so genuinely happy and excited to watch math lectures online :p

• by sqlmonkey on 5/12/20, 7:55 AM

  I started by only reading his book thinking it was enough. I was very wrong, these videos marry themselves beautifully with the content of the book which suddenly became incredibly clear once I started watching the videos. Strang teaching style can also seem odd at first, but don't give up, he is an amazing teacher who makes every concept simple to understand. This course is a true gift.

• by chadcmulligan on 5/12/20, 11:24 AM

  Wow, Prof Strang is 85 and still teaching! Thats very impressive and inspiring.

• by jp0d on 5/12/20, 7:20 AM

  I've been doing the Statistics Micromasters from MIT. It's rigorous and very deep. I look forward to doing this.

• by ivan_ah on 5/12/20, 6:35 PM

  For anyone who is already familiar with the Prof. Strang's lectures from previous years, the main new thing in this five-lecture mini-series is he tries to condense the material even further—maximum intuition and power-ideas, instead of the full-length in-class lecture format with derivations. This makes the material difficult to understand for beginners, but makes a great second source in addition to or after a regular LA class.

  One of the interesting new ways of thinking in these lectures is the A = CR decomposition for any matrix A, where C is a matrix that contains a basis for the column space of A, while R contains the non-zero rows in RREF(A) — in other words a basis for the row space, see https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-li...

  Example you can play with: https://live.sympy.org/?evaluate=C%20%3D%20Matrix(%5B%5B1%2C...

  Thinking of A as CR might be a little intense as first-contact with linear algebra, but I think it contains the "essence" of what is going on, and could potentially set the stage for when these concepts are explained (normally much later in a linear algebra course). Also, I think the "A=CR picture" is a nice justification for where RREF(A) comes about... otherwise students always complain that the first few chapters on Gauss-Jordan elimination is "mind-numbing arithmetic" (which is kind of true...) but maybe if we present the algorithm as "finding the CR-decomposition which will help you understand dozens of other concepts in the remainder of the course" it would motivate more people to learn about RREFs and the G-J algo.

• by inshadows on 5/12/20, 9:47 AM

  Gilbert Strang taught me how to sanely multiply matrices. His Introduction to Linear Algebra is very approachable. It's wildly different experience compared to linear algebra courses I had on university, it actually makes sense and is fun!

• by synaesthesisx on 5/12/20, 8:00 AM

  Linear algebra was one of those classes I was forced to take in undergrad as an engineering requirement - only to end up appreciating it immensely later on when I realized how many real world problems can be converted to matrix operations.

• by glram on 5/12/20, 7:57 AM

  Professor Strang’s lectures helped me greatly during my linear algebra class. I thoroughly appreciated his clear, coherent lecture style.

  On another note, he is such a nice guy. 10/10.

• by abecode on 5/12/20, 6:56 PM

  Two things really made linear algebra click for me: representing camera projections in a computer vision class and spectral graph theory, which basically connects graphs with linear algebra. In both of these, it seems like linear algebra was taken from the electrical engineering domain into computer science, which better fit my perspective.

• by vertak on 5/12/20, 12:05 PM

  Can anyone grow why in the first 5 minutes of part 1 he shows a 3 by 3 matrix multiples by a 1 by 3 vector yet verbally he pulls out of no where this idea that if you have _two_ 1 by 3 vectors that pass through the origin then their linear combinations can be represented by a plane? The jump from the 3D to the 2D has me lost and I gave up

• by penguin_booze on 5/12/20, 9:15 AM

  I recently came across this rather in-depth series on linear algebra: https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS1.... FWIW, I myself have only gone half-way thorugh part 1.

• by DreamScatter on 5/12/20, 10:16 PM

  An interesting alternative to linear algebra is geometric algebra. I recommend googling around a bit for geometric algebra and trying out my implementation https://github.com/chakravala/Grassmann.jl

• by hprotagonist on 5/12/20, 1:39 PM

  my linear algebra professor began his first lecture with a 5 minute rant informally titled “how you could have gotten an A in differential equations last semester without ever having taken calculus”

  That certainly got our attention. I’ve always found linear algebra to be kind of ... almost soothing.

• by longtimegoogler on 5/12/20, 6:02 PM

  IMO, if one is interested in a computational approach to Linear Algebra, Trefethens book, Numerical Linear Algebra, is the best.

  That book discusses the actual algorithms used for computation. It is a bit more advanced, but amazingly clear.

• by elAhmo on 5/12/20, 10:13 AM

  What would you recommend as a good resource for learning about Linear Algebra in 2020?

  I am aware of his course on OCW, but wondering is there something more interactive and/or newer than those lectures that has similar quality.

• by potta_coffee on 5/12/20, 6:54 AM

  Picked up Strang's linear algebra book recently and I'm enjoying it. I've been consistently impressed with the content of MIT books.

• by anandrm on 5/12/20, 12:48 PM

  Just curious .. what really are the usecases where of Linear Algebra is applied ? Any domain of software development ?

• by xchip on 5/12/20, 7:58 PM

  Every science degree studies algebra in their first course, it should be regarded as something pretty basic. How comes people are still talking about this?

• by tomerbd on 5/12/20, 6:50 AM

  It was a good course I watched it online but I didn't understand much.

• by lcuff on 5/12/20, 7:57 AM

  As someone who understands nothing of linear algebra, I have to say this "introduction" was gibberish. He may be a fantastic teacher, and perhaps it's a bit much to expect a 4 minute video to teach me anything, but it reminds me of talks from business people where what they're saying is obvious if you already understand it, and completely obscure otherwise.