from Hacker News

Relearning Matrices as Linear Functions

by dhruvp on 4/23/19, 5:25 PM with 93 comments

by _hardwaregeek on 4/23/19, 8:39 PM
Linear Algebra, at least at my school, is taught pretty poorly. Instead of teaching the beauty of transformations, the course is boggled down in numerical nonsense and tedious calculations (who wants to find the inverse of a 3x3 matrix? Bueller? Bueller?). Only after learning Algebra and homomorphisms, isomorphisms and automorphisms did I appreciate the importance of linear transformations. Stuff like Singular Value Decomposition gets a lot more interesting once you know some basic Algebra. I suppose Linear can't get too abstract because non math majors have to take it, but starting from generalized ideas of transformations is a far better way to teach it imo.
by andrewla on 4/23/19, 7:41 PM
It took until I started learning differential geometry in the form of General Relativity to arrive at this insight, even though I feel like the notion of a matrix as a linear map was drilled in pretty thoroughly. The notion of matrix multiplication as function composition was presented almost as an interesting side effect of matrix multiplication -- that is, multiplication by these rules came first, and, hey, look, they compose!
Personally I found the prospect of tensor algebra to be much more intuitive than either of these; with matrices thrown in mostly as a computational device. Even a vector (through the dot product) is just a linear function on other vectors, and the notion of function composition carries through to that and to higher-order tensors.
Covariance and contravariance are a little more complicated to completely grok, but for most applications in Euclidean space (where the metric is the identity function) the distinction is of more theoretical interest anyway.
by munchbunny on 4/23/19, 8:58 PM
In my high school matrices were first taught in geometry class, starting with using matrices as affine transformations in 2-d and then 3-d, and using that to teach concepts like what eigenvectors/values are, the equivalence of matrix and function composition, etc.
That was taught right after a unit on complex numbers and trigonometry so that we could see the parallels between composing polynomial functions on complex numbers and composing affine transformations.
To this day I think that was one of the most beautiful and eye opening lessons I've had in mathematics.
In hindsight, I think I got lucky that the teachers who wrote the curriculum this way were math, physics, and comp sci masters/phd's who looked at their own educations and decided that geometry class was a great Trojan horse for linear algebra.
by whatshisface on 4/23/19, 7:01 PM
FWIW, I was told that matrices are linear maps pretty early on in my education. Are there any college level linear algebra / matrix calculations courses that don't tell students about that?
by dhruvp on 4/23/19, 7:43 PM
Hey OP here!
When I first was introduced to matrices (high school) it was in the context of systems of equations. Matrices were a shorthand for writing out the equations and happened to have interesting rules for addition etc. It took me a while to think about them as functions on their own right and not just tables. This post is my attempt to relearn them as functions which has helped me develop a much stronger intuition for linear algebra. That’s my motivation for this post and why I decided to work on it. Feedback is more than welcome.
by michelpp on 4/23/19, 8:30 PM
This is great and a nice mathematical approach to the ideas of matrices. Another great resource is 3blue1brown's essence of linear algebra:
https://www.3blue1brown.com/essence-of-linear-algebra-page
Math is Fun also has a nice writeup that explain matrix multiplication from a real world example of a bakery making pies and tracking costs:
https://www.mathsisfun.com/algebra/matrix-multiplying.html
by noobermin on 4/24/19, 1:08 AM
One of the things that always irked me about the term "linear transformation" is it doesn't include affline transformations, which is funny because back in elementary school, you learn that a "linear equation" looks like Mx + b. Of course, the article states the term "linearity" when talking vector spaces (or modules) means linearity in arguments, while the term linear for a child in school means "something like a line on graph paper", and this is yet another example of terminology in the way mathematics is taught, possibly for historical reasons, that leads to even more confusion.
PS. incase you didn't know, affline transformations are not linear:
```
  f(x) = mx + b =>
  f(x+y) = m(x+y) + b /= mx+b + my+b = f(x) + f(y),
  f(cx) = c m x + b /= c(mx + b) = c f(x)
```
by tptacek on 4/23/19, 11:12 PM
Their most recent post about kernels is even better than this:
https://www.dhruvonmath.com/2019/04/04/kernels/
The matrix/function stuff is elementary enough that I understand it intuitively (I suck at math), although it's neat to be reminded that given a enough independent points you can reconstruct the function (this breaks a variety of bad ciphers, sometimes including ciphers that otherwise look strong).
The kernel post actually does some neat stuff with the kernel, which I found more intuitively accessible than (say) what Strang does with nullspaces.
by adenadel on 4/23/19, 7:12 PM
If you're interested in this approach to linear algebra you should read Linear Algebra Done Right by Sheldon Axler.
by meuk on 4/24/19, 7:27 AM
It recently occurred to me that if you use that matrices represent linear functions, you don't have to do tedious math to prove that matrix multiplication is associative (that is, (A * B) * C = A * (B * C), which allows us to write A * B * C without brackets, since it doesn't matter how we place the brackets anyway).
For a matrix M, denote f_M(x) = M * x. Then f_{A * B} = f_A(f_B(x)) so that f_{(A * B) * C} = f_{A * B}(f_C(x)) = f_A(f_B(f_C(x))) and also f_{A * (B * C)} = f_A(f_{B * C}(x)) = f_A(f_B(f_C(x))).
So f_{(A * B) * C} = f_{A * (B * C)} = f_A(f_B(f_C(x)))
by adamnemecek on 4/23/19, 9:53 PM
Conjugate transpose and other adjoints are kinda nuts, they are the other part of the story
http://www.reproducibility.org/RSF/book/bei/conj/paper_html/...
Esp the ray tracing/topology relationship is nuts.
by ivan_ah on 4/23/19, 7:54 PM
Nice! The illustrations + color coding for the vectors are very useful.
Here is a video tutorial that goes through some of the same topics (build up matrix product from the general principle of a linear function with vector inputs): https://www.youtube.com/watch?v=WfrwVMTgrfc
Associated Jupyter notebook here: https://github.com/minireference/noBSLAnotebooks/blob/master...
by Jun8 on 4/23/19, 7:24 PM
Good, intuitive introduction to matrices. Next steps could be showing that there are infinitely many different matrix representations of a linear map (different from the polynomials) and they can be used for function spaces, too.
One question that usually pops up that I was confused about till recently: are rank two tensor equivalent to matrices? Answer is no, e.g. see here: https://physics.stackexchange.com/questions/20437/are-matric...
by S4M on 4/24/19, 7:35 AM
I just skimmed the article quickly. Are there other ways to learn about matrices? If you don't treat them as linear applications, they are just boring grids of numbers and the matrices multiplication doesn't make any sense.
by sytelus on 4/24/19, 10:57 AM
The basic equivalency is fine but what about all other things you can do with matrices but can’t do with functions? For example, what is the equivalent of transpose in functions? How about Eigen values or Gaussian elimination?
by zwieback on 4/23/19, 7:35 PM
Nice article. That's how I learned matrices in high school in Germany. Maybe it's different here in the US, I'll have to take a look at my daughters' textbooks.
by kregasaurusrex on 4/23/19, 7:38 PM
Having not taken a linear algebra course in college, does anyone have a recommendation for a book/course to follow?
by diehunde on 4/23/19, 8:18 PM
Gilbert Strang uses similar approach on his Linear Algebra lectures. Much more intuitive
by mikorym on 4/23/19, 8:09 PM
The next relearning step is to construct the category where arrows are matrices...
by je42 on 4/24/19, 6:16 PM
this was an important result in the linear algebra class for first year math/cs/eng students at my university.
by Grustaf on 4/24/19, 7:04 AM
What could possibly be a more basic understanding of a matrix in mathematics? There’s a reason the first teach you Linear Algebra before anything else.
by j7ake on 4/23/19, 7:08 PM
Who is "we" in this context?