In a computer science context, I have to plug the old FLAME group for publishing and teaching the topic so well. - digging into parallelizing computations efficiently before neural nets took off around the 2014 time.
LibFlame has long been abandoned now but their courses were very strong when you had access to the professors. They have been rebranded as the Science of High Performance Computing (SHPC) group.
As a math educator, I strongly dislike both Strang and Axler for a 1st course. I've heard great things about Strang's lectures, but his book is disorganized and too heavy on computation. Axler's book is wonderful, but as explicitly stated on the back cover, it's designed for a 2nd course and primarily aimed at math majors.
I recommend and teach my YouTube Live series out of Fraleigh [1], but unfortunately it's out of print. Lay seems to be a good modern alternative.
Strang's OCW lectures are fantastic. His voice rings "combinations of columns" in my head almost ten years later whenever I do a matrix or matrix-vector multiply on paper. I found the textbook was a good accompaniment for the lecture course, but maybe not a good book to use on its own for self-study. (IIRC, when following the lectures, you jump around the chapters a bit out of order [or, you read the chapters non-linearly]. So the disorganized thing may be true.)
Hard disagree. Linear algebra is one of the easiest topics in mathematics. In my experience, 3Blue1Brown videos are a great simple introduction. Beyond that, Kostrikin’s books.
I first learned from self studying Finkbeiner’s “Introduction to Linear Transformations and Matrices” which I thought was very good. Great exercise's. Dover still prints it for pretty cheap.
He does a cool thing introducing all sorts of theory of linear transformations, then later showing that matrices are the way to encode them once you chose basis for domain and codomain. I liked that a lot, felt like it removed any magic from matrices.
Also a fan of Axler though.
Strang is simpler and clearer. Axler is more advanced in the sense that it doesn’t tie it to matrices. Strange is a “first course” book, Axler is a second course.
Depends how you think. I found Strang impenetrable and Axler simple and lucid. Some people seem to find abstract vector spaces weird and unmotivated without doing a load of stuff with lists and grids of numbers first. I find determinants weird and unmotivated without learning exterior algebra first. I wish Axler had been my first course.
I've taught out of the first three of these. If I had to pick one for a first study, I'd vote for Strang -- and watch his videos while you go. Our second-semester mostly-math-majors linear algebra course uses Axler, which I think is nice for the purpose, but our students already have done a semester of computational stuff first. (Though the complete absence of any computations in the book means they don't always connect the material from the two courses very well.)
LADW saved me in undergrad, but I was pretty much exactly the target audience in an honors-level freshman math course:
"[per Treil, LADW is for] a student who, while not yet very familiar with abstract reasoning, is willing to study more rigorous mathematics than what is presented in a “cookbook style” calculus type course."
But yeah it's really attempting to introduce you to higher mathematics rather than get you comfortable doing linear algebra per se.
I am a very big fan of Axler's approach, because to me it really gets to the (abstract) essence of things. When I truly started to "get" this approach, it really made linear algebra an important part of how I think.
I really recommend Matrix Analysis and Applied Linear Algebra by Carl Meyer. It's both concise and comprehensive. Strange is very good, but feels kinda vague and long winded in comparison (very good for a high level understanding of the tools you're dealing with)
Wikipedia says "The book has a pure, proof-heavy focus and is aimed at upper-division undergraduates who have been exposed to linear algebra in a prior course." [1], so it seems to be a different category of book?
If you liked 3B1B and prefer intuition/applications-heavy view, then definitely Strang over Axler. Check out especially his newer textbook "Linear Algebra and Learning from Data".
Axler is more of a pure math textbook - if you want to dive more into proofs and abstractions.
I would also say Axler is much better prep for higher level applied math, as well as pure. If you are interested in how the big ideas of linear algebra extend to things like Fourier analysis it's very helpful to see the more abstract explanation of vector spaces.
I strong second Strang. His book is the best first introduction to linear algebra, with "Done Right" marketing itself as a second course. Axler is notoriously shy with matrices, but Axler introduces them up front and uses them for the rest of the book.
if self teaching you need a book that holds your hand and doesnt try to put you through the usual trials of "lets see if you can figure this out yourself.. oh you cant? maybe repeat the subject next term", or major proofs left as exercises for the reader. get a friendly hand-holdy book. im reading linear algebra, theory, intuition, code by mike cohen atm (up to ch4) and so far between that and a chatbot for clarification/links, im re-learning the L.A. i should have learn't at uni but for whatever reason just did not understand back then. those courses move fast and if you get stuck somewhere its game over. i think L.A. is way harder than calculus as a beginner, but once things start to click and fall into place it starts to feel easier.
I enjoyed "No bullshit guide to linear algebra". And I wish "Coding The Matrix: Linear Algebra Through Computer Science Applications" still had the autograder working. Though I wonder if I could throw some agents at replacing it....
The nature of textbooks is that each one is better suited for a certain profile of reader. It depends a lot on the way the reader has learnt to learn things until that point in their life.
If you liked 3B1B's style, you will prefer strang over axler. Axler and treil to a greater extent focus on bringing out the abstract elegance and the kind of rigour a math major enjoys. Strang's book also has videos accompanying - on MIT OCW.
B&V VMLS on your list is interesting - they focus a lot on real-world instantiations of the concepts and have you code up things in the (excellent) exercises. Depending on your goals, you can do only this, or strang and then this. Definitely look at the exercises in any case though.
Note that "done right" means done with Axler's completely subjective and unusual hatred of determinants, chronicled here [0]. It is in no way "done right" in some definitive, rigorous way; most math professors I have spoken to either strongly disagree with the presentation or have no particular preference.
Determinants are easy to use but very hardly to grasp intuitively, this is not a minority point of view. See countless of StackOverflow questions begging for a conceptual exposition of determinants.
The easiest conceptual handle is geometric: volume expansion, but seeing how this is related to the combinatorial sum over all permutations, or how those two point of views are related to the algebraic one (that a set of equations having a solution or not), is not easy to see even in the 2D case.
I won't be surprised if math professors don't have this issue like you said (especially if someone is comfortable with wedge products), but the vast majority of newcomers who are interested in understanding why something works rather than just how to use it struggle all the time with determinants.
I think the volume explanation is one of the most intuitive pieces of math in existence, personally! Uninvertibility of a tranformation corresponds to a volume of zero because the transformation must squish two dimensions together, leaving them impossible to differentiate, det(AB) = det(A)det(B) because applying two transformations applies their scaling successively, det(A^-1) = 1/det(A) because you have to undo the scaling to invert a transformation etc. I don't think the permutation definition is even strictly necessary; if I recall correctly Linear Algebra Done Wrong defines the determinant in terms of its geometric definition and develops its formula from the properties it must have. I think that the concept is well worth the investment of initial confusion. Axler disagrees, however.
Yeah I feel like this "done right" part is responsible for most of the popularity of this book. Makes the reader think they've been learning it wrong. Kind of like these clickbait videos "you've been folding your laundry wrong your whole life!" or whatever
This is supposedly based on Sheldon Axler's earlier and shorter paper "Down With Determinants!" [0]. I lectured mathematics for a while at a "former polytechnic" and used to enjoy leaving print-outs of this sort of paper in the faculty communal areas.
For those who find Linear Algebra Done Right too much to start with, and those who don't get why Strang starts with matrices, I can't recommend more "The dark art of linear algebra" read this first. With this you can then tackle every other book on the topic more easily
My personal favorite is No Bullshit Guide to Linear Algebra - it gives a really good overview of math fundamentals and overall strikes a good balance between keeping things simple and giving enough insight to comfortably dig deeper in the topic.
Nice seeing you on HN. I thoroughly enjoyed the No Bullshit LA and Maths & Physics books. I wish more books would include concept maps, I find them a very useful tool for understanding how things fit in the broader picture.
I remember from my tutoring days how useful they were to organize the different concepts covered in each lesson: I would start with a blank sheet and make the student add concepts to it as the lesson progressed, then by the end of the lesson use the concept map to review what we learned. Specifically, I would ask them to explain in their own words each "arrow" which was a great way to uncover misconceptions and solidify the material.
Once I'm done with editing the current book[1], I hope to have time to work on making dynamic concept maps that you can click on and explore/zoom-in on. I feel it would be cool to jump between detailed view (concepts), intermediate scale (topics), and high-level view (subjects).
In the 1980s, being not entirely adept at mathematics I recall scouring every library and bookstore I could find for any snippet that would explain a proof, or even a concept, so that I could understand it. Videotaped lectures by other professors were sometimes available on campus too.
I got halfway though the exercises with the help of a reading group. They were very hard, bit thought provoking, so I would definitely recommend. Don't feel discouraged if you get stuck and try not to look at the solutions right away.
I wanted to learn the underlying principles of LLM/AI and got myself Shilov's book. Wow that was so thick, each paragraph took a while to figure. This could be a nice option..
I found it really insightful (and always overlooked) to distinguish between vector and co-vector spaces. It doesn't necessarily produce new knowledge, but makes things more clear.
Overrated and tendentious book. There are many better linear algebra texts. His polemic against determinants is poorly motivated, misguided, and distracting. The writing is quite formal and not terribly inspiring. The coverage is adequate but nothing more.
> His polemic against determinants is poorly motivated, misguided, and distracting.
What polemic? Defining the determinant as the unique multilinear alternating form satisfying certain properties is very normal (and in fact the only way that really makes sense for both finite- and infinite-dimensional vector spaces). There are zero unusual things with this book imo.
Lately been deep diving into linear algebra. And a way which i engage with it is that I tell AI to generate interactive examples + questions on Lean or Haskell. Its so fun, just deriving the intuition in these languages.
My bag of tricks is better than your bag of tricks. Alright.
As with most textbooks, it fails to motivate why reading it is worth the investment.
Perhaps it is a millennial old tradition of the Greek mystery schools, that the rite of passage came by proving your commitment to material knowledge without anything but fate in the school itself as motivation.
LibFlame has long been abandoned now but their courses were very strong when you had access to the professors. They have been rebranded as the Science of High Performance Computing (SHPC) group.
https://github.com/flame/libflame
https://shpc.oden.utexas.edu/
The contenders seems to be:
- Linear Algebra Done Right - Sheldon Axler
- Liner Algebra Done Wrong - Sergei Treil
- Introduction to Linea Algebra - Gilbert Strang
- Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd and Lieven Vandenberghe
[1] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
I recommend and teach my YouTube Live series out of Fraleigh [1], but unfortunately it's out of print. Lay seems to be a good modern alternative.
[1] https://linear.mathcanbeahobby.com
Since it doesn't neglect computation, it's great as an introductory text.
After that, and some other math courses you can move on to Linear Algebra Done Right.
I ended up getting a math PhD so it must have done somethings right!
LADW and LADR are great too, for an honors approach with more focus on proofs. To me it would make more sense on a second pass.
That‘s fine, but I would have appreciated notices, which proofs and theorems do not hold in the general case.
It‘s an exercise for the reader.
"[per Treil, LADW is for] a student who, while not yet very familiar with abstract reasoning, is willing to study more rigorous mathematics than what is presented in a “cookbook style” calculus type course."
But yeah it's really attempting to introduce you to higher mathematics rather than get you comfortable doing linear algebra per se.
[1] https://en.wikipedia.org/wiki/Linear_Algebra_(book)
Axler is more of a pure math textbook - if you want to dive more into proofs and abstractions.
If you liked 3B1B's style, you will prefer strang over axler. Axler and treil to a greater extent focus on bringing out the abstract elegance and the kind of rigour a math major enjoys. Strang's book also has videos accompanying - on MIT OCW.
B&V VMLS on your list is interesting - they focus a lot on real-world instantiations of the concepts and have you code up things in the (excellent) exercises. Depending on your goals, you can do only this, or strang and then this. Definitely look at the exercises in any case though.
This is the best I know: https://www.youtube.com/watch?v=Fnfh8jNqBlg&list=PLlXfTHzgMR....
[0] I meant natural language
[0] https://www.axler.net/DwD.html
The easiest conceptual handle is geometric: volume expansion, but seeing how this is related to the combinatorial sum over all permutations, or how those two point of views are related to the algebraic one (that a set of equations having a solution or not), is not easy to see even in the 2D case.
I won't be surprised if math professors don't have this issue like you said (especially if someone is comfortable with wedge products), but the vast majority of newcomers who are interested in understanding why something works rather than just how to use it struggle all the time with determinants.
[0] https://www.axler.net/DwD.html
Linear Algebra Done Right 58 points, July 2023, 4 comments https://news.ycombinator.com/item?id=36576114
Linear Algebra Done Right – 4th Edition, 631 points, Oct 2023, 294 comments https://news.ycombinator.com/item?id=38060159
Linear Algebra Done Right [pdf], 85 points, Sept 2024, 39 comments https://news.ycombinator.com/item?id=41416799
linear_algebra_done_right.pdf, 0 pages read, July 2023
linear_algebra_done_right (1).pdf, 0 pages read, Oct 2023
linear_algebra_done_right (2).pdf, 0 pages read, Sept 2024
Downloading (3) now.
and the printable concept maps here: https://minireference.com/static/conceptmaps/linear_algebra_...
I remember from my tutoring days how useful they were to organize the different concepts covered in each lesson: I would start with a blank sheet and make the student add concepts to it as the lesson progressed, then by the end of the lesson use the concept map to review what we learned. Specifically, I would ask them to explain in their own words each "arrow" which was a great way to uncover misconceptions and solidify the material.
Once I'm done with editing the current book[1], I hope to have time to work on making dynamic concept maps that you can click on and explore/zoom-in on. I feel it would be cool to jump between detailed view (concepts), intermediate scale (topics), and high-level view (subjects).
[1] https://noBSstats.com
On a daytime episode of David Letterman, Isaac Asimov predicted fiber optics would one day bring about television studios in people's homes: https://youtu.be/cIB1b_8hqB0?si=212sGzZ71VIZORML&t=696
All sources of understanding are so very much appreciated.
Thanks!
I am assuming game developers are interested in the applications of Linear Algebra and there are many other books that are far suited for that.
Anyone from game dev community care to elaborate ?
What polemic? Defining the determinant as the unique multilinear alternating form satisfying certain properties is very normal (and in fact the only way that really makes sense for both finite- and infinite-dimensional vector spaces). There are zero unusual things with this book imo.
https://www.axler.net/DwD.html
A strange and unpopular opinion.
Possibly paired with some numerical algebra free text (many on the Internet)
i think lot of ppl are under the impression that long courses are a prerequiste to even start exploring ai/ml.
doing long prerequeste courses is why lot of ppl drop out even before getting to ai
As with most textbooks, it fails to motivate why reading it is worth the investment. Perhaps it is a millennial old tradition of the Greek mystery schools, that the rite of passage came by proving your commitment to material knowledge without anything but fate in the school itself as motivation.
Rigor before Worth.
(Yes this is a pet peeve of mine :)