I would recommend Steven Boyd’s Introduction to Applied Linear Algebra, which is not only nicely written but also legally available in electronic form on the author’s website at http://vmls-book.stanford.edu/.
I would then follow with Data Driven Science and Engineering – Machine Learning, Dynamical Systems and Control by Steve Brunton and Nathan Kutz. Unfortunately, PDF of the book is not available on the book website http://www.databookuw.com/ (well, you can always buy the book), but a whole lot of videos are linked there. In particular, a (sub)section on Dimensionality reduction using SVD (including a careful and accessible intro to PCA) is at Chapter 1: Singular Value Decomposition | DATA DRIVEN SCIENCE & ENGINEERING. Check it out.