Basics of Linear Algebra for Machine Learning: Discover the Mathematical Language of Data in Python
Series: Machine Learning Mastery
Language: English
Category: Ebook - Computer/Internet
Categories: Computer/Internet
Uploader: ad3f
Uploaded At: 2018-02-05 19:10:52
machine, learning, linear, algebra,
Description
Linear algebra is a pillar of machine learning. You cannot develop a deep understanding and application of machine learning without it.
In this new laser-focused Ebook written in the friendly Machine Learning Mastery style that you’re used to, you will finally cut through the equations, Greek letters, and confusion, and discover the topics in linear algebra that you need to know.
Using clear explanations, standard Python libraries, and step-by-step tutorial lessons, you will discover what linear algebra is, the importance of linear algebra to machine learning, vector, and matrix operations, matrix factorization, principal component analysis, and much more.
Why Linear Algebra?
Linear algebra is a sub-field of mathematics concerned with vectors, matrices, and operations on these data structures. It is absolutely key to machine learning.
As a machine learning practitioner, you must have an understanding of linear algebra.
It’s a huge field of study that has made an impact on other fields, such as statistics, as well as engineering and physics. Thankfully, we don’t need to know the breadth and depth of the field of linear algebra in order to improve our understanding and application of machine learning.
Frustrated with the Math?
Have you ever been frustrated reading the description of a machine learning technique?
You’re reading along, things are going well, and then you hit an equation, and you are stopped in your tracks with questions like:
… what do the terms mean?
… why are there no operators between terms?
… what does this Greek letter mean?
Unless you have a basic knowledge of linear algebra, you will not be able to read and understand even the most basic equations.
Tensors!?
Have you heard of TensorFlow, Google’s Python library for deep learning?
Did you know that “tensor” is a term taken directly from the field of linear algebra and it simply means an array with more than two-dimensions?
Why Is Linear Algebra Important to Machine Learning?
So, why is linear algebra used so much to describe machine learning algorithms?
Linear algebra is about vectors and matrices and in machine learning we are always working with vectors and matrices (arrays) of data.
Linear algebra is essentially the mathematics of data.
It provides useful shortcuts for describing data as well as operations on data that we need to perform in machine learning methods.
Linear algebra is not magic
And, linear algebra is not trying to be exclusive or opaque.
As a first step, think of linear algebra as a shortcut language or notation to make describing some operations compact.
There are common machine learning techniques that can only be understood via their linear algebra descriptions because they come from the field. Techniques such as:
Singular-Value Decomposition, or SVD.
Principal Component Analysis.
Linear Least Squares for Linear Regression.
Without a basic understanding of matrices and matrix operations, an understanding of these techniques will elude you.
The 3 Mistakes Made By Beginners
Once you discover the importance of linear algebra to machine learning, there are three key mistakes that beginners make:
1. Beginners Study Linear Algebra Too Early
If you ask how to get started in machine learning, you will very likely be told to start with linear algebra.
We know that knowledge of linear algebra is critically important, but it does not have to be the place to start. Learning linear algebra first, then calculus, probability, statistics, and eventually machine learning theory is a long and slow bottom-up path.
A better fit for developers is to start with systematic procedures that get results, and work back to the deeper understanding of theory, using working results as a context. I call this the top-down, or results-first, approach to machine learning, and linear algebra is not the first step, but perhaps the second or third.
2. Beginners Study Too Much Linear Algebra
When practitioners do circle back to study linear algebra, they learn far more of the field than is required for or relevant to machine learning.
Linear algebra is a large field of study that has tendrils into engineering, physics, and quantum physics. There are also theorems and derivations for nearly everything, most of which will not help you get better skill from, or a deeper understanding of, your machine learning model.
Only a specific subset of linear algebra is required, though you can always go deeper once you have the basics.
3. Beginners Study Linear Algebra Wrong
Linear algebra textbooks will teach you linear algebra in the classical university bottom-up approach. This is too slow (and painful) for your needs as a machine learning practitioner.
Like learning machine learning itself, take the top-down approach. Rather than starting with theorems and abstract concepts, you can learn the basics of linear algebra in a concrete way with data structures and worked examples of operations on those data structures. It’s so much faster (and fun).
Once you know how operations work, you can circle back and learn how they were derived. If you’re interested.
A Better Way into Linear Algebra
I am frustrated at seeing practitioner after practitioner diving into linear algebra textbooks and online courses designed for undergraduate students and giving up. The bottom-up approach is hard, especially if you already have a full time job.
Linear algebra is not only important to machine learning, but it is also a lot of fun, or can be if it is approached in the right way.
I want to help you see the field the way I see it: as just another set of tools we can harness on our journey toward machine learning mastery.
5 Areas of Linear Algebra to Focus On
You don’t need to know all of linear algebra.
The five key areas of linear algebra that I recommend you focus on are:
1. Learn Linear Algebra Notation
You need to be able to read and write vector and matrix notation.
Algorithms are described in books, papers, and on websites using vector and matrix notation.
Linear algebra is the mathematics of data and the notation allows you to describe operations on data precisely with specific operators.
You need to be able to read and write this notation.
2. Learn Linear Algebra Arithmetic
In partnership with the notation of linear algebra are the arithmetic operations performed.
You need to know how to add, subtract, and multiply scalars, vectors, and matrices.
A challenge for newcomers to the field of linear algebra are operations such as matrix multiplication and tensor multiplication that are not implemented as the direct multiplication of the elements of these structures, and at first glance appear nonintuitive.
3. Learn Linear Algebra for Statistics
You must learn linear algebra in order to be able to learn statistics. Especially multivariate statistics.
Statistics is concerned with describing and understanding data. As the mathematics of data, linear algebra has left its fingerprint on many related fields of mathematics, including statistics.
In order to be able to read and interpret statistics, you must learn the notation and operations of linear algebra.
4. Learn Matrix Factorization
Building on notation and arithmetic is the idea of matrix factorization, also called matrix decomposition.
You need to know how to factorize a matrix and what it means.
Matrix factorization is a key tool in linear algebra and used widely as an element of many more complex operations in both linear algebra (such as the matrix inverse) and machine learning (least squares, PCA, SVD, and more).
5. Learn Linear Least Squares
You need to know how to use matrix factorization to solve linear least squares.
Linear algebra was originally developed to solve systems of linear equations. These are equations where there are more equations than there are unknown variables. As a result, they are challenging to solve arithmetically because there is no single solution as there is no line or plane that can fit the data without some error.
Problems of this type can be framed as the minimization of squared error, called least squares, and can be recast in the language of linear algebra, called linear least squares.
Bonus Reason
If I could give one more reason, it would be: because it is fun.
Seriously.
Learning linear algebra, at least the way I teach it with practical examples and executable code, is a lot of fun. Once you can see how the operations work on real data, it is hard to avoid developing a strong intuition for the methods.