The goal for “The Flow of Tensors” series is to allow us understand how to build machine learning apps using the popular TensorFlow library.
But the journey of every traveller always has a beginning. And for this journey, our beginning is a small part of mathematics known as Linear Algebra.
Some people say Linear Algebra is the language Machine Learning was written in. Some say Calculus and Probability are also very inclusive. I agree with all of them.
Linear Algebra is a key concept in Machine Learning. It’s been more than once described as the “mathematics of data”.
It is the branch of continuous mathematics that deals with performing arithmetic on columns of numbers called Vectors and arrays of numbers called Matrices, to create new columns and arrays of numbers.
Note however, that vectors and matrices are typical cases of a more general entity known as Tensors.
So, the above definition of linear algebra can be reshuffled as the branch of continuous mathematics that deals with performing arithmetic on tensors to create new tensors.
Scalars
Scalars are zero-order tensors and is a single number, e.g -3.88.
Vectors
Vectors are first-order tensors, and are 1D arrays, that can either be a row or column.
$$ \begin{bmatrix}x_1 … x_n\end{bmatrix} $$
For instance, the vector below is one column: $$ \begin{bmatrix}3\\1\\4\end{bmatrix} $$
Matrices
Matrices are second-order tensors, 2D arrays, and could be vectors or arrays of vectors, containing both rows and columns.
$$ \begin{bmatrix}x_1{_1}&.&.&x_1{_j}\\ .&.&.&.\\ .&.&.&.\\ x_i{_1}&.&.&x_i{_j}\end{bmatrix} $$
$$ \begin{bmatrix} 2&3&1 \\ 3&1&4 \\ 0&4&5\\ 2&2&1 \end{bmatrix} $$
In the construct of a matrix, (x_i{_j}) represents the element in the (i)th row and (j)th column of the matrix.
Hence, (x_3{_2} = 4)
Note that we can also define vectors as subsets of matrices. So, it’s fine to say that the vector above is a 3 x 1 matrix.
Continue with “The Flow of Tensors” series in [The Flow of Tensors] — Linear Algebra for Machine Learning (Part 2 of 4).