Machine learning is the study of making machines learn concepts
without explicitly programming them. Concretely, we want algorithms that
can
learn from input data and make predictions, which
is supervised learning, and
find interesting patterns in data, which is
unsupervised learning.
In data science, machine learning shows up in a huge range of
applications. This book does not try to cover them one by one. Instead,
we will
formulate machine learning problems at an abstract level, so that
one framework applies across many applications;
study families of algorithms that have been successful in machine
learning tasks; and
systematically analyze how good an algorithm is for a given
task.
Abstraction
Many problems that look different on the surface have the same
shape:
Task
Input
Output
Spam filtering
emails
{spam, not spam}
Object recognition
photos
{building, tree, road, car, sky, …}
Handwritten digit recognition
images of handwritten digits
{0, 1, 2, …, 9}
In each case we encode the input as a vector \(x = (x^{(1)}, \dots, x^{(d)}) \in
\mathcal{X}\), and our goal is to learn a function \(f\) that maps it to an output \(y \in \mathcal{Y}\).
Assumption: there is a (relatively simple) function
\(f^* : \mathcal{X} \to \mathcal{Y}\)
such that \(f^*(x_i) = y_i\) for most
\(i\).
Learning task: given \(n\) examples, find an approximation \(\hat f \approx f^*\).
Goal:\(\hat f\)
gives mostly correct predictions on unseen examples.
This splits into two stages. In training, labeled
training data goes into a learning algorithm, which produces a trained
model \(\hat f\). In
testing, we apply \(\hat
f\) to unlabeled, unseen data to make predictions.
Every learning algorithm in this book makes two choices:
Before training, decide on a pool of models \(\mathcal{F}\): for example, linear
or nonlinear models, parametric or nonparametric models.
Select \(\hat f\) from
\(\mathcal{F}\): for example,
by maximum likelihood (which draws on probability and statistics), or by
optimizing a loss criterion (which draws on linear algebra, calculus and
convex optimization).
Keep these two questions in mind for every method we meet: what
is the pool of models, and how do we select one from it?
Assumption: there is an underlying structure in
\(\mathcal{X}\).
Learning task: discover that structure from \(n\) examples.
Goal: summarize the data using the discovered
structure.
How this book is organized
The book follows the course’s four modules:
Regression: modeling and predicting continuous
outcomes.
Classification: modeling and predicting categorical
outcomes.
Unsupervised learning: discovering hidden structure
in unlabeled data.
Deep learning: training neural networks on large
datasets to perform regression, classification, unsupervised learning
and more.
You will get the most out of it with some background in linear
algebra, calculus and probability. We review the linear algebra we need
as we go, starting in Chapter 1.1.