Introduction

What is machine learning?

Machine learning is the study of making machines learn concepts without explicitly programming them. Concretely, we want algorithms that can

In data science, machine learning shows up in a huge range of applications. This book does not try to cover them one by one. Instead, we will

Abstraction

Many problems that look different on the surface have the same shape:

Task Input Output
Spam filtering emails {spam, not spam}
Object recognition photos {building, tree, road, car, sky, …}
Handwritten digit recognition images of handwritten digits {0, 1, 2, …, 9}

In each case we encode the input as a vector \(x = (x^{(1)}, \dots, x^{(d)}) \in \mathcal{X}\), and our goal is to learn a function \(f\) that maps it to an output \(y \in \mathcal{Y}\).

Supervised learning

This splits into two stages. In training, labeled training data goes into a learning algorithm, which produces a trained model \(\hat f\). In testing, we apply \(\hat f\) to unlabeled, unseen data to make predictions.

Every learning algorithm in this book makes two choices:

  1. Before training, decide on a pool of models \(\mathcal{F}\): for example, linear or nonlinear models, parametric or nonparametric models.
  2. Select \(\hat f\) from \(\mathcal{F}\): for example, by maximum likelihood (which draws on probability and statistics), or by optimizing a loss criterion (which draws on linear algebra, calculus and convex optimization).

Keep these two questions in mind for every method we meet: what is the pool of models, and how do we select one from it?

Unsupervised learning

How this book is organized

The book follows the course’s four modules:

You will get the most out of it with some background in linear algebra, calculus and probability. We review the linear algebra we need as we go, starting in Chapter 1.1.