Machine Learning

In words...

Machine learning is a field at the intersection of applied mathematics (statistics, optimization) and computer science. It is mostly concerned with the way computers can "learn" to predict things from examples.

By learning, we mean that the computer need not be explicitly programed to perform the prediction task. Thus, machine learning is particularly useful to tackle problems with solutions that can hardly be written as standard algorithms.

For instance, we would like a computer to distinguish between pictures of apples and of bananas. This is an extremely difficult problem, since two apples that look similar to humans may appear very differently in the computer memory, depending on their precise specifications, the lighting conditions and so on... So if we were to produce an algorithm to recognize pictures of fruits, an overwhelming number of cases would have to be explicited.

For such problems, we will instead consider a machine learning approach: we will write algorithms that make the computer learn its own definition of fruits after seeing a significant number of pictures of them. If the computer learns from a teacher or supervisor who labels each picture with the correct word (apple or banana), then we perform supervised learning. If the computer has to guess the labels on its own, then this is called unsupervised learning.

These two types of problems are rather different. In unsupervised learning, we have a set of input data from which we would like the machine to extract some knowledge, for instance to answer questions like "how are the data distributed?" or "can they be divided into consistent subgroups?". On the other hand, in supervised learning, we assume that we already know something about the data, such as their classification into subgroups, and we would like the machine to generalize this knowledge to other data for which we do not have the labels.

Note that, beyond the classification of fruits, machine learning also has serious applications in many different areas.

In pictures...

A problem with no (simple) explicit solution

Use the mouse to write/draw a letter in the first box on the left.

Can we write an algorithm that recognize this letter, i.e., a piece of code that tells us whether the picture represent this letter or not?

The answer is yes, and this is not difficult: it suffices to test every pixels one-by-one to see if they match the ones in the picture. However, what is much more difficult (and probably impossible) is to write down such an algorithm that would work as well on a second picture. To see why, draw the same letter again in the second box on the right. You can see that despite looking the same to us, it looks very different in terms of pixel values for the computer.

In maths...

Machine learning is concerned with obtaining a mapping from the input space $\X$ to the space of labels $\Y$ from data in the case where the desired mapping cannot be explicited as a simple algorithm.

If the available data contains only examples from $\X$, then this is unsupervised learning. If for each datum from $\X$ the corresponding label in $\Y$ is also given, then this is supervised learning.