A classification problem is said to be multi-class when the number of categories is 3 or more. This distinction between binary (with two categories) and multi-class problems is important due to the fact that many classification algorithms are devised for the binary case and cannot directly handle multi-class problems, such as the perceptron or support vector machines.
In order to tackle a multi-class problem with a binary algorithm, one typically resorts to one of the three following decomposition methods: the one-versus-all approach, the one-versus-one approach or the approach based on a directed acyclic graph.
There are also some algorithms that can deal with multi-class problems without relying on such decomposition techniques, such as K-nearest neighbors, logistic regression, linear discriminant analysis, the naive Bayes classifier, multi-layer perceptrons or multi-class support vector machines.