Metrics / distance functions

In words...

A metric is a function used to measure the distance between two mathematical objects, such as vectors or functions.

A set of objects equipped with a metric is a metric space.

Various types of metrics can be defined depending on the particular topology we want to induce on the space. The most common metric is the Euclidean distance defined between points of a Euclidean space.

Under certain conditions, a metric defined on a vector space can give rise to a norm computed as the distance between a vector and the origin.

In pictures...

In maths...

A metric $\rho$ is a function from a set $\X$ to the set of positive real numbers $[0,+\infty)$ which satisfies, for all $x$, $x^\prime$ and $x^{\prime\prime}$ in $\X$,

  1. (identity of indiscernibles) $\rho(x, x^\prime) = 0$ if and only if $x = x^\prime$,
  2. (symmetry) $\rho(x, x^\prime) = \rho(x^\prime, x)$,
  3. (triangle inequality) $\rho(x, x^\prime) \leq \rho(x, x^{\prime\prime}) + \rho(x^{\prime\prime}, x^\prime)$ .

If, in addition, the metric satisfies

  1. (translation invariance) $\forall a\in\X$, $\rho(x + a, x^\prime + a) = \rho(x, x^\prime)$
  2. (homogeneity) $\forall \lambda\in \R$, $\rho(\lambda x, \lambda x^\prime) = |\lambda| \rho(x, x^\prime)$
then we can define a norm on $\X$ as the distance to the origin: $$ \|x\| = \rho(x, 0) . $$

Metrics in $\R^d$

The most common metric in $\R^d$ is the Euclidean distance $$ \rho(\g x, \g x^\prime) = \|\g x - \g x^\prime\|_2 = \sqrt{\sum_{i=1}^{d} (x_i - x_i^\prime)^2 } $$ Other metrics can be defined by changing the Euclidean norm for another $\ell_p$-norm.

Metrics in function spaces

Metrics for function spaces can be defined similarly from the $L^p$-norms. For instance, for real-valued functions $f$ and $g$, we can define $$ \rho(f,g) = \|f - g\|_{L^p} = \left(\int_{\X} \left|f(x) - g(x)\right|^p \ dx \right)^{\frac{1}{p}}. $$

Pseudo-metrics

A pseudo-metric is similar to a metric except that it only satisfies a relaxed version of the identity of indiscernibles ($\rho(x, x^\prime) = 0$ if and only if $x = x^\prime$), nameley, that for all $x\in\X$, $\rho(x, x) = 0$, but $$ \rho(x, x^\prime) = 0\ \not\Rightarrow x = x^\prime. $$ Pseudo-metrics are often used to measure the distance between two functions when considering only their value differences at a sample of points.
For instance, the $\g{\ell_{\infty}}$ pseudo-metric can be defined for a function class $\F \subseteq \R^{\X}$ and a sequence of points $\g x_N = \{x_i\}_{i=1}^N \in \X^N$ as $$ \forall (f,f^\prime) \in \F^2,\quad d_{\ell_{\infty}(\g x_N)}(f,f^\prime) = \max_{i\in\{1,\dots,N\}} |f(x_i) - f^\prime(x_i) |. $$ i.e., as $d_{\ell_{\infty}(\g x_N)}(f,f^\prime) = \|\g f - \g f^\prime\|_{\infty}$, where $\g f=[f(x_1,\dots, f(x_N)]^T$ and $\g f^\prime=[f^\prime(x_1,\dots, f^\prime(x_N)]^T$.