Dot/inner product

In words...

An inner product is a bilinear function of two vectors returning a real number that represents the level of alignement of these vectors.

The most common inner product is the Euclidean standard dot product for Euclidean spaces.

In pictures...

In maths...

In a vector space $\X$, an inner product $\inner{\cdot}{\cdot}_{\X}$ (or simply $\inner{\cdot}{\cdot}$ when the vector space is clear from context) is a function from $\X^2$ to $\R$ which satisfies, for all $(u,v) \in\X^2$,

  1. (Symmetry) $\inner{u}{v} = \inner{v}{u}$
  2. (Linearity in the first argument) $\inner{\lambda u}{v} = \lambda \inner{u}{v}$ for all scalars $\lambda\in\R$
  3. (Linearity in the first argument) $\inner{u+v}{w} = \inner{u}{w} + \inner{v}{w}$ for all vector $w\in\X$
  4. (Positive semi-definiteness) $\inner{u}{u} \geq 0$
  5. (Positive definiteness) $\inner{u}{u} = 0 \ \Rightarrow\ u=0$

Any inner product induces a norm as $$ \|x\| = \sqrt{\inner{x}{x}}. $$

Euclidean dot product in $\R^d$

The standard dot product in Euclidean space $\R^d$, $$ \inner{\g u}{\g v} = \g u^T \g v = \sum_{i=1}^d u_iv_i, $$ is an inner product.

Inner product of functions

The standard inner product in the Hilbert space $L^2(\X)$ of square integrable functions $f : \X \rightarrow \R$ is defined as $$ \inner{f}{g}_{L^2(\X)} = \int_{\X} f(\g x)g(\g x) \ d\g x . $$

Cauchy-Schwarz inequality

Inner products satisfy the Cauchy-Schwarz inequality: for all $(u,v) \in\X^2$, $$ \left|\inner{u}{v}_{\X} \right|^2 \leq \inner{u}{u}_{\X} \inner{v}{v}_{\X} , $$ or, with the norm $\|\cdot\|_{\X}$ induced by the inner product as above, $$ \left|\inner{u}{v}_{\X} \right| \leq \|u\|_{\X} \|v\|_{\X} . $$