Deterministic VS stochastic framework

In words...

The deterministic case occurs when there is nothing random in the outcome of the experiment. For supervised learning, this means that given an input pattern, the label can take only one value. In other words, the label is entirely determined by the input pattern.

The stochastic case refers to the case where there is randomness in the outcome. For supervised learning, this means that given an input pattern, there can be multiple more or less probable values for the label. In other words, the input pattern does not contain enough information to fully determine the label.

In pictures...

A deterministic example

Let's determine whether a fruit is an apple or a banana based on its length (or width).

A stochastic example

Now let's try to determine whether a fruit is an apple or a banana but this time based only on its color.

In maths...

The deterministic case is characterized by the existence of a target function $t : \X\rightarrow \Y$ such that $$ \forall X\in\X,\quad Y = t(X), $$ in which case, we have $$ \forall X\in\X, \quad P(Y=t(X)\ |\ X=x) = 1 $$ and thus a minimal risk at zero: $$ R(t) = \E_{X,Y} [\ell(t(X),Y) ] = 0 $$ for any loss function satsifying $\ell(y,y) = 0$.

At the contrary, in the stochastic case, such a function $t$ does not exist and the risk typically cannot be zero.