Independent and identically distributed random variables

Suppose that the random variables ${\displaystyle X}$ and ${\displaystyle Y}$ are defined to assume values in ${\displaystyle I\subseteq \mathbb {R} }$. Let ${\displaystyle F_{X}(x)=\operatorname {P} (X\leq x)}$ and ${\displaystyle F_{Y}(y)=\operatorname {P} (Y\leq y)}$ be the cumulative distribution functions of ${\displaystyle X}$ and ${\displaystyle Y}$, respectively, and denote their joint cumulative distribution function by ${\displaystyle F_{X,Y}(x,y)=\operatorname {P} (X\leq x\land Y\leq y)}$.

Two random variables ${\displaystyle X}$ and ${\displaystyle Y}$ are identically distributed if and only if[5] ${\displaystyle F_{X}(x)=F_{Y}(x)\,\forall x\in I}$.

Two random variables ${\displaystyle X}$ and ${\displaystyle Y}$ are independent if and only if ${\displaystyle F_{X,Y}(x,y)=F_{X}(x)\cdot F_{Y}(y)\,\forall x,y\in I}$. (See further Independence (probability theory) § Two random variables.)

Two random variables ${\displaystyle X}$ and ${\displaystyle Y}$ are i.i.d. if they are independent and identically distributed, i.e. if and only if

${\displaystyle {\begin{aligned}&F_{X}(x)=F_{Y}(x)\,&\forall x\in I\\&F_{X,Y}(x,y)=F_{X}(x)\cdot F_{Y}(y)\,&\forall x,y\in I\end{aligned}}}$

(Eq.1)

The definition extends naturally to more than two random variables. We say that ${\displaystyle n}$ random variables ${\displaystyle X_{1},\ldots ,X_{n}}$ are i.i.d. if they are independent (see further Independence (probability theory)#More than two random variables) and identically distributed, i.e. if and only if

${\displaystyle {\begin{aligned}&F_{X_{1}}(x)=F_{X_{k}}(x)\,&\forall k\in \{1,\ldots ,n\}{\text{ and }}\forall x\in I\\&F_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=F_{X_{1}}(x_{1})\cdot \ldots \cdot F_{X_{n}}(x_{n})\,&\forall x_{1},\ldots ,x_{n}\in I\end{aligned}}}$

(Eq.2)

where ${\displaystyle F_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=\operatorname {P} (X_{1}\leq x_{1}\land \ldots \land X_{n}\leq x_{n})}$ denotes the joint cumulative distribution function of ${\displaystyle X_{1},\ldots ,X_{n}}$.

The most general notion which shares the main properties of i.i.d. variables are exchangeable random variables, introduced by Bruno de Finetti. Exchangeability means that while variables may not be independent, future ones behave like past ones – formally, any value of a finite sequence is as likely as any permutation of those values – the joint probability distribution is invariant under the symmetric group.

This provides a useful generalization – for example, sampling without replacement is not independent, but is exchangeable.

In stochastic calculus, i.i.d. variables are thought of as a discrete time Lévy process: each variable gives how much one changes from one time to another. For example, a sequence of Bernoulli trials is interpreted as the Bernoulli process. One may generalize this to include continuous time Lévy processes, and many Lévy processes can be seen as limits of i.i.d. variables—for instance, the Wiener process is the limit of the Bernoulli process.