Markov chains on a measurable state space

For any measure ${\displaystyle \mu \colon \Sigma \to [0,\infty ]}$ we denote for ${\displaystyle \mu }$-integrable function ${\displaystyle f\colon E\to \mathbb {R} \cup \{\infty ,-\infty \}}$ the Lebesgue integral as ${\displaystyle \int _{E}f(x)\,\mu (dx)}$. For the measure ${\displaystyle \nu _{x}\colon \Sigma \to [0,\infty ]}$ defined by ${\displaystyle \nu _{x}(A):=p(x,A)}$ we used the following notation:

${\displaystyle \int _{E}f(y)\,p(x,dy):=\int _{E}f(y)\,\nu _{x}(dy).}$

If ${\displaystyle \mu }$ is a Dirac measure in ${\displaystyle x}$, we denote for a Markov kernel ${\displaystyle p}$ with starting distribution ${\displaystyle \mu }$ the associated Markov chain as ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ on ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} _{x})}$ and the expectation value

${\displaystyle \mathbb {E} _{x}[X]=\int _{\Omega }X(\omega )\,\mathbb {P} _{x}(d\omega )}$

for a ${\displaystyle \mathbb {P} _{x}}$-integrable function ${\displaystyle X}$. By definition, we have then ${\displaystyle \mathbb {P} _{x}[X_{0}=x]=1}$.

We have for any measurable function ${\displaystyle f\colon E\to [0,\infty ]}$ the following relation:[4]

${\displaystyle \int _{E}f(y)\,p(x,dy)=\mathbb {E} _{x}[f(X_{1})].}$

For a Markov kernel ${\displaystyle p}$ with starting distribution ${\displaystyle \mu }$ one can introduce a family of Markov kernels ${\displaystyle (p_{n})_{n\in \mathbb {N} }}$ by

${\displaystyle p_{n+1}(x,A):=\int _{E}p_{n}(y,A)\,p(x,dy)}$

for ${\displaystyle n\in \mathbb {N} ,\,n\geq 1}$ and ${\displaystyle p_{1}:=p}$. For the associated Markov chain ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ according to ${\displaystyle p}$ and ${\displaystyle \mu }$ one obtains

${\displaystyle \mathbb {P} [X_{0}\in A,\,X_{n}\in B]=\int _{A}p_{n}(x,B)\,\mu (dx)}$.

A probability measure ${\displaystyle \mu }$ is called stationary measure of a Markov kernel ${\displaystyle p}$ if

${\displaystyle \int _{A}\mu (dx)=\int _{E}p(x,A)\,\mu (dx)}$

holds for any ${\displaystyle A\in \Sigma }$. If ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ on ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} )}$ denotes the Markov chain according to a Markov kernel ${\displaystyle p}$ with stationary measure ${\displaystyle \mu }$, and the distribution of ${\displaystyle X_{0}}$ is ${\displaystyle \mu }$, then all ${\displaystyle X_{n}}$ have the same probability distribution, namely:

${\displaystyle \mathbb {P} [X_{n}\in A]=\mu (A)}$

for any ${\displaystyle A\in \Sigma }$.

A Markov kernel ${\displaystyle p}$ is called reversible according to a probability measure ${\displaystyle \mu }$ if

${\displaystyle \int _{A}p(x,B)\,\mu (dx)=\int _{B}p(x,A)\,\mu (dx)}$

holds for any ${\displaystyle A,B\in \Sigma }$. Replacing ${\displaystyle A=E}$ shows that if ${\displaystyle p}$ is reversible according to ${\displaystyle \mu }$, then ${\displaystyle \mu }$ must be a stationary measure of ${\displaystyle p}$.