Ergodic process

In econometrics and signal processing, a stochastic process is said to be ergodic if its statistical properties can be deduced from a single, sufficiently long, random sample of the process. The reasoning is that any collection of random samples from a process must represent the average statistical properties of the entire process. In other words, regardless of what the individual samples are, a birds-eye view of the collection of samples must represent the whole process. Conversely, a process that is not ergodic is a process that changes erratically at an inconsistent rate.[1]

Specific definitions

One can discuss the ergodicity of various statistics of a stochastic process. For example, a wide-sense stationary process $X(t)$ has constant mean

\mu _{X}=E[X(t)]

,

and autocovariance

r_{X}(\tau )=E[(X(t)-\mu _{X})(X(t+\tau )-\mu _{X})]

,

that depends only on the lag $\tau$ and not on time $t$ . The properties $\mu _{X}$ and $r_{X}(\tau )$ are ensemble averages not time averages.

The process $X(t)$ is said to be mean-ergodic[2] or mean-square ergodic in the first moment[3] if the time average estimate

{\hat {\mu }}_{X}={\frac {1}{T}}\int _{0}^{T}X(t)\,dt

converges in squared mean to the ensemble average $\mu _{X}$ as $T\rightarrow \infty$ .

Likewise, the process is said to be autocovariance-ergodic or d moment[3] if the time average estimate

{\hat {r}}_{X}(\tau )={\frac {1}{T}}\int _{0}^{T}[X(t+\tau )-\mu _{X}][X(t)-\mu _{X}]\,dt

converges in squared mean to the ensemble average $r_{X}(\tau )$ , as $T\rightarrow \infty$ . A process which is ergodic in the mean and autocovariance is sometimes called ergodic in the wide sense.

Discrete-time random processes

The notion of ergodicity also applies to discrete-time random processes $X[n]$ for integer $n$ .

A discrete-time random process $X[n]$ is ergodic in mean if

{\hat {\mu }}_{X}={\frac {1}{N}}\sum _{n=1}^{N}X[n]

converges in squared mean to the ensemble average $E[X]$ , as $N\rightarrow \infty$ .

Examples

Ergodicity means the ensemble average equals the time average. Following are examples to illustrate this principle.

Call centre

Each operator in a call centre spends time alternately speaking and listening on the telephone, as well as taking breaks between calls. Each break and each call are of different length, as are the durations of each 'burst' of speaking and listening, and indeed so is the rapidity of speech at any given moment, which could each be modelled as a random process.

Take N call centre operators (N should be a very large integer) and plot the number of words spoken per minute for each operator over a long period (several shifts). For each operator you will have a series of points, which could be joined with lines to create a 'waveform'.
Calculate the average value of those points in the waveform; this gives you the time average.
There are N waveforms and N operators. These N waveforms are known as an ensemble.
Now take a particular instant of time in all those waveforms and find the average value of the number of words spoken per minute. That gives you the ensemble average for that instant.
If ensemble average always equals time average, then the system is ergodic.

Electronics

Each resistor has an associated thermal noise that depends on the temperature. Take N resistors (N should be very large) and plot the voltage across those resistors for a long period. For each resistor you will have a waveform. Calculate the average value of that waveform; this gives you the time average. There are N waveforms as there are N resistors. These N plots are known as an ensemble. Now take a particular instant of time in all those plots and find the average value of the voltage. That gives you the ensemble average for each plot. If ensemble average and time average are the same then it is ergodic.

Examples of non-ergodic random processes

An unbiased random walk is non-ergodic. Its expectation value is zero at all times, whereas its time average is a random variable with divergent variance.

Suppose that we have two coins: one coin is fair and the other has two heads. We choose (at random) one of the coins first, and then perform a sequence of independent tosses of our selected coin. Let X[n] denote the outcome of the nth toss, with 1 for heads and 0 for tails. Then the ensemble average is ¹⁄₂ (¹⁄₂ + 1) = ³⁄₄; yet the long-term average is ¹⁄₂ for the fair coin and 1 for the two-headed coin. So the long term time-average is either 1/2 or 1. Hence, this random process is not ergodic in mean.

Notes

Originally due to L. Boltzmann. See part 2 of Vorlesungen über Gastheorie. Leipzig: J. A. Barth. 1898. OCLC 01712811. ('Ergoden' on p.89 in the 1923 reprint.) It was used to prove equipartition of energy in the kinetic theory of gases
Papoulis, p.428
Porat, p.14

References

Porat, B. (1994). Digital Processing of Random Signals: Theory & Methods. Prentice Hall. p. 14. ISBN 0-13-063751-3.
Papoulis, Athanasios (1991). Probability, random variables, and stochastic processes. New York: McGraw-Hill. pp. 427–442. ISBN 0-07-048477-5.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Originally due to L. Boltzmann. See part 2 of Vorlesungen über Gastheorie. Leipzig: J. A. Barth. 1898. OCLC 01712811. ('Ergoden' on p.89 in the 1923 reprint.) It was used to prove equipartition of energy in the kinetic theory of gases

[2] Papoulis, p.428

[porat-3] Porat, p.14