Median

Formally, a median of a population is any value such that at most half of the population is less than the proposed median and at most half is greater than the proposed median. As seen above, medians may not be unique. If each set contains less than half the population, then some of the population is exactly equal to the unique median.

The median is well-defined for any ordered (one-dimensional) data, and is independent of any distance metric. The median can thus be applied to classes which are ranked but not numerical (e.g. working out a median grade when students are graded from A to F), although the result might be halfway between classes if there is an even number of cases.

A geometric median, on the other hand, is defined in any number of dimensions. A related concept, in which the outcome is forced to correspond to a member of the sample, is the medoid.

There is no widely accepted standard notation for the median, but some authors represent the median of a variable x either as x͂ or as μ_1/2[1] sometimes also M.[3][4] In any of these cases, the use of these or other symbols for the median needs to be explicitly defined when they are introduced.

The median is a special case of other ways of summarising the typical values associated with a statistical distribution: it is the 2nd quartile, 5th decile, and 50th percentile.

The median can be used as a measure of location when one attaches reduced importance to extreme values, typically because a distribution is skewed, extreme values are not known, or outliers are untrustworthy, i.e., may be measurement/transcription errors.

For example, consider the multiset

1, 2, 2, 2, 3, 14.

The median is 2 in this case, (as is the mode), and it might be seen as a better indication of the center than the arithmetic mean of 4, which is larger than all-but-one of the values. However, the widely cited empirical relationship that the mean is shifted "further into the tail" of a distribution than the median is not generally true. At most, one can say that the two statistics cannot be "too far" apart; see § Inequality relating means and medians below.[5]

As a median is based on the middle data in a set, it is not necessary to know the value of extreme results in order to calculate it. For example, in a psychology test investigating the time needed to solve a problem, if a small number of people failed to solve the problem at all in the given time a median can still be calculated.[6]

Because the median is simple to understand and easy to calculate, while also a robust approximation to the mean, the median is a popular summary statistic in descriptive statistics. In this context, there are several choices for a measure of variability: the range, the interquartile range, the mean absolute deviation, and the median absolute deviation.

For practical purposes, different measures of location and dispersion are often compared on the basis of how well the corresponding population values can be estimated from a sample of data. The median, estimated using the sample median, has good properties in this regard. While it is not usually optimal if a given population distribution is assumed, its properties are always reasonably good. For example, a comparison of the efficiency of candidate estimators shows that the sample mean is more statistically efficient when — and only when — data is uncontaminated by data from heavy-tailed distributions or from mixtures of distributions. Even then, the median has a 64% efficiency compared to the minimum-variance mean (for large normal samples), which is to say the variance of the median will be ~50% greater than the variance of the mean.[7][8]

The medians of certain types of distributions can be easily calculated from their parameters; furthermore, they exist even for some distributions lacking a well-defined mean, such as the Cauchy distribution:

• The median of a symmetric unimodal distribution coincides with the mode.
• The median of a symmetric distribution which possesses a mean μ also takes the value μ.
  • The median of a normal distribution with mean μ and variance σ² is μ. In fact, for a normal distribution, mean = median = mode.
  • The median of a uniform distribution in the interval [a, b] is (a + b) / 2, which is also the mean.
• The median of a Cauchy distribution with location parameter x₀ and scale parameter y is x₀, the location parameter.
• The median of a power law distribution x^−a, with exponent a > 1 is 2^{1/(a − 1)}x_min, where x_min is the minimum value for which the power law holds[10]
• The median of an exponential distribution with rate parameter λ is the natural logarithm of 2 divided by the rate parameter: λ⁻¹ln 2.
• The median of a Weibull distribution with shape parameter k and scale parameter λ is λ(ln 2)^1/k.

The mean absolute error of a real variable c with respect to the random variable X is

${\displaystyle E(\left|X-c\right|)\,}$

Provided that the probability distribution of X is such that the above expectation exists, then m is a median of X if and only if m is a minimizer of the mean absolute error with respect to X.[11] In particular, m is a sample median if and only if m minimizes the arithmetic mean of the absolute deviations.[12]

More generally, a median is defined as a minimum of

${\displaystyle E(|X-c|-|X|),}$

as discussed below in the section on multivariate medians (specifically, the spatial median).

This optimization-based definition of the median is useful in statistical data-analysis, for example, in k-medians clustering.

Comparison of mean, median and mode of two log-normal distributions with different skewness

If the distribution has finite variance, then the distance between the median ${\displaystyle {\tilde {X}}}$ and the mean ${\displaystyle {\bar {X}}}$ is bounded by one standard deviation.

This bound was proved by Mallows,[13] who used Jensen's inequality twice, as follows. Using |·| for the absolute value, we have

${\displaystyle {\begin{aligned}|\mu -m|=|\operatorname {E} (X-m)|&\leq \operatorname {E} (|X-m|)\\&\leq \operatorname {E} (|X-\mu |)\\&\leq {\sqrt {\operatorname {E} \left((X-\mu )^{2}\right)}}=\sigma .\end{aligned}}}$

The first and third inequalities come from Jensen's inequality applied to the absolute-value function and the square function, which are each convex. The second inequality comes from the fact that a median minimizes the absolute deviation function ${\displaystyle a\mapsto \operatorname {E} (|X-a|)}$.

Mallows' proof can be generalized to obtain a multivariate version of the inequality[14] simply by replacing the absolute value with a norm:

${\displaystyle \|\mu -m\|\leq {\sqrt {\operatorname {E} \left(\|X-\mu \|^{2}\right)}}={\sqrt {\operatorname {trace} \left(\operatorname {var} (X)\right)}}}$

where m is a spatial median, that is, a minimizer of the function ${\displaystyle a\mapsto \operatorname {E} (\|X-a\|).\,}$ The spatial median is unique when the data-set's dimension is two or more.[15][16]

An alternative proof uses the one-sided Chebyshev inequality; it appears in an inequality on location and scale parameters. This formula also follows directly from Cantelli's inequality.[17]

For the case of unimodal distributions, one can achieve a sharper bound on the distance between the median and the mean:

${\displaystyle \left|{\tilde {X}}-{\bar {X}}\right|\leq \left({\frac {3}{5}}\right)^{\frac {1}{2}}\sigma \approx 0.7746\sigma }$.[18]

A similar relation holds between the median and the mode:

${\displaystyle \left|{\tilde {X}}-\mathrm {mode} \right|\leq 3^{\frac {1}{2}}\sigma \approx 1.732\sigma .}$

Even though comparison-sorting n items requires Ω(n log n) operations, selection algorithms can compute the kth-smallest of n items with only Θ(n) operations. This includes the median, which is the n/2th order statistic (or for an even number of samples, the arithmetic mean of the two middle order statistics).[20]

Selection algorithms still have the downside of requiring Ω(n) memory, that is, they need to have the full sample (or a linear-sized portion of it) in memory. Because this, as well as the linear time requirement, can be prohibitive, several estimation procedures for the median have been developed. A simple one is the median of three rule, which estimates the median as the median of a three-element subsample; this is commonly used as a subroutine in the quicksort sorting algorithm, which uses an estimate of its input's median. A more robust estimator is Tukey's ninther, which is the median of three rule applied with limited recursion:[21] if A is the sample laid out as an array, and

med3(A) = median(A[1], A[n/2], A[n]),

then

ninther(A) = med3(med3(A[1 ... 1/3n]), med3(A[1/3n ... 2/3n]), med3(A[2/3n ... n]))

The remedian is an estimator for the median that requires linear time but sub-linear memory, operating in a single pass over the sample.[22]

The distributions of both the sample mean and the sample median were determined by Laplace.[23] The distribution of the sample median from a population with a density function ${\displaystyle f(x)}$ is asymptotically normal with mean ${\displaystyle m}$ and variance[24]

${\displaystyle {\frac {1}{4nf(m)^{2}}}}$

where ${\displaystyle m}$ is the median of ${\displaystyle f(x)}$ and ${\displaystyle n}$ is the sample size. A modern proof follows below. Laplace's result is now understood as a special case of the asymptotic distribution of arbitrary quantiles.

For normal samples, the density is ${\displaystyle f(m)=1/{\sqrt {2\pi \sigma ^{2}}}}$, thus for large samples the variance of the median equals ${\displaystyle ({\pi }/{2})\cdot (\sigma ^{2}/n).}$[7] (See also section #Efficiency below.)

We take the sample size to be an odd number ${\displaystyle N=2n+1}$ and assume our variable continuous; the formula for the case of discrete variables is given below in § Empirical local density. The sample can be summarized as "below median", "at median", and "above median", which corresponds to a trinomial distribution with probabilities ${\displaystyle F(v-1)}$, ${\displaystyle f(v)}$ and ${\displaystyle 1-F(v)}$. For a continuous variable, the probability of multiple sample values being exactly equal to the median is 0, so one can calculate the density of at the point ${\displaystyle v}$ directly from the trinomial distribution:

${\displaystyle \Pr[\operatorname {Median} =v]\,dv={\frac {(2n+1)!}{n!n!}}F(v)^{n}(1-F(v))^{n}f(v)\,dv}$.

Now we introduce the beta function. For integer arguments ${\displaystyle \alpha }$ and ${\displaystyle \beta }$, this can be expressed as ${\displaystyle \mathrm {B} (\alpha ,\beta )={\frac {(\alpha -1)!(\beta -1)!}{(\alpha +\beta -1)!}}}$. Also, recall that ${\displaystyle f(v)\,dv=dF(v)}$. Using these relationships and setting both ${\displaystyle \alpha }$ and ${\displaystyle \beta }$ equal to ${\displaystyle n+1}$ allows the last expression to be written as

${\displaystyle {\frac {F(v)^{n}(1-F(v))^{n}}{\mathrm {B} (n+1,n+1)}}\,dF(v)}$

Hence the density function of the median is a symmetric beta distribution pushed forward by ${\displaystyle F}$. Its mean, as we would expect, is 0.5 and its variance is ${\displaystyle 1/(4(N+2))}$. By the chain rule, the corresponding variance of the sample median is

${\displaystyle {\frac {1}{4(N+2)f(m)^{2}}}}$.

The additional 2 is negligible in the limit.

In practice, the functions ${\displaystyle f}$ and ${\displaystyle F}$ are often not known or assumed. However, they can be estimated from an observed frequency distribution. In this section, we give an example. Consider the following table, representing a sample of 3,800 (discrete-valued) observations:

Because the observations are discrete-valued, constructing the exact distribution of the median is not an immediate translation of the above expression for ${\displaystyle \Pr(\operatorname {Median} =v)}$; one may (and typically does) have multiple instances of the median in one's sample. So we must sum over all these possibilities:

${\displaystyle \Pr(\operatorname {Median} =v)=\sum _{i=0}^{n}\sum _{k=0}^{n}{\frac {N!}{i!(N-i-k)!k!}}F(v-1)^{i}(1-F(v))^{k}f(v)^{N-i-k}}$

Here, i is the number of points strictly less than the median and k the number strictly greater.

Using these preliminaries, it is possible to investigate the effect of sample size on the standard errors of the mean and median. The observed mean is 3.16, the observed raw median is 3 and the observed interpolated median is 3.174. The following table gives some comparison statistics.

The expected value of the median falls slightly as sample size increases while, as would be expected, the standard errors of both the median and the mean are proportionate to the inverse square root of the sample size. The asymptotic approximation errs on the side of caution by overestimating the standard error.

The value of ${\displaystyle (2f(x))^{-2}}$—the asymptotic value of ${\displaystyle n^{-{\frac {1}{2}}}(\nu -m)}$ where ${\displaystyle \nu }$ is the population median—has been studied by several authors. The standard "delete one" jackknife method produces inconsistent results.[25] An alternative—the "delete k" method—where ${\displaystyle k}$ grows with the sample size has been shown to be asymptotically consistent.[26] This method may be computationally expensive for large data sets. A bootstrap estimate is known to be consistent,[27] but converges very slowly (order of ${\displaystyle n^{-{\frac {1}{4}}}}$).[28] Other methods have been proposed but their behavior may differ between large and small samples.[29]

The efficiency of the sample median, measured as the ratio of the variance of the mean to the variance of the median, depends on the sample size and on the underlying population distribution. For a sample of size ${\displaystyle N=2n+1}$ from the normal distribution, the efficiency for large N is

${\displaystyle {\frac {2}{\pi }}{\frac {N+2}{N}}}$

The efficiency tends to ${\displaystyle {\frac {2}{\pi }}}$ as ${\displaystyle N}$ tends to infinity.

In other words, the relative variance of the median will be ${\displaystyle \pi /2\approx 1.57}$, or 57% greater than the variance of the mean – the relative standard error of the median will be ${\displaystyle (\pi /2)^{\frac {1}{2}}\approx 1.25}$, or 25% greater than the standard error of the mean, ${\displaystyle \sigma /{\sqrt {n}}}$ (see also section #Sampling distribution above.).[30]

For univariate distributions that are symmetric about one median, the Hodges–Lehmann estimator is a robust and highly efficient estimator of the population median.[31]

If data are represented by a statistical model specifying a particular family of probability distributions, then estimates of the median can be obtained by fitting that family of probability distributions to the data and calculating the theoretical median of the fitted distribution. Pareto interpolation is an application of this when the population is assumed to have a Pareto distribution.

The marginal median is defined for vectors defined with respect to a fixed set of coordinates. A marginal median is defined to be the vector whose components are univariate medians. The marginal median is easy to compute, and its properties were studied by Puri and Sen.[31][35]

The geometric median of a discrete set of sample points ${\displaystyle x_{1},\ldots x_{N}}$ in a Euclidean space is the[lower-alpha 1] point minimizing the sum of distances to the sample points.

${\displaystyle {\hat {\mu }}={\underset {\mu \in \mathbb {R} ^{m}}{\operatorname {arg\,min} }}\sum _{n=1}^{N}\left\|\mu -x_{n}\right\|_{2}}$

In contrast to the marginal median, the geometric median is equivariant with respect to Euclidean similarity transformations such as translations and rotations.

An alternative generalization of the median in higher dimensions is the centerpoint.

When dealing with a discrete variable, it is sometimes useful to regard the observed values as being midpoints of underlying continuous intervals. An example of this is a Likert scale, on which opinions or preferences are expressed on a scale with a set number of possible responses. If the scale consists of the positive integers, an observation of 3 might be regarded as representing the interval from 2.50 to 3.50. It is possible to estimate the median of the underlying variable. If, say, 22% of the observations are of value 2 or below and 55.0% are of 3 or below (so 33% have the value 3), then the median ${\displaystyle m}$ is 3 since the median is the smallest value of ${\displaystyle x}$ for which ${\displaystyle F(x)}$ is greater than a half. But the interpolated median is somewhere between 2.50 and 3.50. First we add half of the interval width ${\displaystyle w}$ to the median to get the upper bound of the median interval. Then we subtract that proportion of the interval width which equals the proportion of the 33% which lies above the 50% mark. In other words, we split up the interval width pro rata to the numbers of observations. In this case, the 33% is split into 28% below the median and 5% above it so we subtract 5/33 of the interval width from the upper bound of 3.50 to give an interpolated median of 3.35. More formally, if the values ${\displaystyle f(x)}$ are known, the interpolated median can be calculated from

${\displaystyle m_{\text{int}}=m+w\left[{\frac {1}{2}}-{\frac {F(m)-{\frac {1}{2}}}{f(m)}}\right].}$

Alternatively, if in an observed sample there are ${\displaystyle k}$ scores above the median category, ${\displaystyle j}$ scores in it and ${\displaystyle i}$ scores below it then the interpolated median is given by

${\displaystyle m_{\text{int}}=m-{\frac {w}{2}}\left[{\frac {k-i}{j}}\right].}$

For univariate distributions that are symmetric about one median, the Hodges–Lehmann estimator is a robust and highly efficient estimator of the population median; for non-symmetric distributions, the Hodges–Lehmann estimator is a robust and highly efficient estimator of the population pseudo-median, which is the median of a symmetrized distribution and which is close to the population median.[37] The Hodges–Lehmann estimator has been generalized to multivariate distributions.[38]

The Theil–Sen estimator is a method for robust linear regression based on finding medians of slopes.[39]

In the context of image processing of monochrome raster images there is a type of noise, known as the salt and pepper noise, when each pixel independently becomes black (with some small probability) or white (with some small probability), and is unchanged otherwise (with the probability close to 1). An image constructed of median values of neighborhoods (like 3×3 square) can effectively reduce noise in this case.

In cluster analysis, the k-medians clustering algorithm provides a way of defining clusters, in which the criterion of maximising the distance between cluster-means that is used in k-means clustering, is replaced by maximising the distance between cluster-medians.

This is a method of robust regression. The idea dates back to Wald in 1940 who suggested dividing a set of bivariate data into two halves depending on the value of the independent parameter ${\displaystyle x}$: a left half with values less than the median and a right half with values greater than the median.[40] He suggested taking the means of the dependent ${\displaystyle y}$ and independent ${\displaystyle x}$ variables of the left and the right halves and estimating the slope of the line joining these two points. The line could then be adjusted to fit the majority of the points in the data set.

Nair and Shrivastava in 1942 suggested a similar idea but instead advocated dividing the sample into three equal parts before calculating the means of the subsamples.[41] Brown and Mood in 1951 proposed the idea of using the medians of two subsamples rather the means.[42] Tukey combined these ideas and recommended dividing the sample into three equal size subsamples and estimating the line based on the medians of the subsamples.[43]