Wallenius' noncentral hypergeometric distribution

Multivariate Wallenius' Noncentral Hypergeometric Distribution
Parameters	; ; ; ;
Support
PMF	; where
Mean	Approximated by solution to ; ;
Variance	Approximated by variance of Fisher's noncentral hypergeometric distribution with same mean.

Univariate Wallenius' Noncentral Hypergeometric Distribution
Parameters	; ; ;
Support	; ;
PMF	; where
Mean	Approximated by solution to ;
Variance	, where;

In probability theory and statistics, Wallenius' noncentral hypergeometric distribution (named after Kenneth Ted Wallenius) is a generalization of the hypergeometric distribution where items are sampled with bias.

Probability mass function for Wallenius' Noncentral Hypergeometric Distribution for different values of the odds ratio ω.
m₁ = 80, m₂ = 60, n = 100, ω = 0.1 ... 20

This distribution can be illustrated as an urn model with bias. Assume, for example, that an urn contains m₁ red balls and m₂ white balls, totalling N = m₁ + m₂ balls. Each red ball has the weight ω₁ and each white ball has the weight ω₂. We will say that the odds ratio is ω = ω₁ / ω₂. Now we are taking n balls, one by one, in such a way that the probability of taking a particular ball at a particular draw is equal to its proportion of the total weight of all balls that lie in the urn at that moment. The number of red balls x₁ that we get in this experiment is a random variable with Wallenius' noncentral hypergeometric distribution.

The matter is complicated by the fact that there is more than one noncentral hypergeometric distribution. Wallenius' noncentral hypergeometric distribution is obtained if balls are sampled one by one in such a way that there is competition between the balls. Fisher's noncentral hypergeometric distribution is obtained if the balls are sampled simultaneously or independently of each other. Unfortunately, both distributions are known in the literature as "the" noncentral hypergeometric distribution. It is important to be specific about which distribution is meant when using this name.

The two distributions are both equal to the (central) hypergeometric distribution when the odds ratio is 1.

It is far from obvious why these two distributions are different. See the Wikipedia entry on noncentral hypergeometric distributions for a more detailed explanation of the difference between these two probability distributions.

Univariate distribution

Recursive calculation of probability f(x,n) in Wallenius' distribution. The light grey fields are possible points on the way to the final point. The arrows indicate an arbitrary trajectory.

Wallenius' distribution is particularly complicated because each ball has a probability of being taken that depends not only on its weight, but also on the total weight of its competitors. And the weight of the competing balls depends on the outcomes of all preceding draws.

This recursive dependency gives rise to a difference equation with a solution that is given in open form by the integral in the expression of the probability mass function in the table above.

Closed form expressions for the probability mass function exist (Lyons, 1980), but they are not very useful for practical calculations because of extreme numerical instability, except in degenerate cases.

Several other calculation methods are used, including recursion, Taylor expansion and numerical integration (Fog, 2007, 2008).

The most reliable calculation method is recursive calculation of f(x,n) from f(x,n-1) and f(x-1,n-1) using the recursion formula given below under properties. The probabilities of all (x,n) combinations on all possible trajectories leading to the desired point are calculated, starting with f(0,0) = 1 as shown on the figure to the right. The total number of probabilities to calculate is n(x+1)-x². Other calculation methods must be used when n and x are so big that this method is too inefficient.

The probability that all balls have the same color is easier to calculate. See the formula below under multivariate distribution.

No exact formula for the mean is known (short of complete enumeration of all probabilities). The equation given above is reasonably accurate. This equation can be solved for μ by Newton-Raphson iteration. The same equation can be used for estimating the odds from an experimentally obtained value of the mean.

Properties of the univariate distribution

Wallenius' distribution has fewer symmetry relations than Fisher's noncentral hypergeometric distribution has. The only symmetry relates to the swapping of colors:

\operatorname {wnchypg} (x;n,m_{1},m_{2},\omega )=\operatorname {wnchypg} (n-x;n,m_{2},m_{1},1/\omega )\,.

Unlike Fisher's distribution, Wallenius' distribution has no symmetry relating to the number of balls not taken.

The following recursion formula is useful for calculating probabilities:

\operatorname {wnchypg} (x;n,m_{1},m_{2},\omega )=

\operatorname {wnchypg} (x-1;n-1,m_{1},m_{2},\omega ){\frac {(m_{1}-x+1)\omega }{(m_{1}-x+1)\omega +m_{2}+x-n}}+

\operatorname {wnchypg} (x;n-1,m_{1},m_{2},\omega ){\frac {m_{2}+x-n+1}{(m_{1}-x)\omega +m_{2}+x-n+1}}

Another recursion formula is also known:

\operatorname {wnchypg} (x;n,m_{1},m_{2},\omega )=

\operatorname {wnchypg} (x-1;n-1,m_{1}-1,m_{2},\omega ){\frac {m_{1}\omega }{m_{1}\omega +m_{2}}}+

\operatorname {wnchypg} (x;n-1,m_{1},m_{2}-1,\omega ){\frac {m_{2}}{m_{1}\omega +m_{2}}}\,.

The probability is limited by

\operatorname {f} _{1}(x)\leq \operatorname {wnchypg} (x;n,m_{1},m_{2},\omega )\leq \operatorname {f} _{2}(x)\,,\,\,{\text{for}}\,\,\omega <1\,,

\operatorname {f} _{1}(x)\geq \operatorname {wnchypg} (x;n,m_{1},m_{2},\omega )\geq \operatorname {f} _{2}(x)\,,\,\,{\text{for}}\,\,\omega >1\,,{\text{where}}

\operatorname {f} _{1}(x)={\binom {m_{1}}{x}}{\binom {m_{2}}{n-x}}{\frac {n!}{(m_{1}+m_{2}/\omega )^{\underline {x}}\,(m_{2}+\omega (m_{1}-x))^{\underline {n-x}}}}

\operatorname {f} _{2}(x)={\binom {m_{1}}{x}}{\binom {m_{2}}{n-x}}{\frac {n!}{(m_{1}+(m_{2}-x_{2})/\omega )^{\underline {x}}\,(m_{2}+\omega m_{1})^{\underline {n-x}}}}\,,

where the underlined superscript indicates the falling factorial $a^{\underline {b}}=a(a-1)\ldots (a-b+1)$ .

Multivariate distribution

The distribution can be expanded to any number of colors c of balls in the urn. The multivariate distribution is used when there are more than two colors.

The probability mass function can be calculated by various Taylor expansion methods or by numerical integration (Fog, 2008).

The probability that all balls have the same color, j, can be calculated as:

\operatorname {mwnchypg} ((0,\ldots ,0,x_{j},0,\ldots );n,\mathbf {m} ,{\boldsymbol {\omega }})={\frac {m_{j}^{\,\,{\underline {n}}}}{\left({\frac {1}{\omega _{j}}}\sum _{i=1}^{c}m_{i}\omega _{i}\right)^{\underline {n}}}}

for x_j = n ≤ m_j, where the underlined superscript denotes the falling factorial.

A reasonably good approximation to the mean can be calculated using the equation given above. The equation can be solved by defining θ so that

\mu _{i}=m_{i}(1-e^{\omega _{i}\theta })

and solving

\sum _{i=1}^{c}\mu _{i}=n

for θ by Newton-Raphson iteration.

The equation for the mean is also useful for estimating the odds from experimentally obtained values for the mean.

No good way of calculating the variance is known. The best known method is to approximate the multivariate Wallenius distribution by a multivariate Fisher's noncentral hypergeometric distribution with the same mean, and insert the mean as calculated above in the approximate formula for the variance of the latter distribution.

Properties of the multivariate distribution

The order of the colors is arbitrary so that any colors can be swapped.

The weights can be arbitrarily scaled:

\operatorname {mwnchypg} (\mathbf {x} ;n,\mathbf {m} ,{\boldsymbol {\omega }})=\operatorname {mwnchypg} (\mathbf {x} ;n,\mathbf {m} ,r{\boldsymbol {\omega }})\,\,

for all

r\in \mathbb {R} _{+}

.

Colors with zero number (m_i = 0) or zero weight (ω_i = 0) can be omitted from the equations.

Colors with the same weight can be joined:

\operatorname {mwnchypg} \left(\mathbf {x} ;n,\mathbf {m} ,(\omega _{1},\ldots ,\omega _{c-1},\omega _{c-1})\right)\,=

\operatorname {mwnchypg} \left((x_{1},\ldots ,x_{c-1}+x_{c});n,(m_{1},\ldots ,m_{c-1}+m_{c}),(\omega _{1},\ldots ,\omega _{c-1})\right)\,\cdot

\operatorname {hypg} (x_{c};x_{c-1}+x_{c},m_{c},m_{c-1}+m_{c})\,,

where $\operatorname {hypg} (x;n,m,N)$ is the (univariate, central) hypergeometric distribution probability.

Complementary Wallenius' noncentral hypergeometric distribution

Probability mass function for the Complementary Wallenius' Noncentral Hypergeometric Distribution for different values of the odds ratio ω.
m₁ = 80, m₂ = 60, n = 40, ω = 0.05 ... 10

The balls that are not taken in the urn experiment have a distribution that is different from Wallenius' noncentral hypergeometric distribution, due to a lack of symmetry. The distribution of the balls not taken can be called the complementary Wallenius' noncentral hypergeometric distribution.

Probabilities in the complementary distribution are calculated from Wallenius' distribution by replacing n with N-n, x_i with m_i - x_i, and ω_i with 1/ω_i.

Software available

WalleniusHypergeometricDistribution in Mathematica.
An implementation for the R programming language is available as the package named BiasedUrn. Includes univariate and multivariate probability mass functions, distribution functions, quantiles, random variable generating functions, mean and variance.
Implementation in C++ is available from www.agner.org.

References

Chesson, J. (1976). "A non-central multivariate hypergeometric distribution arising from biased sampling with application to selective predation". Journal of Applied Probability. 13 (4). Applied Probability Trust. pp. 795–797. doi:10.2307/3212535. JSTOR 3212535.
Fog, A. (2007). "Random number theory".
Fog, A. (2008). "Calculation Methods for Wallenius' Noncentral Hypergeometric Distribution". Communications in Statictics, Simulation and Computation. 37 (2): 258–273. doi:10.1080/03610910701790269. S2CID 9040568.
Johnson, N. L.; Kemp, A. W.; Kotz, S. (2005). Univariate Discrete Distributions. Hoboken, New Jersey: Wiley and Sons.
Lyons, N. I. (1980). "Closed Expressions for Noncentral Hypergeometric Probabilities". Communications in Statistics - Simulation and Computation. 9 (3). pp. 313–314. doi:10.1080/03610918008812156.
Manly, B. F. J. (1974). "A Model for Certain Types of Selection Experiments". Biometrics. 30 (2). International Biometric Society. pp. 281–294. doi:10.2307/2529649. JSTOR 2529649.
Wallenius, K. T. (1963). Biased Sampling: The Non-central Hypergeometric Probability Distribution. Ph.D. Thesis (Thesis). Stanford University, Department of Statistics.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

Probability distributions (List)
Discrete univariate with finite support	Benford Bernoulli beta-binomial binomial categorical hypergeometric Poisson binomial Rademacher soliton discrete uniform Zipf Zipf–Mandelbrot
Discrete univariate with infinite support	beta negative binomial Borel Conway–Maxwell–Poisson discrete phase-type Delaporte extended negative binomial Flory–Schulz Gauss–Kuzmin geometric logarithmic negative binomial Panjer parabolic fractal Poisson Skellam Yule–Simon zeta
Continuous univariate supported on a bounded interval	arcsine ARGUS Balding–Nichols Bates beta beta rectangular continuous Bernoulli Irwin–Hall Kumaraswamy logit-normal noncentral beta PERT raised cosine reciprocal triangular U-quadratic uniform Wigner semicircle
Continuous univariate supported on a semi-infinite interval	Benini Benktander 1st kind Benktander 2nd kind beta prime Burr chi-squared chi Dagum Davis exponential-logarithmic Erlang exponential F folded normal Fréchet gamma gamma/Gompertz generalized gamma generalized inverse Gaussian Gompertz half-logistic half-normal Hotelling's T-squared hyper-Erlang hyperexponential hypoexponential inverse chi-squared scaled inverse chi-squared inverse Gaussian inverse gamma Kolmogorov Lévy log-Cauchy log-Laplace log-logistic log-normal Lomax matrix-exponential Maxwell–Boltzmann Maxwell–Jüttner Mittag-Leffler Nakagami noncentral chi-squared noncentral F Pareto phase-type poly-Weibull Rayleigh relativistic Breit–Wigner Rice shifted Gompertz truncated normal type-2 Gumbel Weibull discrete Weibull Wilks's lambda
Continuous univariate supported on the whole real line	Cauchy exponential power Fisher's z Gaussian q generalized normal generalized hyperbolic geometric stable Gumbel Holtsmark hyperbolic secant Johnson's S_U Landau Laplace asymmetric Laplace logistic noncentral t normal (Gaussian) normal-inverse Gaussian skew normal slash stable Student's t type-1 Gumbel Tracy–Widom variance-gamma Voigt
Continuous univariate with support whose type varies	generalized chi-squared generalized extreme value generalized Pareto Marchenko–Pastur q-exponential q-Gaussian q-Weibull shifted log-logistic Tukey lambda
Mixed continuous-discrete univariate	rectified Gaussian
Multivariate (joint)	Discrete Ewens multinomial Dirichlet-multinomial negative multinomial Continuous Dirichlet generalized Dirichlet multivariate Laplace multivariate normal multivariate stable multivariate t normal-inverse-gamma normal-gamma Matrix-valued inverse matrix gamma inverse-Wishart matrix normal matrix t matrix gamma normal-inverse-Wishart normal-Wishart Wishart
Directional	Univariate (circular) directional Circular uniform univariate von Mises wrapped normal wrapped Cauchy wrapped exponential wrapped asymmetric Laplace wrapped Lévy Bivariate (spherical) Kent Bivariate (toroidal) bivariate von Mises Multivariate von Mises–Fisher Bingham
Degenerate and singular	Degenerate Dirac delta function Singular Cantor
Families	Circular compound Poisson elliptical exponential natural exponential location–scale maximum entropy mixture Pearson Tweedie wrapped

Univariate Wallenius' Noncentral Hypergeometric Distribution
Parameters	$m_{1},m_{2}\in \mathbb {N}$ $N=m_{1}+m_{2}$ $n\in [0,N)$ $\omega \in \mathbb {R} _{+}$
Support	$x\in [x_{min},x_{max}]$ $x_{min}=\max(0,n-m_{2})$ $x_{max}=\min(n,m_{1})$
PMF	${\binom {m_{1}}{x}}{\binom {m_{2}}{n-x}}\int _{0}^{1}(1-t^{\omega /D})^{x}(1-t^{1/D})^{n-x}\operatorname {d} t$ where $D=\omega (m_{1}-x)+(m_{2}-(n-x))$
Mean	Approximated by solution $\mu$ to ${\frac {\mu }{m_{1}}}+\left(1-{\frac {n-\mu }{m_{2}}}\right)^{\omega }=1$
Variance	$\approx {\frac {Nab}{(N-1)(m_{1}b+m_{2}a)}}\,$ , where $a=\mu (m_{1}-\mu ),\;b=(n-\mu )(\mu +m_{2}-n)$

Multivariate Wallenius' Noncentral Hypergeometric Distribution
Parameters	$c\in \mathbb {N}$ $\mathbf {m} =(m_{1},\ldots ,m_{c})\in \mathbb {N} ^{c}$ $N=\sum _{i=1}^{c}m_{i}$ $n\in [0,N)$ ${\boldsymbol {\omega }}=(\omega _{1},\ldots ,\omega _{c})\in \mathbb {R} _{+}^{c}$
Support	$\mathrm {S} =\left\{\mathbf {x} \in \mathbb {Z} _{0+}^{c}\,:\,\sum _{i=1}^{c}x_{i}=n\right\}$
PMF	$\left(\prod _{i=1}^{c}{\binom {m_{i}}{x_{i}}}\right)\int _{0}^{1}\prod _{i=1}^{c}(1-t^{\omega _{i}/D})^{x_{i}}\operatorname {d} t\,,$ where $D={\boldsymbol {\omega }}\cdot (\mathbf {m} -\mathbf {x} )=\sum _{i=1}^{c}\omega _{i}(m_{i}-x_{i})$
Mean	Approximated by solution $\mu _{1},\ldots ,\mu _{c}$ to $\left(1-{\frac {\mu _{1}}{m_{1}}}\right)^{1/\omega _{1}}=\left(1-{\frac {\mu _{2}}{m_{2}}}\right)^{1/\omega _{2}}=\ldots =\left(1-{\frac {\mu _{c}}{m_{c}}}\right)^{1/\omega _{c}}$ $\wedge \,\sum _{i=1}^{c}\mu _{i}=n\,\wedge \,\forall \,i\in [0,c]\,:\,0\leq \mu _{i}\leq m_{i}\,.$
Variance	Approximated by variance of Fisher's noncentral hypergeometric distribution with same mean.