Jackknife resampling

The jackknife estimate of a parameter can be found by estimating the parameter for each subsample omitting the i-th observation.[2] For example, if the parameter to be estimated is the population mean of x, we compute the mean ${\displaystyle {\bar {x}}_{i}}$ for each subsample consisting of all but the i-th data point:

${\displaystyle {\bar {x}}_{i}={\frac {1}{n-1}}\sum _{j=1,j\neq i}^{n}x_{j},\quad \quad i=1,\dots ,n.}$

These n estimates form an estimate of the distribution of the sample statistic if it were computed over a large number of samples. In particular, the mean of this sampling distribution is the average of these n estimates:

${\displaystyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}{\bar {x}}_{i}.}$

One can show explicitly that this ${\displaystyle {\bar {x}}}$ equals the usual estimate ${\displaystyle {\frac {1}{n}}\sum _{i=1}^{n}x_{i}}$, so the real point emerges for higher moments than the mean. A jackknife estimate of the variance of the estimator can be calculated from the variance of this distribution of ${\displaystyle {\bar {x}}_{i}}$:[3][4]

${\displaystyle \operatorname {Var} ({\bar {x}})={\frac {n-1}{n}}\sum _{i=1}^{n}({\bar {x}}_{i}-{\bar {x}})^{2}={\frac {1}{n(n-1)}}\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}.}$

The jackknife technique can be used to estimate the bias of an estimator calculated over the entire sample. Say ${\displaystyle {\hat {\theta }}}$ is the calculated estimator of the parameter of interest based on all ${\displaystyle {n}}$ observations. Let

${\displaystyle {\hat {\theta }}_{\mathrm {(.)} }={\frac {1}{n}}\sum _{i=1}^{n}{\hat {\theta }}_{(i)}}$

where ${\displaystyle {\hat {\theta }}_{(i)}}$ is the estimate of interest based on the sample with the i-th observation removed, and ${\displaystyle {\hat {\theta }}_{\mathrm {(.)} }}$ is the average of these "leave-one-out" estimates. The jackknife estimate of the bias of ${\displaystyle {\hat {\theta }}}$ is given by:

${\displaystyle {\widehat {\text{bias}}}_{\mathrm {(\theta )} }=(n-1)({\hat {\theta }}_{\mathrm {(.)} }-{\hat {\theta }})}$

and the resulting bias-corrected jackknife estimate of ${\displaystyle \theta }$ is given by:

${\displaystyle {\hat {\theta }}_{\text{jack}}={\hat {\theta }}-{\widehat {\text{bias}}}_{\mathrm {(\theta )} }=n{\hat {\theta }}-(n-1){\hat {\theta }}_{\mathrm {(.)} }}$

This removes the bias in the special case that the bias is ${\displaystyle O(n^{-1})}$ and removes it to ${\displaystyle O(n^{-2})}$ in other cases.[1]

• Leave-one-out error

• Cameron, Adrian; Trivedi, Pravin K. (2005). Microeconometrics : methods and applications. Cambridge New York: Cambridge University Press. ISBN 9780521848053.
• Efron, Bradley; Stein, Charles (May 1981). "The Jackknife Estimate of Variance". The Annals of Statistics. 9 (3): 586–596. doi:10.1214/aos/1176345462. JSTOR 2240822.
• Efron, Bradley (1982). The jackknife, the bootstrap, and other resampling plans. Philadelphia, PA: Society for Industrial and Applied Mathematics. ISBN 9781611970319.
• Quenouille, Maurice H. (September 1949). "Problems in Plane Sampling". The Annals of Mathematical Statistics. 20 (3): 355–375. doi:10.1214/aoms/1177729989. JSTOR 2236533.
• Quenouille, Maurice H. (1956). "Notes on Bias in Estimation". Biometrika. 43 (3–4): 353–360. doi:10.1093/biomet/43.3-4.353. JSTOR 2332914.
• Tukey, John W. (1958). "Bias and confidence in not quite large samples (abstract)". The Annals of Mathematical Statistics. 29 (2): 614. doi:10.1214/aoms/1177706647.