Mixed model

Ronald Fisher introduced random effects models to study the correlations of trait values between relatives.[2] In the 1950s, Charles Roy Henderson provided best linear unbiased estimates (BLUE) of fixed effects and best linear unbiased predictions (BLUP) of random effects.[3][4][5][6] Subsequently, mixed modeling has become a major area of statistical research, including work on computation of maximum likelihood estimates, non-linear mixed effects models, missing data in mixed effects models, and Bayesian estimation of mixed effects models. Mixed models are applied in many disciplines where multiple correlated measurements are made on each unit of interest. They are prominently used in research involving human and animal subjects in fields ranging from genetics to marketing, and have also been used in baseball[7] and industrial statistics.[8]

In matrix notation a linear mixed model can be represented as

${\displaystyle {\boldsymbol {y}}=X{\boldsymbol {\beta }}+Z{\boldsymbol {u}}+{\boldsymbol {\epsilon }}}$

where

• ${\displaystyle {\boldsymbol {y}}}$ is a known vector of observations, with mean ${\displaystyle E({\boldsymbol {y}})=X{\boldsymbol {\beta }}}$;
• ${\displaystyle {\boldsymbol {\beta }}}$ is an unknown vector of fixed effects;
• ${\displaystyle {\boldsymbol {u}}}$ is an unknown vector of random effects, with mean ${\displaystyle E({\boldsymbol {u}})={\boldsymbol {0}}}$ and variance–covariance matrix ${\displaystyle \operatorname {var} ({\boldsymbol {u}})=G}$;
• ${\displaystyle {\boldsymbol {\epsilon }}}$ is an unknown vector of random errors, with mean ${\displaystyle E({\boldsymbol {\epsilon }})={\boldsymbol {0}}}$ and variance ${\displaystyle \operatorname {var} ({\boldsymbol {\epsilon }})=R}$;
• ${\displaystyle X}$ and ${\displaystyle Z}$ are known design matrices relating the observations ${\displaystyle {\boldsymbol {y}}}$ to ${\displaystyle {\boldsymbol {\beta }}}$ and ${\displaystyle {\boldsymbol {u}}}$, respectively.

The joint density of ${\displaystyle {\boldsymbol {y}}}$ and ${\displaystyle {\boldsymbol {u}}}$ can be written as: ${\displaystyle f({\boldsymbol {y}},{\boldsymbol {u}})=f({\boldsymbol {y}}|{\boldsymbol {u}})\,f({\boldsymbol {u}})}$. Assuming normality, ${\displaystyle {\boldsymbol {u}}\sim {\mathcal {N}}({\boldsymbol {0}},G)}$, ${\displaystyle {\boldsymbol {\epsilon }}\sim {\mathcal {N}}({\boldsymbol {0}},R)}$ and ${\displaystyle \mathrm {Cov} ({\boldsymbol {u}},{\boldsymbol {\epsilon }})={\boldsymbol {0}}}$, and maximizing the joint density over ${\displaystyle {\boldsymbol {\beta }}}$ and ${\displaystyle {\boldsymbol {u}}}$, gives Henderson's "mixed model equations" (MME) for linear mixed models:[3][5][9]

${\displaystyle {\begin{pmatrix}X'R^{-1}X&X'R^{-1}Z\\Z'R^{-1}X&Z'R^{-1}Z+G^{-1}\end{pmatrix}}{\begin{pmatrix}{\hat {\boldsymbol {\beta }}}\\{\hat {\boldsymbol {u}}}\end{pmatrix}}={\begin{pmatrix}X'R^{-1}{\boldsymbol {y}}\\Z'R^{-1}{\boldsymbol {y}}\end{pmatrix}}}$

The solutions to the MME, ${\displaystyle \textstyle {\hat {\boldsymbol {\beta }}}}$ and ${\displaystyle \textstyle {\hat {\boldsymbol {u}}}}$ are best linear unbiased estimates (BLUE) and predictors (BLUP) for ${\displaystyle {\boldsymbol {\beta }}}$ and ${\displaystyle {\boldsymbol {u}}}$, respectively. This is a consequence of the Gauss–Markov theorem when the conditional variance of the outcome is not scalable to the identity matrix. When the conditional variance is known, then the inverse variance weighted least squares estimate is BLUE. However, the conditional variance is rarely, if ever, known. So it is desirable to jointly estimate the variance and weighted parameter estimates when solving MMEs.

One method used to fit such mixed models is that of the EM algorithm where the variance components are treated as unobserved nuisance parameters in the joint likelihood.[10] Currently, this is the implemented method for the major statistical software packages R (lme in the nlme package, or lmer in the lme4 package), Python (statsmodels package), Julia (MixedModels.jl package), and SAS (proc mixed). The solution to the mixed model equations is a maximum likelihood estimate when the distribution of the errors is normal.[11][12]

• Nonlinear mixed-effects model
• Fixed effects model
• Generalized linear mixed model
• Linear regression
• Mixed-design analysis of variance
• Multilevel model
• Random effects model
• Repeated measures design

• Gałecki, Andrzej; Burzykowski, Tomasz (2013). Linear Mixed-Effects Models Using R: A Step-by-Step Approach. New York: Springer. ISBN 978-1-4614-3900-4.
• Milliken, G. A.; Johnson, D. E. (1992). Analysis of Messy Data: Vol. I. Designed Experiments. New York: Chapman & Hall.
• West, B. T.; Welch, K. B.; Galecki, A. T. (2007). Linear Mixed Models: A Practical Guide Using Statistical Software. New York: Chapman & Hall/CRC.