Seemingly unrelated regressions

Suppose there are m regression equations

${\displaystyle y_{ir}=x_{ir}^{\mathsf {T}}\;\!\beta _{i}+\varepsilon _{ir},\quad i=1,\ldots ,m.}$

Here i represents the equation number, r = 1, …, R is the time period and we are taking the transpose of the ${\displaystyle x_{ir}}$ column vector. The number of observations R is assumed to be large, so that in the analysis we take R → ${\displaystyle \infty }$, whereas the number of equations m remains fixed.

Each equation i has a single response variable y_ir, and a k_i-dimensional vector of regressors x_ir. If we stack observations corresponding to the i-th equation into R-dimensional vectors and matrices, then the model can be written in vector form as

${\displaystyle y_{i}=X_{i}\beta _{i}+\varepsilon _{i},\quad i=1,\ldots ,m,}$

where y_i and ε_i are R×1 vectors, X_i is a R×k_i matrix, and β_i is a k_i×1 vector.

Finally, if we stack these m vector equations on top of each other, the system will take the form [4]^{(eq. (2.2))}

${\displaystyle {\begin{pmatrix}y_{1}\\y_{2}\\\vdots \\y_{m}\end{pmatrix}}={\begin{pmatrix}X_{1}&0&\ldots &0\\0&X_{2}&\ldots &0\\\vdots &\vdots &\ddots &\vdots \\0&0&\ldots &X_{m}\end{pmatrix}}{\begin{pmatrix}\beta _{1}\\\beta _{2}\\\vdots \\\beta _{m}\end{pmatrix}}+{\begin{pmatrix}\varepsilon _{1}\\\varepsilon _{2}\\\vdots \\\varepsilon _{m}\end{pmatrix}}=X\beta +\varepsilon \,.}$

(1)

The assumption of the model is that error terms ε_ir are independent across time, but may have cross-equation contemporaneous correlations. Thus we assume that E[ ε_ir ε_is | X ] = 0 whenever r ≠ s, whereas E[ ε_ir ε_jr | X ] = σ_ij. Denoting Σ = [σ_ij] the m×m skedasticity matrix of each observation, the covariance matrix of the stacked error terms ε will be equal to [4]^{(eq. (2.4))}[3]^:332

${\displaystyle \Omega \equiv \operatorname {E} [\,\varepsilon \varepsilon ^{\mathsf {T}}\,|X\,]=\Sigma \otimes I_{R},}$

where I_R is the R-dimensional identity matrix and ⊗ denotes the matrix Kronecker product.

The SUR model is usually estimated using the feasible generalized least squares (FGLS) method. This is a two-step method where in the first step we run ordinary least squares regression for (1). The residuals from this regression are used to estimate the elements of matrix ${\displaystyle \Sigma }$:[6]^:198

${\displaystyle {\hat {\sigma }}_{ij}={\frac {1}{R}}\,{\hat {\varepsilon }}_{i}^{\mathsf {T}}{\hat {\varepsilon }}_{j}.}$

In the second step we run generalized least squares regression for (1) using the variance matrix ${\displaystyle \scriptstyle {\hat {\Omega }}\;=\;{\hat {\Sigma }}\,\otimes \,I_{R}}$:

${\displaystyle {\hat {\beta }}={\Big (}X^{\mathsf {T}}({\hat {\Sigma }}^{-1}\otimes I_{R})X{\Big )}^{\!-1}X^{\mathsf {T}}({\hat {\Sigma }}^{-1}\otimes I_{R})\,y.}$

This estimator is unbiased in small samples assuming the error terms ε_ir have symmetric distribution; in large samples it is consistent and asymptotically normal with limiting distribution[6]^:198

${\displaystyle {\sqrt {R}}({\hat {\beta }}-\beta )\ {\xrightarrow {d}}\ {\mathcal {N}}{\Big (}\,0,\;{\Big (}{\tfrac {1}{R}}X^{\mathsf {T}}(\Sigma ^{-1}\otimes I_{R})X{\Big )}^{\!-1}\,{\Big )}.}$

Other estimation techniques besides FGLS were suggested for SUR model:[7] the maximum likelihood (ML) method under the assumption that the errors are normally distributed; the iterative generalized least squares (IGLS), where the residuals from the second step of FGLS are used to recalculate the matrix ${\displaystyle \scriptstyle {\hat {\Sigma }}}$, then estimate ${\displaystyle \scriptstyle {\hat {\beta }}}$ again using GLS, and so on, until convergence is achieved; the iterative ordinary least squares (IOLS) scheme, where estimation is performed on equation-by-equation basis, but every equation includes as additional regressors the residuals from the previously estimated equations in order to account for the cross-equation correlations, the estimation is run iteratively until convergence is achieved. Kmenta and Gilbert (1968) ran a Monte-Carlo study and established that all three methods—IGLS, IOLS and ML—yield numerically equivalent results, they also found that the asymptotic distribution of these estimators is the same as the distribution of the FGLS estimator, whereas in small samples neither of the estimators was more superior than the others.[8] Zellner and Ando (2010) developed a direct Monte Carlo method for the Bayesian analysis of SUR model.[9]

There are two important cases when the SUR estimates turn out to be equivalent to the equation-by-equation OLS, so that there is no gain in estimating the system jointly. These cases are:

1. When the matrix Σ is known to be diagonal, that is, there are no cross-equation correlations between the error terms. In this case the system becomes not seemingly but truly unrelated.
2. When each equation contains exactly the same set of regressors, that is X₁ = X₂ = … = X_m. That the estimators turn out to be numerically identical to OLS estimates follows from Kruskal's tree theorem,[1]^:313 or can be shown via the direct calculation.[6]^:197

• In R, SUR can estimated using the package “systemfit”.[10][11][12][13]
• In SAS, SUR can be estimated using the syslin procedure.[14]
• In Stata, SUR can be estimated using the sureg and suest commands.[15][16][17]
• In Limdep, SUR can be estimated using the sure command [18]
• In Python, SUR can be estimated using the command SUR in the “linearmodels” package.[19]
• In gretl, SUR can be estimated using the system command.

• General linear model
• Simultaneous equations models

• Davidson, James (2000). Econometric Theory. Oxford: Blackwell. pp. 308–314. ISBN 978-0-631-17837-8.
• Fiebig, Denzil G. (2001). "Seemingly Unrelated Regression". In Baltagi, Badi H. (ed.). A Companion to Theoretical Econometrics. Oxford: Blackwell. pp. 101–121. ISBN 978-0-631-21254-6.
• Greene, William H. (2012). Econometric Analysis (Seventh ed.). Upper Saddle River: Pearson Prentice-Hall. pp. 332–344. ISBN 978-0-273-75356-8.