Semiparametric regression

A partially linear model is given by

${\displaystyle Y_{i}=X'_{i}\beta +g\left(Z_{i}\right)+u_{i},\,\quad i=1,\ldots ,n,\,}$

where ${\displaystyle Y_{i}}$ is the dependent variable, ${\displaystyle X_{i}}$ is a ${\displaystyle p\times 1}$ vector of explanatory variables, ${\displaystyle \beta }$ is a ${\displaystyle p\times 1}$ vector of unknown parameters and ${\displaystyle Z_{i}\in \operatorname {R} ^{q}}$. The parametric part of the partially linear model is given by the parameter vector ${\displaystyle \beta }$ while the nonparametric part is the unknown function ${\displaystyle g\left(Z_{i}\right)}$. The data is assumed to be i.i.d. with ${\displaystyle E\left(u_{i}|X_{i},Z_{i}\right)=0}$ and the model allows for a conditionally heteroskedastic error process ${\displaystyle E\left(u_{i}^{2}|x,z\right)=\sigma ^{2}\left(x,z\right)}$ of unknown form. This type of model was proposed by Robinson (1988) and extended to handle categorical covariates by Racine and Li (2007).

This method is implemented by obtaining a ${\displaystyle {\sqrt {n}}}$ consistent estimator of ${\displaystyle \beta }$ and then deriving an estimator of ${\displaystyle g\left(Z_{i}\right)}$ from the nonparametric regression of ${\displaystyle Y_{i}-X'_{i}{\hat {\beta }}}$ on ${\displaystyle z}$ using an appropriate nonparametric regression method.[1]

A single index model takes the form

${\displaystyle Y=g\left(X'\beta _{0}\right)+u,\,}$

where ${\displaystyle Y}$, ${\displaystyle X}$ and ${\displaystyle \beta _{0}}$ are defined as earlier and the error term ${\displaystyle u}$ satisfies ${\displaystyle E\left(u|X\right)=0}$. The single index model takes its name from the parametric part of the model ${\displaystyle x'\beta }$ which is a scalar single index. The nonparametric part is the unknown function ${\displaystyle g\left(\cdot \right)}$.

The single index model method developed by Ichimura (1993) is as follows. Consider the situation in which ${\displaystyle y}$ is continuous. Given a known form for the function ${\displaystyle g\left(\cdot \right)}$, ${\displaystyle \beta _{0}}$ could be estimated using the nonlinear least squares method to minimize the function

${\displaystyle \sum _{i=1}\left(Y_{i}-g\left(X'_{i}\beta \right)\right)^{2}.}$

Since the functional form of ${\displaystyle g\left(\cdot \right)}$ is not known, we need to estimate it. For a given value for ${\displaystyle \beta }$ an estimate of the function

${\displaystyle G\left(X'_{i}\beta \right)=E\left(Y_{i}|X'_{i}\beta \right)=E\left[g\left(X'_{i}\beta _{o}\right)|X'_{i}\beta \right]}$

using kernel method. Ichimura (1993) proposes estimating ${\displaystyle g\left(X'_{i}\beta \right)}$ with

${\displaystyle {\hat {G}}_{-i}\left(X'_{i}\beta \right),\,}$

the leave-one-out nonparametric kernel estimator of ${\displaystyle G\left(X'_{i}\beta \right)}$.

If the dependent variable ${\displaystyle y}$ is binary and ${\displaystyle X_{i}}$ and ${\displaystyle u_{i}}$ are assumed to be independent, Klein and Spady (1993) propose a technique for estimating ${\displaystyle \beta }$ using maximum likelihood methods. The log-likelihood function is given by

${\displaystyle L\left(\beta \right)=\sum _{i}\left(1-Y_{i}\right)\ln \left(1-{\hat {g}}_{-i}\left(X'_{i}\beta \right)\right)+\sum _{i}Y_{i}\ln \left({\hat {g}}_{-i}\left(X'_{i}\beta \right)\right),}$

where ${\displaystyle {\hat {g}}_{-i}\left(X'_{i}\beta \right)}$ is the leave-one-out estimator.

Hastie and Tibshirani (1993) propose a smooth coefficient model given by

${\displaystyle Y_{i}=\alpha \left(Z_{i}\right)+X'_{i}\beta \left(Z_{i}\right)+u_{i}=\left(1+X'_{i}\right)\left({\begin{array}{c}\alpha \left(Z_{i}\right)\\\beta \left(Z_{i}\right)\end{array}}\right)+u_{i}=W'_{i}\gamma \left(Z_{i}\right)+u_{i},}$

where ${\displaystyle X_{i}}$ is a ${\displaystyle k\times 1}$ vector and ${\displaystyle \beta \left(z\right)}$ is a vector of unspecified smooth functions of ${\displaystyle z}$.

${\displaystyle \gamma \left(\cdot \right)}$ may be expressed as

${\displaystyle \gamma \left(Z_{i}\right)=\left(E\left[W_{i}W'_{i}|Z_{i}\right]\right)^{-1}E\left[W_{i}Y_{i}|Z_{i}\right].}$