Poisson regression

If ${\displaystyle \mathbf {x} \in \mathbb {R} ^{n}}$ is a vector of independent variables, then the model takes the form

${\displaystyle \log(\operatorname {E} (Y\mid \mathbf {x} ))=\alpha +\mathbf {\beta } '\mathbf {x} ,}$

where ${\displaystyle \alpha \in \mathbb {R} }$ and ${\displaystyle \mathbf {\beta } \in \mathbb {R} ^{n}}$. Sometimes this is written more compactly as

${\displaystyle \log(\operatorname {E} (Y\mid \mathbf {x} ))={\boldsymbol {\theta }}'\mathbf {x} ,\,}$

where x is now an (n + 1)-dimensional vector consisting of n independent variables concatenated to the number one. Here θ is simply α concatenated to β.

Thus, when given a Poisson regression model θ and an input vector x, the predicted mean of the associated Poisson distribution is given by

${\displaystyle \operatorname {E} (Y\mid \mathbf {x} )=e^{{\boldsymbol {\theta }}'\mathbf {x} }.\,}$

If Y_i are independent observations with corresponding values x_i of the predictor variables, then θ can be estimated by maximum likelihood. The maximum-likelihood estimates lack a closed-form expression and must be found by numerical methods. The probability surface for maximum-likelihood Poisson regression is always concave, making Newton–Raphson or other gradient-based methods appropriate estimation techniques.

Given a set of parameters θ and an input vector x, the mean of the predicted Poisson distribution, as stated above, is given by

${\displaystyle \lambda$ :=\operatorname {E} (Y\mid x)=e^{\theta 'x},\,}

and thus, the Poisson distribution's probability mass function is given by

${\displaystyle p(y\mid x;\theta )={\frac {\lambda ^{y}}{y!}}e^{-\lambda }={\frac {e^{y\theta 'x}e^{-e^{\theta 'x}}}{y!}}}$

Now suppose we are given a data set consisting of m vectors ${\displaystyle x_{i}\in \mathbb {R} ^{n+1},\,i=1,\ldots ,m}$, along with a set of m values ${\displaystyle y_{1},\ldots ,y_{m}\in \mathbb {N} }$. Then, for a given set of parameters θ, the probability of attaining this particular set of data is given by

${\displaystyle p(y_{1},\ldots ,y_{m}\mid x_{1},\ldots ,x_{m};\theta )=\prod _{i=1}^{m}{\frac {e^{y_{i}\theta 'x_{i}}e^{-e^{\theta 'x_{i}}}}{y_{i}!}}.}$

By the method of maximum likelihood, we wish to find the set of parameters θ that makes this probability as large as possible. To do this, the equation is first rewritten as a likelihood function in terms of θ:

${\displaystyle L(\theta \mid X,Y)=\prod _{i=1}^{m}{\frac {e^{y_{i}\theta 'x_{i}}e^{-e^{\theta 'x_{i}}}}{y_{i}!}}.}$

Note that the expression on the right hand side has not actually changed. A formula in this form is typically difficult to work with; instead, one uses the log-likelihood:

${\displaystyle \ell (\theta \mid X,Y)=\log L(\theta \mid X,Y)=\sum _{i=1}^{m}\left(y_{i}\theta 'x_{i}-e^{\theta 'x_{i}}-\log(y_{i}!)\right).}$

Notice that the parameters θ only appear in the first two terms of each term in the summation. Therefore, given that we are only interested in finding the best value for θ we may drop the y_i! and simply write

${\displaystyle \ell (\theta \mid X,Y)=\sum _{i=1}^{m}\left(y_{i}\theta 'x_{i}-e^{\theta 'x_{i}}\right).}$

To find a maximum, we need to solve an equation ${\displaystyle {\frac {\partial \ell (\theta \mid X,Y)}{\partial \theta }}=0}$ which has no closed-form solution. However, the negative log-likelihood, ${\displaystyle -\ell (\theta \mid X,Y)}$, is a convex function, and so standard convex optimization techniques such as gradient descent can be applied to find the optimal value of θ.

Poisson regression may be appropriate when the dependent variable is a count, for instance of events such as the arrival of a telephone call at a call centre.[1] The events must be independent in the sense that the arrival of one call will not make another more or less likely, but the probability per unit time of events is understood to be related to covariates such as time of day.

glm(y ~ offset(log(exposure)) + x, family=poisson(link=log) )

• Zero-inflated model
• Poisson distribution
• Fixed-effect Poisson model
• Partial likelihood methods for panel data § Example: pooled QMLE for Poisson models
• Control function (econometrics) § Endogeneity in Poisson regression

• Cameron, A. C.; Trivedi, P. K. (1998). Regression analysis of count data. Cambridge University Press. ISBN 978-0-521-63201-0.
• Christensen, Ronald (1997). Log-linear models and logistic regression. Springer Texts in Statistics (Second ed.). New York: Springer-Verlag. ISBN 978-0-387-98247-2. MR 1633357.
• Gouriéroux, Christian (2000). "The Econometrics of Discrete Positive Variables: the Poisson Model". Econometrics of Qualitative Dependent Variables. New York: Cambridge University Press. pp. 270–83. ISBN 978-0-521-58985-7.
• Greene, William H. (2008). "Models for Event Counts and Duration". Econometric Analysis (8th ed.). Upper Saddle River: Prentice Hall. pp. 906–944. ISBN 978-0-13-600383-0.
• Hilbe, J. M. (2007). Negative Binomial Regression. Cambridge University Press. ISBN 978-0-521-85772-7.
• Jones, Andrew M.; et al. (2013). "Models for count data". Applied Health Economics. London: Routledge. pp. 295–341. ISBN 978-0-415-67682-3.