Inverse-gamma distribution

For ${\displaystyle \alpha >0}$ and ${\displaystyle \beta >0}$,

${\displaystyle \mathbb {E} [\ln(X)]=\ln(\beta )-\psi (\alpha )\,}$

and

${\displaystyle \mathbb {E} [X^{-1}]={\frac {\alpha }{\beta }},\,}$

The information entropy is

${\displaystyle {\begin{aligned}\operatorname {H} (X)&=\operatorname {E} [-\ln(p(X))]\\&=\operatorname {E} \left[-\alpha \ln(\beta )+\ln(\Gamma (\alpha ))+(\alpha +1)\ln(X)+{\frac {\beta }{X}}\right]\\&=-\alpha \ln(\beta )+\ln(\Gamma (\alpha ))+(\alpha +1)\ln(\beta )-(\alpha +1)\psi (\alpha )+\alpha \\&=\alpha +\ln(\beta \Gamma (\alpha ))-(\alpha +1)\psi (\alpha ).\end{aligned}}}$

where ${\displaystyle \psi (\alpha )}$ is the digamma function.

The Kullback-Leibler divergence of Inverse-Gamma(α_p, β_p) from Inverse-Gamma(α_q, β_q) is the same as the KL-divergence of Gamma(α_p, β_p) from Gamma(α_q, β_q):

${\displaystyle D_{\mathrm {KL} }(\alpha _{p},\beta _{p};\alpha _{q},\beta _{q})=\mathbb {E} \left[\log {\frac {\rho (X)}{\pi (X)}}\right]=\mathbb {E} \left[\log {\frac {\rho (1/Y)}{\pi (1/Y)}}\right]=\mathbb {E} \left[\log {\frac {\rho _{G}(Y)}{\pi _{G}(Y)}}\right],}$

where ${\displaystyle \rho ,\pi }$ are the pdfs of the Inverse-Gamma distributions and ${\displaystyle \rho _{G},\pi _{G}}$ are the pdfs of the Gamma distributions, ${\displaystyle Y}$ is Gamma(α_p, β_p) distributed.

${\displaystyle {\begin{aligned}D_{\mathrm {KL} }(\alpha _{p},\beta _{p};\alpha _{q},\beta _{q})={}&(\alpha _{p}-\alpha _{q})\psi (\alpha _{p})-\log \Gamma (\alpha _{p})+\log \Gamma (\alpha _{q})+\alpha _{q}(\log \beta _{p}-\log \beta _{q})+\alpha _{p}{\frac {\beta _{q}-\beta _{p}}{\beta _{p}}}.\end{aligned}}}$

• If ${\displaystyle X\sim {\mbox{Inv-Gamma}}(\alpha ,\beta )}$ then ${\displaystyle kX\sim {\mbox{Inv-Gamma}}(\alpha ,k\beta )\,}$
• If ${\displaystyle X\sim {\mbox{Inv-Gamma}}(\alpha ,{\tfrac {1}{2}})}$ then ${\displaystyle X\sim {\mbox{Inv-}}\chi ^{2}(2\alpha )\,}$ (inverse-chi-squared distribution)
• If ${\displaystyle X\sim {\mbox{Inv-Gamma}}({\tfrac {\alpha }{2}},{\tfrac {1}{2}})}$ then ${\displaystyle X\sim {\mbox{Scaled Inv-}}\chi ^{2}(\alpha ,{\tfrac {1}{\alpha }})\,}$ (scaled-inverse-chi-squared distribution)
• If ${\displaystyle X\sim {\textrm {Inv-Gamma}}({\tfrac {1}{2}},{\tfrac {c}{2}})}$ then ${\displaystyle X\sim {\textrm {Levy}}(0,c)\,}$ (Lévy distribution)
• If ${\displaystyle X\sim {\textrm {Inv-Gamma}}(1,c)}$ then ${\displaystyle {\tfrac {1}{X}}\sim {\textrm {Exp}}(c)\,}$ (Exponential distribution)
• If ${\displaystyle X\sim {\mbox{Gamma}}(\alpha ,\beta )\,}$ (Gamma distribution with rate parameter ${\displaystyle \beta }$) then ${\displaystyle {\tfrac {1}{X}}\sim {\mbox{Inv-Gamma}}(\alpha ,\beta )\,}$ (see derivation in the next paragraph for details)
• Note that If X ~ Gamma(k, θ) (Gamma distribution with scale parameter θ ) then 1/X ~ Inv-Gamma(k, θ⁻¹)
• Inverse gamma distribution is a special case of type 5 Pearson distribution
• A multivariate generalization of the inverse-gamma distribution is the inverse-Wishart distribution.
• For the distribution of a sum of independent inverted Gamma variables see Witkovsky (2001)

Let ${\displaystyle X\sim {\mbox{Gamma}}(\alpha ,\beta )}$, and recall that the pdf of the gamma distribution is

${\displaystyle f_{X}(x)={\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}x^{\alpha -1}e^{-\beta x}}$, ${\displaystyle x>0}$.

Note that ${\displaystyle \beta }$ is the rate parameter from the perspective of the gamma distribution.

Define the transformation ${\displaystyle Y=g(X)={\tfrac {1}{X}}}$. Then, the pdf of ${\displaystyle Y}$ is

${\displaystyle {\begin{aligned}f_{Y}(y)&=f_{X}\left(g^{-1}(y)\right)\left|{\frac {d}{dy}}g^{-1}(y)\right|\\[6pt]&={\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\left({\frac {1}{y}}\right)^{\alpha -1}\exp \left({\frac {-\beta }{y}}\right){\frac {1}{y^{2}}}\\[6pt]&={\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\left({\frac {1}{y}}\right)^{\alpha +1}\exp \left({\frac {-\beta }{y}}\right)\\[6pt]&={\frac {\beta ^{\alpha }}{\Gamma (\alpha )}}\left(y\right)^{-\alpha -1}\exp \left({\frac {-\beta }{y}}\right)\\[6pt]\end{aligned}}}$

Note that ${\displaystyle \beta }$ is the scale parameter from the perspective of the inverse gamma distribution.

• Gamma distribution
• Inverse-chi-squared distribution
• Normal distribution

• Hoff, P. (2009). "A first course in bayesian statistical methods". Springer.
• Witkovsky, V. (2001). "Computing the Distribution of a Linear Combination of Inverted Gamma Variables". Kybernetika. 37 (1): 79–90. MR 1825758. Zbl 1263.62022.