|
|
| (588 intermediate revisions by more than 100 users not shown) |
| Line 1: |
Line 1: |
| {{Distinguish|Gamma function}}{{refimprove|date=September 2012}}
| | This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users. |
|
| |
|
| {{Infobox probability distribution 2
| | If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]] |
| | name =Gamma
| | * Only registered users will be able to execute this rendering mode. |
| | type =density
| | * Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere. |
| | pdf_image =[[Image:Gamma distribution pdf.svg|325px|Probability density plots of gamma distributions]]
| |
| | cdf_image =[[Image:Gamma distribution cdf.svg|325px|Cumulative distribution plots of gamma distributions]]
| |
| | parameters =
| |
| * ''k'' > 0 [[shape parameter|shape]]
| |
| * θ > 0 [[scale parameter|scale]]
| |
| | support =<math>\scriptstyle x \;\in\; (0,\, \infty)</math>
| |
| | pdf =<math>\frac{1}{\Gamma(k) \theta^k} x^{k \,-\, 1} e^{-\frac{x}{\theta}}</math>
| |
| | cdf =<math>\frac{1}{\Gamma(k)} \gamma\left(k,\, \frac{x}{\theta}\right)</math>
| |
| | mean =<math>\scriptstyle \mathbf{E}[ X] = k \theta </math><br /><math>\scriptstyle \mathbf{E}[\ln X] = \psi(k) +\ln(\theta)</math><br />(see [[digamma function]])
| |
| | median =No simple closed form
| |
| | mode =<math>\scriptstyle (k \,-\, 1)\theta \text{ for } k \;{\geq}\; 1</math>
| |
| | variance =<math>\scriptstyle\operatorname{Var}[ X] = k \theta^2 </math><br/><math>\scriptstyle\operatorname{Var}[\ln X] = \psi_1(k)</math><br />(see [[trigamma function]])
| |
| | skewness =<math>\scriptstyle \frac{2}{\sqrt{k}}</math>
| |
| | kurtosis =<math>\scriptstyle \frac{6}{k}</math>
| |
| | entropy =<math>\scriptstyle \begin{align}
| |
| \scriptstyle k &\scriptstyle \,+\, \ln\theta \,+\, \ln[\Gamma(k)]\\
| |
| \scriptstyle &\scriptstyle \,+\, (1 \,-\, k)\psi(k)
| |
| \end{align}</math>
| |
| | mgf =<math>\scriptstyle (1 \,-\, \theta t)^{-k} \text{ for } t \;<\; \frac{1}{\theta}</math>
| |
| | char =<math>\scriptstyle (1 \,-\, \theta i\,t)^{-k}</math>
| |
| | parameters2 =
| |
| * α > 0 [[shape parameter|shape]] | |
| * β > 0 [[rate parameter|rate]] | |
| | support2 =<math>\scriptstyle x \;\in\; (0,\, \infty)</math>
| |
| | pdf2 =<math>\frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha \,-\, 1} e^{- \beta x }</math>
| |
| | cdf2 =<math>\frac{1}{\Gamma(\alpha)} \gamma(\alpha,\, \beta x)</math>
| |
| | mean2 =<math>\scriptstyle\mathbf{E}[ X] = \frac{\alpha}{\beta}</math><br /><math>\scriptstyle \mathbf{E}[\ln X] = \psi(\alpha) -\ln(\beta)</math><br />(see [[digamma function]])
| |
| | median2 =No simple closed form
| |
| | mode2 =<math>\scriptstyle \frac{\alpha \,-\, 1}{\beta} \text{ for } \alpha \;{\geq}\; 1</math>
| |
| | variance2 =<math>\scriptstyle \operatorname{Var}[ X] = \frac{\alpha}{\beta^2}</math><br/><math>\scriptstyle\operatorname{Var}[\ln X] = \psi_1(\alpha)</math><br />(see [[trigamma function]])
| |
| | skewness2 =<math>\scriptstyle \frac{2}{\sqrt{\alpha}}</math>
| |
| | kurtosis2 =<math>\scriptstyle \frac{6}{\alpha}</math>
| |
| | entropy2 =<math>\scriptstyle \begin{align}
| |
| \scriptstyle \alpha &\scriptstyle \,-\, \ln \beta \,+\, \ln[\Gamma(\alpha)]\\
| |
| \scriptstyle &\scriptstyle \,+\, (1 \,-\, \alpha)\psi(\alpha)
| |
| \end{align}</math>
| |
| | mgf2 =<math>\scriptstyle \left(1 \,-\, \frac{t}{\beta}\right)^{-\alpha} \text{ for } t \;<\; \beta</math>
| |
| | char2 =<math>\scriptstyle \left(1 \,-\, \frac{i\,t}{\beta}\right)^{-\alpha}</math>
| |
| }}
| |
|
| |
|
| In [[probability theory]] and [[statistics]], the '''gamma distribution''' is a two-parameter family of continuous [[probability distribution]]s. The common [[exponential distribution]] and [[chi-squared distribution]] are special cases of the gamma distribution. There are three different [[parametrization]]s in common use:
| | Registered users will be able to choose between the following three rendering modes: |
| #With a [[shape parameter]] ''k'' and a [[scale parameter]] θ.
| |
| #With a shape parameter ''α'' = ''k'' and an inverse scale parameter β = 1/θ, called a [[rate parameter]].
| |
| #With a shape parameter ''k'' and a mean parameter μ = ''k''/β.
| |
| In each of these three forms, both parameters are positive real numbers.
| |
|
| |
|
| The parameterization with ''k'' and θ appears to be more common in [[econometrics]] and certain other applied fields, where e.g. the gamma distribution is frequently used to model waiting times. For instance, in [[Accelerated life testing|life testing]], the waiting time until death is a [[random variable]] that is frequently modeled with a gamma distribution.<ref>See Hogg and Craig (1978, Remark 3.3.1) for an explicit motivation</ref>
| | '''MathML''' |
| | :<math forcemathmode="mathml">E=mc^2</math> |
|
| |
|
| The parameterization with α and β is more common in [[Bayesian statistics]], where the gamma distribution is used as a [[conjugate prior]] distribution for various types of inverse scale (aka rate) parameters, such as the λ of an [[exponential distribution]] or a [[Poisson distribution]]<ref>[http://arxiv.org/pdf/1311.1704v3.pdf ''Scalable Recommendation with Poisson Factorization''], Prem Gopalan, Jake M. Hofman, [[David Blei]], arXiv.org 2014</ref> – or for that matter, the β of the gamma distribution itself. (The closely related [[inverse gamma distribution]] is used as a conjugate prior for scale parameters, such as the [[variance]] of a [[normal distribution]].)
| | <!--'''PNG''' (currently default in production) |
| | :<math forcemathmode="png">E=mc^2</math> |
|
| |
|
| If ''k'' is an [[integer]], then the distribution represents an [[Erlang distribution]]; i.e., the sum of ''k'' independent [[exponential distribution|exponentially distributed]] [[random variable]]s, each of which has a mean of θ (which is equivalent to a rate parameter of 1/θ).
| | '''source''' |
| | :<math forcemathmode="source">E=mc^2</math> --> |
|
| |
|
| The gamma distribution is the [[maximum entropy probability distribution]] for a random variable ''X'' for which '''E'''[''X''] = ''k''θ = α/β is fixed and greater than zero, and '''E'''[ln(''X'')] = ψ(''k'') + ln(θ) = ψ(α) − ln(β) is fixed (ψ is the [[digamma function]]).<ref>{{cite journal |last1=Park |first1=Sung Y. |last2=Bera |first2=Anil K. |year=2009 |title=Maximum entropy autoregressive conditional heteroskedasticity model |journal=Journal of Econometrics |volume= |issue= |pages=219–230 |publisher=Elsevier |doi= |url=http://www.wise.xmu.edu.cn/Master/Download/..%5C..%5CUploadFiles%5Cpaper-masterdownload%5C2009519932327055475115776.pdf |accessdate=2011-06-02|doi=10.1016/j.jeconom.2008.12.014 }}</ref>
| | <span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples]. |
|
| |
|
| ==Characterization using shape ''k'' and scale θ== | | ==Demos== |
| A random variable ''X'' that is gamma-distributed with shape ''k'' and scale θ is denoted by
| |
|
| |
|
| :<math>X \sim \Gamma(k, \theta) \equiv \textrm{Gamma}(k, \theta)</math> | | Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]: |
|
| |
|
| ===Probability density function===
| |
| [[Image:Gamma-PDF-3D.png|thumb|right|320px|Illustration of the gamma PDF for parameter values over ''k'' and ''x'' with θ set to 1, 2, 3, 4, 5 and 6. One can see each θ layer by itself here [http://commons.wikimedia.org/wiki/File:Gamma-PDF-3D-by-k.png] as well as by ''k'' [http://commons.wikimedia.org/wiki/File:Gamma-PDF-3D-by-Theta.png] and ''x''. [http://commons.wikimedia.org/wiki/File:Gamma-PDF-3D-by-x.png].]]
| |
|
| |
|
| The [[probability density function]] using the shape-scale parametrization is
| | * accessibility: |
| | ** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)] |
| | ** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]]. |
| | ** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]]. |
| | ** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode. |
|
| |
|
| :<math>f(x;k,\theta) = \frac{x^{k-1}e^{-\frac{x}{\theta}}}{\theta^k\Gamma(k)} \quad \text{ for } x > 0 \text{ and } k, \theta > 0.</math>
| | ==Test pages == |
|
| |
|
| Here Γ(''k'') is the [[gamma function]] evaluated at ''k''.
| | To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages: |
| | *[[Displaystyle]] |
| | *[[MathAxisAlignment]] |
| | *[[Styling]] |
| | *[[Linebreaking]] |
| | *[[Unique Ids]] |
| | *[[Help:Formula]] |
|
| |
|
| ===Cumulative distribution function===
| | *[[Inputtypes|Inputtypes (private Wikis only)]] |
| The [[cumulative distribution function]] is the regularized gamma function:
| | *[[Url2Image|Url2Image (private Wikis only)]] |
| | | ==Bug reporting== |
| :<math> F(x;k,\theta) = \int_0^x f(u;k,\theta)\,du = \frac{\gamma\left(k, \frac{x}{\theta}\right)}{\Gamma(k)}</math>
| | If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de . |
| | |
| where γ(''k'', ''x''/θ) is the lower [[incomplete gamma function]].
| |
| | |
| It can also be expressed as follows, if ''k'' is a positive [[integer]] (i.e., the distribution is an [[Erlang distribution]]):<ref name="Papoulis">Papoulis, Pillai, ''Probability, Random Variables, and Stochastic Processes'', Fourth Edition</ref>
| |
| | |
| :<math>F(x;k,\theta) = 1-\sum_{i=0}^{k-1} \frac{1}{i!} \left(\frac{x}{\theta}\right)^i e^{-\frac{x}{\theta}} = e^{-\frac{x}{\theta}} \sum_{i=k}^{\infty} \frac{1}{i!} \left(\frac{x}{\theta}\right)^i</math>
| |
| <!-- The parameter θ used here is equivalent to the β defined above! -->
| |
| | |
| ==Characterization using shape α and rate β==
| |
| Alternatively, the gamma distribution can be parameterized in terms of a [[shape parameter]] α = ''k'' and an inverse scale parameter β = 1/θ, called a [[rate parameter]]. A random variable ''X'' that is gamma-distributed with shape ''α'' and rate ''β'' is denoted
| |
| | |
| :<math>X \sim \Gamma(\alpha, \beta) \equiv \textrm{Gamma}(\alpha,\beta)</math>
| |
| | |
| ===Probability density function===
| |
| The corresponding density function in the shape-rate parametrization is
| |
| | |
| :<math>g(x;\alpha,\beta) = \frac{\beta^{\alpha} x^{\alpha-1} e^{-x\beta}}{\Gamma(\alpha)} \quad \text{ for } x \geq 0 \text{ and } \alpha, \beta > 0</math>
| |
| | |
| Both parametrizations are common because either can be more convenient depending on the situation.
| |
| | |
| ===Cumulative distribution function===
| |
| The [[cumulative distribution function]] is the regularized gamma function:
| |
| | |
| :<math> F(x;\alpha,\beta) = \int_0^x f(u;\alpha,\beta)\,du= \frac{\gamma(\alpha, \beta x)}{\Gamma(\alpha)}</math>
| |
| | |
| where γ(α, β''x'') is the lower [[incomplete gamma function]].
| |
| | |
| If α is a positive [[integer]] (i.e., the distribution is an [[Erlang distribution]]), the cumulative distribution function has the following series expansion:<ref name="Papoulis"/>
| |
| | |
| :<math>F(x;\alpha,\beta) = 1-\sum_{i=0}^{\alpha-1} \frac{(\beta x)^i}{i!} e^{-\beta x} = e^{-\beta x} \sum_{i=\alpha}^{\infty} \frac{(\beta x)^i}{i!}</math>
| |
| | |
| ==Properties==
| |
| | |
| ===Skewness===
| |
| The skewness is equal to <math> 2/\sqrt{k} </math>, it depends only on the shape parameter (k) and approaches a normal distribution when k is large (approximately when k > 10).
| |
| | |
| ===Median calculation===
| |
| Unlike the mode and the mean which have readily calculable formulas based on the parameters, the median does not have an easy closed form equation. The median for this distribution is defined as the value ν such that
| |
| | |
| : <math>\frac{1}{\Gamma(k) \theta^k} \int_0^\nu x^{ k - 1 } e^{ - \frac{ x }{ \theta } } dx = \tfrac{1}{2}.</math>
| |
| | |
| A formula for approximating the median for any gamma distribution, when the mean is known, has been derived based on the fact that the ratio μ/(μ − ν) is approximately a linear function of ''k'' when ''k'' ≥ 1.<ref name=Banneheka2009>Banneheka BMSG, Ekanayake GEMUPD (2009) "A new point estimator for the median of gamma distribution". ''Viyodaya J Science'', 14:95–103</ref> The approximation formula is
| |
| | |
| : <math> \nu \approx \mu \frac{3 k - 0.8}{3 k + 0.2} ,</math>
| |
| | |
| where <math>\mu (=k\theta)</math> is the mean.
| |
| | |
| ===Summation===
| |
| If ''X''<sub>''i''</sub> has a Gamma(''k''<sub>''i''</sub>, θ) distribution for ''i'' = 1, 2, ..., ''N'' (i.e., all distributions have the same scale parameter θ), then
| |
| | |
| :<math> \sum_{i=1}^N X_i \sim\mathrm{Gamma} \left( \sum_{i=1}^N k_i, \theta \right)</math>
| |
| | |
| provided all ''X''<sub>''i''</sub> are [[statistical independence|independent]].
| |
| | |
| For the cases where the ''X''<sub>''i''</sub> are [[statistical independence|independent]] but have different scale parameters see Mathai (1982) and Moschopoulos (1984).
| |
| | |
| The gamma distribution exhibits [[Infinite divisibility (probability)|infinite divisibility]].
| |
| | |
| ===Scaling=== | |
| If | |
| | |
| : <math>X \sim \mathrm{Gamma}(k, \theta),</math>
| |
| | |
| then for any ''c'' > 0,
| |
| | |
| : <math>cX \sim \mathrm{Gamma}( k, c\theta).</math>
| |
| | |
| Hence the use of the term "[[scale parameter]]" to describe θ.
| |
| | |
| Equivalently, if
| |
| | |
| : <math>X \sim \mathrm{Gamma}(\alpha, \beta),</math> | |
| | |
| then for any ''c'' > 0,
| |
| | |
| : <math>cX \sim \mathrm{Gamma}( \alpha, \beta/c).</math>
| |
| | |
| Hence the use of the term "inverse scale parameter" to describe β.
| |
| | |
| ===Exponential family===
| |
| The gamma distribution is a two-parameter [[exponential family]] with [[natural parameters]] ''k'' − 1 and −1/θ (equivalently, α − 1 and −β), and [[natural statistics]] ''X'' and ln(''X'').
| |
| | |
| If the shape parameter k is held fixed, the resulting one-parameter family of distributions is a [[natural exponential family]].
| |
| | |
| ===Logarithmic expectation=== | |
| One can show that
| |
| | |
| : <math>\mathbf{E}[\ln(X)] = \psi(\alpha) - \ln(\beta)</math>
| |
| | |
| or equivalently,
| |
| | |
| : <math>\mathbf{E}[\ln(X)] = \psi(k) + \ln(\theta)</math>
| |
| | |
| where ψ is the [[digamma function]].
| |
| | |
| This can be derived using the [[exponential family]] formula for the [[exponential family#Moment generating function of the sufficient statistic|moment generating function of the sufficient statistic]], because one of the sufficient statistics of the gamma distribution is ln(''x'').
| |
| | |
| ===Information entropy===
| |
| The [[information entropy]] is
| |
| :<math>\operatorname{H}(X) = \mathbf{E}[-\ln(p(X))] = \mathbf{E}[-\alpha\ln(\beta) + \ln(\Gamma(\alpha)) - (\alpha-1)\ln(X) + \beta X] = \alpha - \ln(\beta) + \ln(\Gamma(\alpha)) + (1-\alpha)\psi(\alpha).</math>
| |
| | |
| In the ''k'', θ parameterization, the [[information entropy]] is given by
| |
| | |
| :<math>\operatorname{H}(X) =k + \ln(\theta) + \ln(\Gamma(k)) + (1-k)\psi(k).</math>
| |
| | |
| ===Kullback–Leibler divergence===
| |
| [[Image:Gamma-KL-3D.png|thumb|right|320px|Illustration of the Kullback–Leibler (KL) divergence for two gamma PDFs. Here β = β<sub>0</sub> + 1 which are set to 1, 2, 3, 4, 5 and 6. The typical asymmetry for the KL divergence is clearly visible.]]
| |
| | |
| The [[Kullback–Leibler divergence]] (KL-divergence), of Gamma(α<sub>''p''</sub>, β<sub>''p''</sub>) ("true" distribution) from Gamma(α<sub>''q''</sub>, β<sub>''q''</sub>) ("approximating" distribution) is given by<ref>W.D. Penny, [www.fil.ion.ucl.ac.uk/~wpenny/publications/densities.ps KL-Divergences of Normal, Gamma, Dirichlet, and Wishart densities]{{full|date=November 2012}}</ref>
| |
| :<math> D_{\mathrm{KL}}(\alpha_p,\beta_p; \alpha_q, \beta_q) = (\alpha_p-\alpha_q)\psi(\alpha_p) - \log\Gamma(\alpha_p) + \log\Gamma(\alpha_q) + \alpha_q(\log \beta_p - \log \beta_q) + \alpha_p\frac{\beta_q-\beta_p}{\beta_p} </math>
| |
| | |
| Written using the ''k'', θ parameterization, the KL-divergence of Gamma(''k<sub>p''</sub>, θ<sub>''p''</sub>) from Gamma(''k<sub>q''</sub>, θ<sub>''q''</sub>) is given by
| |
| :<math> D_{\mathrm{KL}}(k_p,\theta_p; k_q, \theta_q) = (k_p-k_q)\psi(k_p) - \log\Gamma(k_p) + \log\Gamma(k_q) + k_q(\log \theta_q - \log \theta_p) + k_p\frac{\theta_p - \theta_q}{\theta_q} </math>
| |
| | |
| === Laplace transform ===
| |
| The [[Laplace transform]] of the gamma PDF is
| |
| | |
| :<math>F(s) = (1 + \theta s)^{-k} = \frac{\beta^\alpha}{(s + \beta)^\alpha} .</math>
| |
| | |
| | |
| === [[Differential equation]] ===
| |
| | |
| | |
| <math>
| |
| \left\{\beta x f'(x)+f(x) (-\alpha \beta +\beta
| |
| +x)=0,f(1)=\frac{e^{-1/\beta } \beta ^{-\alpha }}{\Gamma (\alpha
| |
| )}\right\}
| |
| </math>
| |
| <br>
| |
| <math>
| |
| \left\{x f'(x)+f(x) (-k+\theta x+1)=0,f(1)=\frac{e^{-\theta }
| |
| \left(\frac{1}{\theta }\right)^{-k}}{\Gamma (k)}\right\}
| |
| </math>
| |
| | |
| == Parameter estimation ==
| |
| | |
| === Maximum likelihood estimation ===
| |
| The likelihood function for ''N'' [[independent and identically-distributed random variables|iid]] observations (''x''<sub>1</sub>, ..., ''x''<sub>''N''</sub>) is
| |
| | |
| :<math>L(k, \theta) = \prod_{i=1}^N f(x_i;k,\theta)</math>
| |
| | |
| from which we calculate the log-likelihood function
| |
| | |
| :<math>\ell(k, \theta) = (k - 1) \sum_{i=1}^N \ln{(x_i)} - \sum_{i=1}^N \frac{x_i}{\theta} - Nk\ln(\theta) - N\ln(\Gamma(k))</math>
| |
| | |
| Finding the maximum with respect to θ by taking the derivative and setting it equal to zero yields the [[maximum likelihood]] estimator of the θ parameter:
| |
| | |
| :<math>\hat{\theta} = \frac{1}{kN}\sum_{i=1}^N x_i</math>
| |
| | |
| Substituting this into the log-likelihood function gives
| |
| | |
| :<math>\ell = (k-1)\sum_{i=1}^N\ln{(x_i)} - Nk - Nk\ln{\left(\frac{\sum x_i}{kN}\right)} - N\ln(\Gamma(k))</math>
| |
| | |
| Finding the maximum with respect to ''k'' by taking the derivative and setting it equal to zero yields
| |
| | |
| :<math>\ln(k) - \psi(k) = \ln\left(\frac{1}{N}\sum_{i=1}^N x_i\right) - \frac{1}{N}\sum_{i=1}^N\ln(x_i)</math>
| |
| | |
| There is no closed-form solution for ''k''. The function is numerically very well behaved, so if a numerical solution is desired, it can be found using, for example, [[Newton's method]]. An initial value of ''k'' can be found either using the [[method of moments (statistics)|method of moments]], or using the approximation
| |
| | |
| :<math>\ln(k) - \psi(k) \approx \frac{1}{2k}\left(1 + \frac{1}{6k + 1}\right)</math>
| |
| | |
| If we let
| |
| | |
| :<math>s = \ln{\left(\frac{1}{N}\sum_{i=1}^N x_i\right)} - \frac{1}{N}\sum_{i=1}^N\ln{(x_i)}</math>
| |
| | |
| then ''k'' is approximately
| |
| | |
| :<math>k \approx \frac{3 - s + \sqrt{(s - 3)^2 + 24s}}{12s}</math>
| |
| | |
| which is within 1.5% of the correct value.<ref>Minka, Thomas P. (2002) "Estimating a Gamma distribution". http://research.microsoft.com/en-us/um/people/minka/papers/minka-gamma.pdf</ref> An explicit form for the Newton–Raphson update of this initial guess is:<ref>Choi, S.C.; Wette, R. (1969) "Maximum Likelihood Estimation of the Parameters of the Gamma Distribution and Their Bias", ''Technometrics'', 11(4) 683–690</ref>
| |
| | |
| :<math>k \leftarrow k - \frac{ \ln(k) - \psi(k) - s }{ \frac{1}{k} - \psi^{\prime}(k) }.</math>
| |
| | |
| === Bayesian minimum mean squared error ===
| |
| With known ''k'' and unknown θ, the posterior density function for theta (using the standard scale-invariant [[prior probability|prior]] for θ) is
| |
| :<math>P(\theta | k, x_1, \dots, x_N) \propto \frac{1}{\theta} \prod_{i=1}^N f(x_i; k, \theta)</math>
| |
| | |
| Denoting
| |
| | |
| :<math> y \equiv \sum_{i=1}^Nx_i , \qquad P(\theta | k, x_1, \dots, x_N) = C(x_i) \theta^{-N k-1} e^{-\frac{y}{\theta}}</math>
| |
| | |
| Integration over θ can be carried out using a change of variables, revealing that 1/θ is gamma-distributed with parameters α = ''Nk'', β = ''y''.
| |
| | |
| :<math>\int_0^{\infty} \theta^{-Nk - 1 + m} e^{-\frac{y}{\theta}}\, d\theta = \int_0^{\infty} x^{Nk - 1 - m} e^{-xy} \, dx = y^{-(Nk - m)} \Gamma(Nk - m) \!</math>
| |
| | |
| The moments can be computed by taking the ratio (''m'' by ''m'' = 0)
| |
| | |
| :<math>\mathbf{E} [x^m] = \frac {\Gamma (Nk - m)} {\Gamma(Nk)} y^m</math>
| |
| | |
| which shows that the mean ± standard deviation estimate of the posterior distribution for θ is
| |
| | |
| :<math> \frac {y} {Nk - 1} \pm \frac {y^2} {(Nk - 1)^2 (Nk - 2)} </math>
| |
| | |
| == Generating gamma-distributed random variables ==
| |
| Given the scaling property above, it is enough to generate gamma variables with θ = 1 as we can later convert to any value of β with simple division.
| |
| | |
| Using the fact that a Gamma(1, 1) distribution is the same as an Exp(1) distribution, and noting the method of [[exponential distribution#Generating exponential variates|generating exponential variables]], we conclude that if ''U'' is [[uniform distribution (continuous)|uniformly distributed]] on (0, 1], then −ln(''U'') is distributed Gamma(1, 1) Now, using the "α-addition" property of gamma distribution, we expand this result:
| |
| | |
| : <math>\sum_{k=1}^n {-\ln U_k} \sim \Gamma(n, 1)</math>
| |
| | |
| where ''U<sub>k</sub>'' are all uniformly distributed on (0, 1] and [[statistical independence|independent]]. All that is left now is to generate a variable distributed as Gamma(δ, 1) for 0 < δ < 1 and apply the "α-addition" property once more. This is the most difficult part.
| |
| | |
| Random generation of gamma variates is discussed in detail by Devroye,<ref>{{cite book|title=Non-Uniform Random Variate Generation|author=Luc Devroye|year=1986|publisher=Springer-Verlag|location=New York|url=http://luc.devroye.org/rnbookindex.html|ref=harv}} See Chapter 9, Section 3, pages 401–428.</ref> noting that none are uniformly fast for all shape parameters. For small values of the shape parameter, the algorithms are often not valid.<ref>Devroye (1986), p. 406.</ref> For arbitrary values of the shape parameter, one can apply the Ahrens and Dieter<ref name=AD>Ahrens, J. H. and Dieter, U. (1982). Generating gamma variates by a modified rejection technique. ''Communications of the ACM'', 25, 47–54. Algorithm GD, p. 53.</ref> modified acceptance–rejection method Algorithm GD (shape ''k'' ≥ 1), or transformation method<ref>{{cite journal | last1 = Ahrens | first1 = J. H. | last2 = Dieter | first2 = U. | year = 1974 | title = Computer methods for sampling from gamma, beta, Poisson and binomial distributions | journal = Computing | volume = 12 | pages = 223–246 | id = {{citeseerx|10.1.1.93.3828}} | doi=10.1007/BF02293108}}</ref> when 0 < ''k'' < 1. Also see Cheng and Feast Algorithm GKM 3<ref>Cheng, R.C.H., and Feast, G.M. Some simple gamma variate generators. Appl. Stat. 28 (1979), 290–295.</ref> or Marsaglia's squeeze method.<ref>Marsaglia, G. The squeeze method for generating gamma variates. Comput, Math. Appl. 3 (1977), 321–325.</ref>
| |
| | |
| The following is a version of the Ahrens-Dieter [[rejection sampling|acceptance–rejection method]]:<ref name=AD/>
| |
| | |
| # Let ''m'' be 1.
| |
| # Generate ''V<sub>3m−2</sub>'', ''V<sub>3m−1</sub>'' and ''V<sub>3m</sub>'' as independent uniformly distributed on (0, 1] variables.
| |
| # If <math>V_{3m - 2} \le v_0</math>, where <math>v_0 = \frac{e}{e + \delta}</math>, then go to step 4, else go to step 5.
| |
| # Let <math>\xi_m = V_{3m - 1}^{1 / \delta}, \ \eta_m = V_{3m} \xi_m^{\delta - 1}</math>. Go to step 6.
| |
| # Let <math>\xi_m = 1 - \ln {V_{3m - 1}}, \ \eta_m = V_{3m} e^{-\xi_m}</math>.
| |
| # If <math>\eta_m > \xi_m^{\delta - 1} e^{-\xi_m}</math>, then increment ''m'' and go to step 2.
| |
| # Assume ξ = ξ<sub>''m''</sub> to be the realization of Γ(δ, 1).
| |
| | |
| A summary of this is
| |
| | |
| : <math> \theta \left( \xi - \sum_{i=1}^{\lfloor{k}\rfloor} {\ln(U_i)} \right) \sim \Gamma (k, \theta)</math>
| |
| | |
| where
| |
| | |
| * <math>\scriptstyle \lfloor{k}\rfloor</math> is the integral part of ''k'',
| |
| * ξ has been generated using the algorithm above with δ = {''k''} (the fractional part of ''k''),
| |
| * ''U<sub>k</sub>'' and ''V<sub>l</sub>'' are distributed as explained above and are all independent.
| |
| | |
| While the above approach is technically correct, Devroye notes that it is linear in the value of ''k'' and in general is not a good choice. Instead he recommends using either rejection-based or table-based methods, depending on context.<ref>{{cite book|title=Non-Uniform Random Variate Generation|author=Luc Devroye|year=1986|publisher=Springer-Verlag|location=New York|url=http://luc.devroye.org/rnbookindex.html}} See Chapter 9, Section 3, pages 401–428.</ref>
| |
| | |
| ==Related distributions==
| |
| | |
| === Special cases ===
| |
| | |
| === Conjugate prior ===
| |
| In [[Bayesian inference]], the '''gamma distribution''' is the [[conjugate prior]] to many likelihood distributions: the [[Poisson distribution|Poisson]], [[Exponential distribution|exponential]], [[Normal distribution|normal]] (with known mean), [[Pareto distribution|Pareto]], gamma with known shape σ, [[Inverse-gamma distribution|inverse gamma]] with known shape parameter, and [[Gompertz distribution|Gompertz]] with known scale parameter. <!-- reference: see article [[conjugate prior]] //-->
| |
| | |
| The gamma distribution's [[conjugate prior]] is:<ref name="fink">Fink, D. 1995 [http://www.stat.columbia.edu/~cook/movabletype/mlm/CONJINTRnew%2BTEX.pdf A Compendium of Conjugate Priors]. In progress report: Extension and enhancement of methods for setting data quality objectives. (DOE contract 95‑831).</ref>
| |
| | |
| :<math>p(k,\theta | p, q, r, s) = \frac{1}{Z} \frac{p^{k-1} e^{-\theta^{-1} q}}{\Gamma(k)^r \theta^{k s}},</math>
| |
| | |
| where ''Z'' is the normalizing constant, which has no closed-form solution.
| |
| The posterior distribution can be found by updating the parameters as follows:
| |
| | |
| :<math>\begin{align}
| |
| p' &= p\prod\nolimits_i x_i,\\
| |
| q' &= q + \sum\nolimits_i x_i,\\
| |
| r' &= r + n,\\
| |
| s' &= s + n,
| |
| \end{align}</math>
| |
| | |
| where ''n'' is the number of observations, and ''x<sub>i</sub>'' is the ''i''th observation.
| |
| | |
| === Compound gamma ===
| |
| If the shape parameter of the gamma distribution is known, but the inverse-scale parameter is unknown, then a gamma distribution for the inverse-scale forms a conjugate prior. The [[compound distribution]], which results from integrating out the inverse-scale has a closed form solution, known as the [[compound gamma distribution]].<ref name=Dubey>{{cite journal|last=Dubey|first=Satya D. | title=Compound gamma, beta and F distributions|journal=Metrika|date=December 1970|volume=16|pages=27–31 |doi=10.1007/BF02613934| url=http://www.springerlink.com/content/u750hg4630387205/}}</ref>
| |
| | |
| === Others ===
| |
| * If ''X'' ~ Gamma(1, λ), then ''X'' has an [[exponential distribution]] with rate parameter λ.
| |
| * If ''X'' ~ Gamma(ν/2, 2), then ''X'' is identical to χ<sup>2</sup>(ν), the [[chi-squared distribution]] with ν degrees of freedom. Conversely, if ''Q'' ~ χ<sup>2</sup>(ν) and ''c'' is a positive constant, then ''cQ'' ~ Gamma(ν/2, 2''c'').
| |
| * If ''k'' is an [[integer]], the gamma distribution is an [[Erlang distribution]] and is the probability distribution of the waiting time until the ''k''th "arrival" in a one-dimensional [[Poisson process]] with intensity 1/θ. If
| |
| ::<math>X \sim \Gamma(k \in \mathbf{Z}, \theta), \qquad Y \sim \mathrm{Pois}\left(\frac{x}{\theta}\right),</math>
| |
| :then
| |
| ::<math>P(X > x) = P(Y < k).</math>
| |
| * If ''X'' has a [[Maxwell–Boltzmann distribution]] with parameter ''a'', then
| |
| ::<math>X^2 \sim \Gamma\left(\tfrac{3}{2}, 2a^2\right)</math>.
| |
| <!--
| |
| * <math>Y \sim N(\mu = \alpha \beta, \sigma^2 = \alpha \beta^2)</math> is a [[normal distribution]] as <math>Y = \lim_{\alpha \to \infty} X</math> where ''X'' ~ Gamma(α, β). -->
| |
| * If ''X'' ~ Gamma(''k'', θ), then <math>\sqrt{X}</math> follows a [[generalized gamma distribution]] with parameters ''p'' = 2, ''d'' = 2''k'', and <math>a = \sqrt{\theta}</math> {{citation needed|date=September 2012}} .
| |
| * If ''X'' ~ Gamma(''k'', θ), then 1/''X'' ~ Inv-Gamma(''k'', θ<sup>-1</sup>) (see [[Inverse-gamma distribution]] for derivation).
| |
| * If ''X'' ~ Gamma(α, θ) and ''Y'' ~ Gamma(β, θ) are independently distributed, then ''X''/(''X'' + ''Y'') has a [[beta distribution]] with parameters α and β.
| |
| * If ''X<sub>i</sub>'' ~ Gamma(α<sub>''i''</sub>, 1) are independently distributed, then the vector (''X''<sub>1</sub>/''S'', ..., ''X<sub>n</sub>''/''S''), where ''S'' = ''X''<sub>1</sub> + ... + ''X<sub>n</sub>'', follows a [[Dirichlet distribution]] with parameters α<sub>1</sub>, ..., α<sub>''n''</sub>.
| |
| * For large ''k'' the gamma distribution converges to Gaussian distribution with mean μ = ''k''θ and variance σ<sup>2</sup> = ''k''θ<sup>2</sup>.
| |
| * The gamma distribution is the [[conjugate prior]] for the precision of the [[normal distribution]] with known [[mean]].
| |
| * The [[Wishart distribution]] is a multivariate generalization of the gamma distribution (samples are positive-definite matrices rather than positive real numbers).
| |
| * The gamma distribution is a special case of the [[generalized gamma distribution]], the [[generalized integer gamma distribution]], and the [[generalized inverse Gaussian distribution]].
| |
| * Among the discrete distributions, the [[negative binomial distribution]] is sometimes considered the discrete analogue of the Gamma distribution.
| |
| * [[Tweedie distribution]]s – the gamma distribution is a member of the family of Tweedie [[exponential dispersion model]]s.
| |
| | |
| ==Applications==
| |
| | |
| {{Expand section|date=March 2009}}
| |
| | |
| The gamma distribution has been used to model the size of [[insurance policy|insurance claims]]<ref>p. 43, Philip J. Boland, Statistical and Probabilistic Methods in Actuarial Science, Chapman & Hall CRC 2007</ref> and rainfalls.<ref name="Aksoy">Aksoy, H. (2000) [http://journals.tubitak.gov.tr/engineering/issues/muh-00-24-6/muh-24-6-7-9909-13.pdf "Use of Gamma Distribution in Hydrological Analysis"], ''Turk J. Engin Environ Sci'', 24, 419 – 428.</ref> This means that aggregate insurance claims and the amount of rainfall accumulated in a reservoir are modelled by a [[gamma process]]. The gamma distribution is also used to model errors in multi-level [[Poisson regression]] models, because the combination of the [[Poisson distribution]] and a gamma distribution is a [[negative binomial distribution]].
| |
| | |
| In [[neuroscience]], the gamma distribution is often used to describe the distribution of [[Temporal coding|inter-spike intervals]].<ref name="Robson">J. G. Robson and J. B. Troy, "Nature of the maintained discharge of Q, X, and Y retinal ganglion cells of the cat", J. Opt. Soc. Am. A 4, 2301–2307 (1987)</ref> Although in practice the gamma distribution often provides a good fit, there is no underlying biophysical motivation for using it.
| |
| | |
| In [[Bacterial genetics|bacterial]] [[gene expression]], the [[Copy number analysis|copy number]] of a [[constitutively expressed]] protein often follows the gamma distribution, where the scale and shape parameter are, respectively, the mean number of bursts per cell cycle and the mean number of [[protein molecule]]s produced by a single mRNA during its lifetime.<ref name="Friedman">N. Friedman, L. Cai and X. S. Xie (2006) "Linking stochastic dynamics to population distribution: An analytical framework of gene expression", ''Phys. Rev. Lett.'' 97, 168302.</ref>
| |
| | |
| In [[genomics]], the gamma distribution was applied in [[peak calling]] step (i.e. in recognition of signal) in [[ChIP-chip]]<ref name="Reiss">DJ Reiss, MT Facciotti and NS Baliga (2008) [http://bioinformatics.oxfordjournals.org/content/24/3/396.full.pdf+html "Model-based deconvolution of genome-wide DNA binding"], ''Bioinformatics'', 24, 396–403 </ref> and [[ChIP-seq]]<ref name="Mendoza">MA Mendoza-Parra, M Nowicka, W Van Gool, H Gronemeyer (2013) [http://www.biomedcentral.com/1471-2164/14/834 "Characterising ChIP-seq binding patterns by model-based peak shape deconvolution"], ''BMC Genomics'', 14:834</ref> data analysis.
| |
| | |
| The gamma distribution is widely used as a [[conjugate prior]] in Bayesian statistics. It is the conjugate prior for the precision (i.e. inverse of the variance) of a [[normal distribution]]. It is also the conjugate prior for the [[exponential distribution]].
| |
| | |
| ==Notes==
| |
| | |
| <references/>
| |
| | |
| == References ==
| |
| | |
| * R. V. Hogg and A. T. Craig (1978) ''Introduction to Mathematical Statistics'', 4th edition. New York: Macmillan. (See Section 3.3.)'
| |
| * P. G. Moschopoulos (1985) ''The distribution of the sum of independent gamma random variables'', '''Annals of the Institute of Statistical Mathematics''', 37, 541–544
| |
| * A. M. Mathai (1982) ''Storage capacity of a dam with gamma type inputs'', '''Annals of the Institute of Statistical Mathematics''', 34, 591–597
| |
| | |
| ==External links==
| |
| | |
| {{wikibooks|Statistics|Distributions/Gamma|Gamma distribution}}
| |
| * {{springer|title=Gamma-distribution|id=p/g043300}}
| |
| * {{MathWorld|urlname=GammaDistribution|title=Gamma distribution}}
| |
| * [http://www.itl.nist.gov/div898/handbook/eda/section3/eda366b.htm Engineering Statistics Handbook]
| |
| | |
| {{ProbDistributions|continuous-semi-infinite}}
| |
| {{Common univariate probability distributions}}
| |
| | |
| {{DEFAULTSORT:Gamma Distribution}}
| |
| [[Category:Continuous distributions]]
| |
| [[Category:Factorial and binomial topics]]
| |
| [[Category:Conjugate prior distributions]]
| |
| [[Category:Exponential family distributions]]
| |
| [[Category:Infinitely divisible probability distributions]]
| |
| [[Category:Probability distributions]]
| |
