en>Randomguess at 09:50, 24 February 2012

2012-02-24T09:50:03Z

New page

{{more footnotes|date=March 2011}}

In [[probability theory]], an '''empirical measure''' is a [[random measure]] arising from a particular realization of a (usually finite) sequence of [[random variable]]s. The precise definition is found below. Empirical measures are relevant to [[mathematical statistics]].

The motivation for studying empirical measures is that it is often impossible to know the true underlying [[probability measure]] <math>P</math>. We collect observations <math>X_1, X_2, \dots , X_n</math> and compute [[relative frequencies]]. We can estimate <math>P</math>, or a related distribution function <math>F</math> by means of the empirical measure or empirical distribution function, respectively. These are uniformly good estimates under certain conditions. Theorems in the area of [[empirical process]]es provide rates of this convergence.

==Definition==

Let <math>X_1, X_2, \dots</math> be a sequence of [[independent random variables|independent]] identically distributed [[random variable]]s with values in the state space ''S'' with [[probability measure]] ''P''.

'''Definition'''
:The ''empirical measure'' ''P''''n'' is defined for measurable subsets of ''S'' and given by
::<math>P_n(A) = {1 \over n} \sum_{i=1}^n I_A(X_i)=\frac{1}{n}\sum_{i=1}^n \delta_{X_i}(A)</math>
:where <math>I_A</math> is the [[indicator function]] and <math>\delta_X</math> is the [[Dirac measure]].

For a fixed measurable set ''A'', ''nP''''n''(''A'') is a [[binomial distribution|binomial]] random variable with mean ''nP''(''A'') and variance ''nP''(''A'')(1 − ''P''(''A'')). In particular, ''P''''n''(''A'') is an [[bias of an estimator|unbiased estimator]] of ''P''(''A'').

'''Definition'''
:<math>\bigl(P_n(c)\bigr)_{c\in\mathcal{C}}</math> is the ''empirical measure'' indexed by <math>\mathcal{C}</math>, a collection of measurable subsets of ''S''.

To generalize this notion further, observe that the empirical measure <math>P_n</math> maps [[measurable function]]s <math>f:S\to \mathbb{R}</math> to their ''[[empirical mean]]'',

:<math>f\mapsto P_n f=\int_S f \, dP_n=\frac{1}{n}\sum_{i=1}^n f(X_i)</math>

In particular, the empirical measure of ''A'' is simply the empirical mean of the indicator function, ''P''''n''(''A'') = ''P''''n'' ''I''''A''.

For a fixed measurable function <math>f</math>, <math>P_nf</math> is a random variable with mean <math>\mathbb{E}f</math> and variance <math>\frac{1}{n}\mathbb{E}(f -\mathbb{E} f)^2</math>.

By the strong [[law of large numbers]], ''P''n(''A'') converges to ''P''(''A'') [[almost surely]] for fixed ''A''. Similarly <math>P_nf</math> converges to <math>\mathbb{E} f</math> almost surely for a fixed measurable function <math>f</math>. The problem of uniform convergence of ''P''''n'' to ''P'' was open until [[Vapnik]] and [[Chervonenkis]] solved it in 1968.<ref>{{cite journal|last=Vapnik|first=V.|coauthors=Chervonenkis, A|title=Uniform convergence of frequencies of occurrence of events to their probabilities|journal=Dokl. Akad. Nauk SSSR|year=1968|volume=181}}</ref>

If the class <math>\mathcal{C}</math> (or <math>\mathcal{F}</math>) is [[Glivenko–Cantelli class|Glivenko–Cantelli]] with respect to ''P'' then ''P''n'' converges to ''P'' uniformly over <math>c\in\mathcal{C}</math> (or <math>f\in \mathcal{F}</math>). In other words, with probability 1 we have
:<math>\|P_n-P\|_\mathcal{C}=\sup_{c\in\mathcal{C}}|P_n(c)-P(c)|\to 0,</math>
:<math>\|P_n-P\|_\mathcal{F}=\sup_{f\in\mathcal{F}}|P_nf-\mathbb{E}f|\to 0.</math>

==Empirical distribution function==
{{main|Empirical distribution function}}
The ''empirical distribution function'' provides an example of empirical measures. For real-valued [[iid]] random variables <math>X_1,\dots,X_n</math> it is given by

:<math>F_n(x)=P_n((-\infty,x])=P_nI_{(-\infty,x]}.</math>

In this case, empirical measures are indexed by a class <math>\mathcal{C}=\{(-\infty,x]:x\in\mathbb{R}\}.</math> It has been shown that <math>\mathcal{C}</math> is a uniform [[Glivenko–Cantelli class]], in particular,

:<math>\sup_F\|F_n(x)-F(x)\|_\infty\to 0</math>

with probability 1.

==See also==
* [[Poisson random measure]]

==References==
{{Reflist}}

==Further reading==
*{{cite book |first=P. |last=Billingsley |title=Probability and Measure |publisher=John Wiley and Sons |location=New York |edition=Third |year=1995 |isbn=0-471-80478-9 }}
*{{cite journal |first=M. D. |last=Donsker |title=Justification and extension of Doob's heuristic approach to the [[Kolmogorov–Smirnov]] theorems |journal=[[Annals of Mathematical Statistics]] |volume=23 |issue=2 |pages=277–281 |year=1952 |doi=10.1214/aoms/1177729445 }}
*{{cite journal |first=R. M. |last=Dudley |title=Central limit theorems for empirical measures |journal=[[Annals of Probability]] |volume=6 |issue=6 |pages=899–929 |year=1978 |jstor=2243028 }}
*{{cite book |first=R. M. |last=Dudley |title=Uniform Central Limit Theorems |series=Cambridge Studies in Advanced Mathematics |volume=63 |publisher=Cambridge University Press |location=Cambridge, UK |year=1999 |isbn=0-521-46102-2 }}
*{{cite journal |first=J. |last=Wolfowitz |title=Generalization of the theorem of Glivenko–Cantelli |journal=Annals of Mathematical Statistics |volume=25 |issue=1 |pages=131–138 |year=1954 |jstor=2236518 }}

[[Category:Probability theory]]
[[Category:Measures (measure theory)]]
[[Category:Empirical process]]

Marchenko equation - Revision history

en>Randomguess at 09:50, 24 February 2012