# Convergence of random variables

In probability theory, there exist several different notions of **convergence of random variables**. The convergence of sequences of random variables to some limit random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. The same concepts are known in more general mathematics as **stochastic convergence** and they formalize the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behaviour that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behaviour can be characterised: two readily understood behaviours are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.

## Contents

## Background

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be

- Convergence in the classical sense to a fixed value, perhaps itself coming from a random event
- An increasing similarity of outcomes to what a purely deterministic function would produce
- An increasing preference towards a certain outcome
- An increasing "aversion" against straying far away from a certain outcome

Some less obvious, more theoretical patterns could be

- That the probability distribution describing the next outcome may grow increasingly similar to a certain distribution
- That the series formed by calculating the expected value of the outcome's distance from a particular value may converge to 0
- That the variance of the random variable describing the next event grows smaller and smaller.

These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied.

While the above discussion has related to the convergence of a single series to a limiting value, the notion of the convergence of two series towards each other is also important, but this is easily handled by studying the sequence defined as either the difference or the ratio of the two series.

For example, if the average of *n* independent random variables *Y*_{i}, *i* = 1, ..., *n*, all having the same finite mean and variance, is given by

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}={\frac {1}{n}}\sum _{{i=1}}^{n}Y_{i}\,,**

then as *n* tends to infinity, Template:Mvar converges *in probability* (see below) to the common mean, μ, of the random variables *Y*_{i}. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem.

Throughout the following, we assume that (*X*_{n}) is a sequence of random variables, and *X* is a random variable, and all of them are defined on the same probability space **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \scriptstyle (\Omega ,{\mathcal {F}},P)**
.

## Convergence in distribution

With this mode of convergence, we increasingly expect to see the next outcome in a sequence of random experiments becoming better and better modeled by a given probability distribution.

Convergence in distribution is the weakest form of convergence, since it is implied by all other types of convergence mentioned in this article. However convergence in distribution is very frequently used in practice; most often it arises from application of the central limit theorem.

### Definition

A sequence Template:Math of random variables is said to **converge in distribution**, or **converge weakly**, or **converge in law** to a random variable Template:Mvar if

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n\to\infty} F_n(x) = F(x),**

for every number Template:Math at which Template:Mvar is continuous. Here Template:Mvar and Template:Mvar are the cumulative distribution functions of random variables Template:Mvar and Template:Mvar, respectively.

The requirement that only the continuity points of Template:Mvar should be considered is essential. For example if Template:Mvar are distributed uniformly on intervals Template:Math, then this sequence converges in distribution to a degenerate random variable Template:Math. Indeed, Template:Math for all *n* when Template:Math, and Template:Math for all Template:Math when Template:Math. However, for this limiting random variable Template:Math, even though Template:Math for all Template:Mvar. Thus the convergence of cdfs fails at the point Template:Math where Template:Mvar is discontinuous.

Convergence in distribution may be denoted as

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\begin{aligned}&X_{n}\ {\xrightarrow {d}}\ X,\ \ X_{n}\ {\xrightarrow {{\mathcal {D}}}}\ X,\ \ X_{n}\ {\xrightarrow {{\mathcal {L}}}}\ X,\ \ X_{n}\ {\xrightarrow {d}}\ {\mathcal {L}}_{X},\\&X_{n}\rightsquigarrow X,\ \ X_{n}\Rightarrow X,\ \ {\mathcal {L}}(X_{n})\to {\mathcal {L}}(X),\\\end{aligned}}**

where **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \scriptstyle {\mathcal {L}}_{X}**
is the law (probability distribution) of Template:Mvar. For example if Template:Mvar is standard normal we can write **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\,{\xrightarrow {d}}\,{\mathcal {N}}(0,\,1)**
.

For random vectors Template:Math the convergence in distribution is defined similarly. We say that this sequence **converges in distribution** to a random Template:Mvar-vector Template:Mvar if

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n\to\infty} \operatorname{Pr}(X_n\in A) = \operatorname{Pr}(X\in A)**

for every Template:Math which is a continuity set of Template:Mvar.

The definition of convergence in distribution may be extended from random vectors to more general random elements in arbitrary metric spaces, and even to the “random variables” which are not measurable — a situation which occurs for example in the study of empirical processes. This is the “weak convergence of laws without laws being defined” — except asymptotically.^{[1]}

In this case the term **weak convergence** is preferable (see weak convergence of measures), and we say that a sequence of random elements Template:Math converges weakly to Template:Mvar (denoted as Template:Math) if

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \operatorname{E}^*h(X_n) \to \operatorname{E}\,h(X)**

for all continuous bounded functions Template:Mvar.^{[2]} Here E* denotes the *outer expectation*, that is the expectation of a “smallest measurable function Template:Mvar that dominates Template:Math”.

### Properties

- Since Template:Math, the convergence in distribution means that the probability for Template:Mvar to be in a given range is approximately equal to the probability that the value of Template:Mvar is in that range, provided Template:Mvar is sufficiently large.

- In general, convergence in distribution does not imply that the sequence of corresponding probability density functions will also converge. As an example one may consider random variables with densities Template:Math. These random variables converge in distribution to a uniform
*U*(0, 1), whereas their densities do not converge at all.^{[3]}- However, Scheffé’s lemma implies that convergence of the probability density functions implies convergence in distribution.
^{[4]}

- However, Scheffé’s lemma implies that convergence of the probability density functions implies convergence in distribution.

- The
**portmanteau lemma**provides several equivalent definitions of convergence in distribution. Although these definitions are less intuitive, they are used to prove a number of statistical theorems. The lemma states that Template:Math converges in distribution to Template:Mvar if and only if any of the following statements are true:Template:Citation needed- Eƒ(
*X*) → Eƒ(_{n}*X*) for all bounded, continuous functions ƒ (where E denotes the expected value); - Eƒ(
*X*) → Eƒ(_{n}*X*) for all bounded, Lipschitz functions ƒ; - limsup{ Eƒ(
*X*) } ≤ Eƒ(_{n}*X*) for every upper semi-continuous function ƒ bounded from above; - liminf{ Eƒ(
*X*) } ≥ Eƒ(_{n}*X*) for every lower semi-continuous function ƒ bounded from below; - Template:Math for all closed sets Template:Mvar;
- Template:Math for all open sets Template:Mvar;
- Template:Math for all continuity sets Template:Mvar of random variable Template:Mvar.

- Eƒ(

- The
**continuous mapping theorem**states that for a continuous function Template:Mvar, if the sequence Template:Math converges in distribution to Template:Mvar, then Template:Math converges in distribution to Template:Math.- Note however that convergence in distribution of Template:Math to Template:Mvar and Template:Math to Template:Mvar does in general
*not*imply convergence in distribution of Template:Math to Template:Math or of Template:Math to Template:Mvar.

- Note however that convergence in distribution of Template:Math to Template:Mvar and Template:Math to Template:Mvar does in general

**Lévy’s continuity theorem:**the sequence Template:Math converges in distribution to Template:Mvar if and only if the sequence of corresponding characteristic functions Template:Math converges pointwise to the characteristic function Template:Mvar of Template:Mvar.

- Convergence in distribution is metrizable by the Lévy–Prokhorov metric.

- A natural link to convergence in distribution is the Skorokhod's representation theorem.

## Convergence in probability

The basic idea behind this type of convergence is that the probability of an “unusual” outcome becomes smaller and smaller as the sequence progresses.

The concept of convergence in probability is used very often in statistics. For example, an estimator is called consistent if it converges in probability to the quantity being estimated. Convergence in probability is also the type of convergence established by the weak law of large numbers.

### Definition

A sequence {*X*_{n}} of random variables **converges in probability** towards the random variable *X* if for all *ε* > 0

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n\to\infty}\Pr\big(|X_n-X| \geq \varepsilon\big) = 0.**

Formally, pick any Template:Math and any Template:Math. Let Template:Mvar be the probability that Template:Mvar is outside the ball of radius *ε* centered at *X*. Then for Template:Mvar to converge in probability to *X* there should exist a number *N* (which will depend on *ε* and *δ*) such that for all Template:Math, Template:Mvar.

Convergence in probability is denoted by adding the letter *p* over an arrow indicating convergence, or using the “plim” probability limit operator:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n \ \xrightarrow{p}\ X,\ \ X_n \ \xrightarrow{P}\ X,\ \ \underset{n\to\infty}{\operatorname{plim}}\, X_n = X.**

For random elements {*X*_{n}} on a separable metric space Template:Math, convergence in probability is defined similarly by^{[5]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \forall\varepsilon>0, \Pr\big(d(X_n,X)\geq\varepsilon\big) \to 0.**

### Properties

- Convergence in probability implies convergence in distribution.
^{[proof]}

- In the opposite direction, convergence in distribution implies convergence in probability when the limiting random variable
*X*is a constant.^{[proof]}

- Convergence in probability does not imply almost sure convergence.
^{[proof]}

- The continuous mapping theorem states that for every continuous function
*g*(·), if**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \scriptstyle X_{n}{\xrightarrow {p}}X**, then also**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \scriptstyle g(X_{n}){\xrightarrow {p}}g(X)**.

- Convergence in probability defines a topology on the space of random variables over a fixed probability space. This topology is metrizable by the
*Ky Fan metric*:^{[6]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): d(X,Y) = \inf\!\big\{ \varepsilon>0:\ \Pr\big(|X-Y|>\varepsilon\big)\leq\varepsilon\big\}**

or

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): d(X,Y)={\mathbb E}\left[\min(|X-Y|,1)\right]**.

## Almost sure convergence

This is the type of stochastic convergence that is most similar to pointwise convergence known from elementary real analysis.

### Definition

To say that the sequence Template:Mvar converges **almost surely** or **almost everywhere** or **with probability 1** or **strongly** towards *X* means that

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \operatorname{Pr}\!\left( \lim_{n\to\infty}\! X_n = X \right) = 1.**

This means that the values of Template:Mvar approach the value of *X*, in the sense (see almost surely) that events for which Template:Mvar does not converge to *X* have probability 0. Using the probability space **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \scriptstyle (\Omega, \mathcal{F}, \operatorname{Pr} )**
and the concept of the random variable as a function from Ω to **R**, this is equivalent to the statement

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \operatorname{Pr}\Big( \omega \in \Omega : \lim_{n \to \infty} X_n(\omega) = X(\omega) \Big) = 1.**

Using the notion of the limit inferior of a sequence of sets, almost sure convergence can also be defined as follows:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \operatorname{Pr}\Big( \liminf_{n\to\infty} \big\{\omega \in \Omega : | X_n(\omega) - X(\omega) | < \varepsilon \big\} \Big) = 1 \quad\text{for all}\quad \varepsilon>0.**

Almost sure convergence is often denoted by adding the letters *a.s.* over an arrow indicating convergence:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n \, \xrightarrow{\mathrm{a.s.}} \, X.**

For generic random elements {*X _{n}*} on a metric space (

*S*,

*d*), convergence almost surely is defined similarly:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \operatorname{Pr}\Big( \omega\in\Omega:\, d\big(X_n(\omega),X(\omega)\big)\,\underset{n\to\infty}{\longrightarrow}\,0 \Big) = 1**

### Properties

- Almost sure convergence implies convergence in probability (by Fatou's lemma), and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers.
- The concept of almost sure convergence does not come from a topology on the space of random variables. This means there is no topology on the space of random variables such that the almost surely convergent sequences are exactly the converging sequences with respect to that topology. In particular, there is no metric of almost sure convergence.

## Sure convergence

To say that the sequence of random variables (*X*_{n}) defined over the same probability space (i.e., a random process) converges **surely** or **everywhere** or **pointwise** towards *X* means

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n\to\infty}X_n(\omega)=X(\omega), \, \, \forall \omega \in \Omega.**

where Ω is the sample space of the underlying probability space over which the random variables are defined.

This is the notion of pointwise convergence of sequence functions extended to sequence of random variables. (Note that random variables themselves are functions).

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\big \{}\omega \in \Omega \,|\,\lim _{{n\to \infty }}X_{n}(\omega )=X(\omega ){\big \}}=\Omega .**

Sure convergence of a random variable implies all the other kinds of convergence stated above, but there is no payoff in probability theory by using sure convergence compared to using almost sure convergence. The difference between the two only exists on sets with probability zero. This is why the concept of sure convergence of random variables is very rarely used.

## Convergence in mean

Given a real number Template:Math, we say that the sequence Template:Mvar converges **in the r-th mean** (or

**in the**) towards the random variable

*L*-norm^{r}*X*, if the Template:Mvar-th absolute moments E(|

*X*|

_{n}^{r}) and E(|

*X*|

^{r}) of Template:Mvar and

*X*exist, and

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n\to\infty} \operatorname{E}\left( |X_n-X|^r \right) = 0,**

where the operator E denotes the expected value. Convergence in Template:Mvar-th mean tells us that the expectation of the Template:Mvar-th power of the difference between *X _{n}* and

*X*converges to zero.

This type of convergence is often denoted by adding the letter *L ^{r}* over an arrow indicating convergence:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\,{\xrightarrow {L^{r}}}\,X.**

The most important cases of convergence in *r*-th mean are:

- When Template:Mvar converges in
*r*-th mean to*X*for*r*= 1, we say that Template:Mvar converges**in mean**to*X*. - When Template:Mvar converges in
*r*-th mean to*X*for*r*= 2, we say that Template:Mvar converges**in mean square**to*X*.

Convergence in the *r*-th mean, for *r* ≥ 1, implies convergence in probability (by Markov's inequality). Furthermore, if *r* > *s* ≥ 1, convergence in *r*-th mean implies convergence in *s*-th mean. Hence, convergence in mean square implies convergence in mean.

It is also worth noticing that if **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n \xrightarrow{L^r} X **
, then

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \lim_{n \to \infty} E[|X_n|^r] = E[|X|^r]**

## Properties

Provided the probability space is complete:

- If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {p}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {p}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X=Y**almost surely. - If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {a.s.}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {a.s.}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X=Y**almost surely. - If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {L^{r}}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {L^{r}}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X=Y**almost surely. - If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {p}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): Y_{n}\ {\xrightarrow {p}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): aX_{n}+bY_{n}\ {\xrightarrow {p}}\ aX+bY**(for any real numbers**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): a**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): b**) and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}Y_{n}{\xrightarrow {p}}\ XY**. - If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {a.s.}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): Y_{n}\ {\xrightarrow {a.s.}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): aX_{n}+bY_{n}\ {\xrightarrow {a.s.}}\ aX+bY**(for any real numbers**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): a**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): b**) and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}Y_{n}{\xrightarrow {a.s.}}\ XY**. - If
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {L^{r}}}\ X**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): Y_{n}\ {\xrightarrow {L^{r}}}\ Y**, then**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): aX_{n}+bY_{n}\ {\xrightarrow {L^{r}}}\ aX+bY**(for any real numbers**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): a**and**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): b**). - None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\begin{matrix}{\xrightarrow {L^{s}}}&{\underset {s>r\geq 1}{\Rightarrow }}&{\xrightarrow {L^{r}}}&&\\&&\Downarrow &&\\{\xrightarrow {a.s.}}&\Rightarrow &{\xrightarrow {\ p\ }}&\Rightarrow &{\xrightarrow {\ d\ }}\end{matrix}}**

These properties, together with a number of other special cases, are summarized in the following list:

- Template:Anchor Almost sure convergence implies convergence in probability:
^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {as}}\ X\quad \Rightarrow \quad X_{n}\ {\xrightarrow {p}}\ X**

- Template:Anchor Convergence in probability implies there exists a sub-sequence
**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): (k_{n})**which almost surely converges:^{[8]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{p}\ X \quad\Rightarrow\quad X_{k_n}\ \xrightarrow{as}\ X**

- Template:Anchor Convergence in probability implies convergence in distribution:
^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{p}\ X \quad\Rightarrow\quad X_n\ \xrightarrow{d}\ X**

- Template:Anchor Convergence in
*r*-th order mean implies convergence in probability:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{L^r}\ X \quad\Rightarrow\quad X_n\ \xrightarrow{p}\ X**

- Template:Anchor Convergence in
*r*-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{L^r}\ X \quad\Rightarrow\quad X_n\ \xrightarrow{L^s}\ X,**provided*r*≥*s*≥ 1.

- Template:Anchor If
*X*_{n}converges in distribution to a constant*c*, then*X*_{n}converges in probability to*c*:^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{d}\ c \quad\Rightarrow\quad X_n\ \xrightarrow{p}\ c,**provided*c*is a constant.

- Template:Anchor If Template:Mvar converges in distribution to
*X*and the difference between*X*and_{n}*Y*converges in probability to zero, then_{n}*Y*also converges in distribution to_{n}*X*:^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{d}\ X,\ \ |X_n-Y_n|\ \xrightarrow{p}\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow{d}\ X**

- Template:Anchor If Template:Mvar converges in distribution to
*X*and*Y*converges in distribution to a constant_{n}*c*, then the joint vector (*X*_{n},*Y*_{n}) converges in distribution to*(X, c)*:^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_n\ \xrightarrow{d}\ X,\ \ Y_n\ \xrightarrow{d}\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow{d}\ (X,c)**provided*c*is a constant.

- Note that the condition that Template:Mvar converges to a constant is important, if it were to converge to a random variable
*Y*then we wouldn’t be able to conclude that (*X*_{n},*Y*_{n}) converges to*(X, Y)*.

- Template:Anchor If
*X*converges in probability to_{n}*X*and*Y*converges in probability to_{n}*Y*, then the joint vector (*X*_{n},*Y*_{n}) converges in probability to (*X*,*Y*):^{[7]}^{[proof]}

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}\ {\xrightarrow {p}}\ X,\ \ Y_{n}\ {\xrightarrow {p}}\ Y\ \quad \Rightarrow \quad (X_{n},Y_{n})\ {\xrightarrow {p}}\ (X,Y)**

- If Template:Mvar converges in probability to
*X*, and if Template:Math for all*n*and some*b*, then Template:Mvar converges in*r*th mean to*X*for all Template:Math. In other words, if Template:Mvar converges in probability to*X*and all random variables Template:Mvar are almost surely bounded above and below, then Template:Mvar converges to*X*also in any*r*th mean.

**Almost sure representation**. Usually, convergence in distribution does not imply convergence almost surely. However for a given sequence {*X*} which converges in distribution to_{n}*X*_{0}it is always possible to find a new probability space (Ω,*F*, P) and random variables {*Y*,_{n}*n*= 0, 1, ...} defined on it such that*Y*is equal in distribution to Template:Mvar for each Template:Math, and_{n}*Y*converges to_{n}*Y*_{0}almost surely.^{[9]}

- If for all
*ε*> 0,

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \sum _{n}{\mathbb {P}}\left(|X_{n}-X|>\varepsilon \right)<\infty ,**

- then we say that Template:Mvar
*converges almost completely*, or*almost in probability*towards*X*. When Template:Mvar converges almost completely towards*X*then it also converges almost surely to*X*. In other words, if Template:Mvar converges in probability to*X*sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all Template:Math), then Template:Mvar also converges almost surely to*X*. This is a direct implication from the Borel-Cantelli lemma.

- If Template:Mvar is a sum of
*n*real independent random variables:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): S_{n}=X_{1}+\cdots +X_{n}\,**

- then Template:Mvar converges almost surely if and only if Template:Mvar converges in probability.

- The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply
*L*^{1}-convergence:

**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): \left. \begin{matrix} X_n\xrightarrow{a.s.} X \\ |X_n| < Y \\ \mathrm{E}(Y) < \infty \end{matrix}\right\} \quad\Rightarrow \quad X_n\xrightarrow{L^1} X**

- A necessary and sufficient condition for
*L*^{1}convergence is**Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): X_{n}{\xrightarrow {P}}X**and the sequence (*X*) is uniformly integrable._{n}

## See also

- Proofs of convergence of random variables
- Convergence of measures
- Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
- Asymptotic distribution
- Big O in probability notation
- Skorokhod's representation theorem
- The Tweedie convergence theorem

## Notes

## References

- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book
- Template:Cite book

- ↑ Template:Harvnb
- ↑ Template:Harvnb
- ↑ Template:Harvnb
- ↑ Template:Cite web
- ↑ Template:Harvnb
- ↑ Template:Harvnb
- ↑
^{7.0}^{7.1}^{7.2}^{7.3}^{7.4}^{7.5}Template:Harvnb - ↑ Template:Cite book
- ↑ Template:Harvnb