# Covariance matrix

A bivariate Gaussian probability density function centered at (0, 0), with covariance matrix [ 1.00, 0.50 ; 0.50, 1.00 ].
Sample points from a multivariate Gaussian distribution with a standard deviation of 3 in roughly the lower left-upper right direction and of 1 in the orthogonal direction. Because the x and y components co-vary, the variances of x and y do not fully describe the distribution. A 2×2 covariance matrix is needed; the directions of the arrows correspond to the eigenvectors of this covariance matrix and their lengths to the square roots of the eigenvalues.

In probability theory and statistics, a covariance matrix (also known as dispersion matrix or variance–covariance matrix) is a matrix whose element in the i, j position is the covariance between the i th and j th elements of a random vector (that is, of a vector of random variables). Each element of the vector is a scalar random variable, either with a finite number of observed empirical values or with a finite or infinite number of potential values specified by a theoretical joint probability distribution of all the random variables.

Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions. As an example, the variation in a collection of random points in two-dimensional space cannot be characterized fully by a single number, nor would the variances in the x and y directions contain all of the necessary information; a 2×2 matrix would be necessary to fully characterize the two-dimensional variation.

Because the covariance of the i th random variable with itself is simply that random variable's variance, each element on the principal diagonal of the covariance matrix is the variance of one of the random variables. Because the covariance of the i th random variable with the j th one is the same thing as the covariance of the j th random variable with the i th one, every covariance matrix is symmetric. In addition, every covariance matrix is positive semi-definite.

## Definition

Throughout this article, boldfaced unsubscripted X and Y are used to refer to random vectors, and unboldfaced subscripted Xi and Yi are used to refer to random scalars.

If the entries in the column vector

${\displaystyle \mathbf {X} ={\begin{bmatrix}X_{1}\\\vdots \\X_{n}\end{bmatrix}}}$

are random variables, each with finite variance, then the covariance matrix Σ is the matrix whose (ij) entry is the covariance

${\displaystyle \Sigma _{ij}=\mathrm {cov} (X_{i},X_{j})=\mathrm {E} {\begin{bmatrix}(X_{i}-\mu _{i})(X_{j}-\mu _{j})\end{bmatrix}}}$

where

${\displaystyle \mu _{i}=\mathrm {E} (X_{i})\,}$

is the expected value of the ith entry in the vector X. In other words,

${\displaystyle \Sigma ={\begin{bmatrix}\mathrm {E} [(X_{1}-\mu _{1})(X_{1}-\mu _{1})]&\mathrm {E} [(X_{1}-\mu _{1})(X_{2}-\mu _{2})]&\cdots &\mathrm {E} [(X_{1}-\mu _{1})(X_{n}-\mu _{n})]\\\\\mathrm {E} [(X_{2}-\mu _{2})(X_{1}-\mu _{1})]&\mathrm {E} [(X_{2}-\mu _{2})(X_{2}-\mu _{2})]&\cdots &\mathrm {E} [(X_{2}-\mu _{2})(X_{n}-\mu _{n})]\\\\\vdots &\vdots &\ddots &\vdots \\\\\mathrm {E} [(X_{n}-\mu _{n})(X_{1}-\mu _{1})]&\mathrm {E} [(X_{n}-\mu _{n})(X_{2}-\mu _{2})]&\cdots &\mathrm {E} [(X_{n}-\mu _{n})(X_{n}-\mu _{n})]\end{bmatrix}}.}$

## Which matrices are covariance matrices?

From the identity just above, let ${\displaystyle \mathbf {b} }$ be a ${\displaystyle (p\times 1)}$ real-valued vector, then

${\displaystyle \operatorname {var} (\mathbf {b} ^{\rm {T}}\mathbf {X} )=\mathbf {b} ^{\rm {T}}\operatorname {var} (\mathbf {X} )\mathbf {b} ,\,}$

which must always be nonnegative since it is the variance of a real-valued random variable. From the symmetry of the covariance matrix's definition it follows that only a positive-semidefinite matrix can be a covariance matrix.{{ safesubst:#invoke:Unsubst||date=__DATE__ |$B= {{#invoke:Category handler|main}}{{#invoke:Category handler|main}}[citation needed] }} Conversely, every symmetric positive semi-definite matrix is a covariance matrix. To see this, suppose M is a p×p positive-semidefinite matrix. From the finite-dimensional case of the spectral theorem, it follows that M has a nonnegative symmetric square root, that can be denoted by M1/2. Let ${\displaystyle \mathbf {X} }$ be any p×1 column vector-valued random variable whose covariance matrix is the p×p identity matrix. Then ${\displaystyle \operatorname {var} ({\mathbf {M} }^{1/2}{\mathbf {X} })={\mathbf {M} }^{1/2}(\operatorname {var} ({\mathbf {X} })){\mathbf {M} }^{1/2}={\mathbf {M} }.\,}$ ## How to find a valid correlation matrix In some applications (e.g., building data models from only partially observed data) one wants to find the "nearest" correlation matrix to a given symmetric matrix (e.g., of observed covariances). In 2002, Higham[5] formalized the notion of nearness using a weighted Frobenius norm and provided a method for computing the nearest correlation matrix. ## Complex random vectors The variance of a complex scalar-valued random variable with expected value μ is conventionally defined using complex conjugation:{{ safesubst:#invoke:Unsubst||date=__DATE__ |$B= {{#invoke:Category handler|main}}{{#invoke:Category handler|main}}[citation needed] }}

${\displaystyle \operatorname {var} (z)=\operatorname {E} \left[(z-\mu )(z-\mu )^{*}\right]}$

where the complex conjugate of a complex number ${\displaystyle z}$ is denoted ${\displaystyle z^{*}}$; thus the variance of a complex number is a real number.

If ${\displaystyle Z}$ is a column-vector of complex-valued random variables, then the conjugate transpose is formed by both transposing and conjugating. In the following expression, the product of a vector with its conjugate transpose results in a square matrix, as its expectation:

${\displaystyle \operatorname {E} \left[(Z-\mu )(Z-\mu )^{\dagger }\right],}$

where ${\displaystyle Z^{\dagger }}$ denotes the conjugate transpose, which is applicable to the scalar case since the transpose of a scalar is still a scalar. The matrix so obtained will be Hermitian positive-semidefinite,[6] with real numbers in the main diagonal and complex numbers off-diagonal.

## Estimation

{{#invoke:main|main}} If ${\displaystyle \mathbf {M} _{\mathbf {X} }}$ and ${\displaystyle \mathbf {M} _{\mathbf {Y} }}$ are centred data matrices of dimension n-by-p and n-by-q respectively, i.e. with n rows of observations of p and q columns of variables, from which the column means have been subtracted, then, if the column means were estimated from the data, sample correlation matrices ${\displaystyle \mathbf {Q} _{\mathbf {X} }}$ and ${\displaystyle \mathbf {Q} _{\mathbf {XY} }}$ can be defined to be

${\displaystyle \mathbf {Q} _{\mathbf {X} }={\frac {1}{n-1}}\mathbf {M} _{\mathbf {X} }^{T}\mathbf {M} _{\mathbf {X} },\qquad \mathbf {Q} _{\mathbf {XY} }={\frac {1}{n-1}}\mathbf {M} _{\mathbf {X} }^{T}\mathbf {M} _{\mathbf {Y} }}$

or, if the column means were known a-priori,

${\displaystyle \mathbf {Q} _{\mathbf {X} }={\frac {1}{n}}\mathbf {M} _{\mathbf {X} }^{T}\mathbf {M} _{\mathbf {X} },\qquad \mathbf {Q} _{\mathbf {XY} }={\frac {1}{n}}\mathbf {M} _{\mathbf {X} }^{T}\mathbf {M} _{\mathbf {Y} }}$

These empirical sample correlation matrices are the most straightforward and most often used estimators for the correlation matrices, but other estimators also exist, including regularised or shrinkage estimators, which may have better properties.

## As a parameter of a distribution

If a vector of n possibly correlated random variables is jointly normally distributed, or more generally elliptically distributed, then its probability density function can be expressed in terms of the covariance matrix.

## Applications

### In financial economics

The covariance matrix plays a key role in financial economics, especially in portfolio theory and its mutual fund separation theorem and in the capital asset pricing model. The matrix of covariances among various assets' returns is used to determine, under certain assumptions, the relative amounts of different assets that investors should (in a normative analysis) or are predicted to (in a positive analysis) choose to hold in a context of diversification.

## References

1. {{#invoke:citation/CS1|citation |CitationClass=book }}
2. {{#invoke:citation/CS1|citation |CitationClass=book }}
3. Template:Cite web
4. {{#invoke:citation/CS1|citation |CitationClass=book }}
5. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
6. {{#invoke:Citation/CS1|citation |CitationClass=journal }}