Bunch–Nielsen–Sorensen formula

In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik.)

Proof

Proof using the trace formula

It is easy to show that for matrices $M$ and $N$, the Hadamard product $M\circ N$ considered as a bilinear form acts on vectors $a,b$ as

$a^T(M\circ N)b=\mathrm{Tr}(M\mathrm{diag}(a)N\mathrm{diag}(b))$

where $\mathrm{Tr}$ is the matrix trace and $\mathrm{diag}(a)$ is the diagonal matrix having as diagonal entries the elements of $a$.

Since $M$ and $N$ are positive definite, we can consider their square-roots $M^{1/2}$ and $N^{1/2}$ and write

$\mathrm{Tr}(M\mathrm{diag}(a)N\mathrm{diag}(b))=\mathrm{Tr}(M^{1/2}{M}^{1/2}\mathrm{diag}(a)N^{1/2}{N}^{1/2}\mathrm{diag}(b))=\mathrm{Tr}(M^{1/2}\mathrm{diag}(a)N^{1/2}{N}^{1/2}\mathrm{diag}(b)M^{1/2})$

Then, for $a=b$, this is written as $\mathrm{Tr}(A^{T}A)$ for $A=N^{1/2}\mathrm{diag}(a)M^{1/2}$ and thus is positive. This shows that $(M\circ N)$ is a positive definite matrix.

Proof using Gaussian integration

Case of M = N

Let $X$ be an $n$-dimensional centered Gaussian random variable with covariance $⟨X_i{X}_j⟩=M_{ij}$. Then the covariance matrix of $X_i^2$ and $X_j^2$ is

$\mathrm{Cov}(X_i^2,X_j^2)=⟨X_i^2{X}_j^2⟩-⟨X_i^2⟩⟨X_j^2⟩$

Using Wick's theorem to develop $⟨X_i^2{X}_j^2⟩=2⟨X_i{X}_j{⟩}^2+⟨X_i^2⟩⟨X_j^2⟩$ we have

$\mathrm{Cov}(X_i^2,X_j^2)=2⟨X_i{X}_j{⟩}^2=2M_{ij}^2$

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{ij}^2$ is a positive definite matrix.

General case

Let $X$ and $Y$ be $n$-dimensional centered Gaussian random variables with covariances $⟨X_i{X}_j⟩=M_{ij}$, $⟨Y_i{Y}_j⟩=N_{ij}$ and independt from each other so that we have

$⟨X_i{Y}_j⟩=0$ for any $i,j$

Then the covariance matrix of $X_i{Y}_i$ and $X_j{Y}_j$ is

$\mathrm{Cov}(X_i{Y}_i,X_j{Y}_j)=⟨X_i{Y}_i{X}_j{Y}_j⟩-⟨X_i{Y}_i⟩⟨X_j{Y}_j⟩$

Using Wick's theorem to develop

$⟨X_i{Y}_i{X}_j{Y}_j⟩=⟨X_i{X}_j⟩⟨Y_i{Y}_j⟩+⟨X_i{Y}_i⟩⟨X_i{Y}_j⟩+⟨X_i{Y}_j⟩⟨X_j{Y}_i⟩$

and also using the independence of $X$ and $Y$, we have

$\mathrm{Cov}(X_i{Y}_i,X_j{Y}_j)=⟨X_i{X}_j⟩⟨Y_i{Y}_j⟩=M_{ij}{N}_{ij}$

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{ij}{N}_{ij}$ is a positive definite matrix.

Proof using eigendecomposition

Proof of positivity

Let $M=\sum {\mu }_i{m}_i{m}_i^T$ and $N=\sum {\nu }_i{n}_i{n}_i^T$. Then

$M\circ N=\sum _{ij}{\mu }_i{\nu }_j(m_i{m}_i^T)\circ (n_j{n}_j^T)=\sum _{ij}{\mu }_i{\nu }_j(m_i\circ n_j)(m_i\circ n_j{)}^T$

Each $(m_i\circ n_j)(m_i\circ n_j{)}^T$ is positive (but, except in the 1-dimensional case, not positive definite, since they are rank 1 matrices) and ${\mu }_i{\nu }_j>0$, thus the sum giving $M\circ N$ is also positive.

Complete proof

To show that the result is positive definite requires further proof. We shall show that for any vector $a\ne 0$, we have $a^T(M\circ N)a>0$. Continuing as above, each $a^T(m_i\circ n_j)(m_i\circ n_j{)}^{T}a\ge 0$, so it remains to show that there exist $i$ and $j$ for which the inequality is strict. For this we observe that

$a^T(m_i\circ n_j)(m_i\circ n_j{)}^{T}a={\left(\sum _k{m}_{i,k}{n}_{j,k}{a}_k\right)}^2$

Since $N$ is positive definite, there is a $j$ for which $n_{j,k}{a}_k$ is not 0 for all $k$, and then, since $M$ is positive definite, there is an $i$ for which $m_{i,k}{n}_{j,k}{a}_k$ is not 0 for all $k$. Then for this $i$and $j$ we have ${\left(\sum _k{m}_{i,k}{n}_{j,k}{a}_k\right)}^2>0$. This completes the proof.

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

External links

• Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen at EUDML