Main Page: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
← Older edit
No edit summary
No edit summary
 
(473 intermediate revisions by more than 100 users not shown)
Line 1: Line 1:
In [[statistics]], '''Cochran's theorem''', devised by [[William G. Cochran]],<ref name="Cochran">{{cite journal|last=Cochran|first=W. G.|authorlink=William Gemmell Cochran|title=The distribution of quadratic forms in a normal system, with applications to the analysis of covariance|journal=[[Mathematical Proceedings of the Cambridge Philosophical Society]]|date=April 1934|volume=30|issue=2|pages=178–191|doi=10.1017/S0305004100016595}}</ref> is a [[theorem]] used to justify results relating to the [[probability distribution]]s of statistics that are used in the [[analysis of variance]].<ref>{{cite book |author= Bapat, R. B.|title=Linear Algebra and Linear Models|edition=Second|publisher= Springer |year=2000|isbn=978-0-387-98871-9}}</ref>
This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users.


== Statement ==
If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]]
Suppose ''U''<sub>1</sub>, ..., ''U''<sub>''n''</sub> are [[statistical independence|independent]] standard [[normal distribution|normally distributed]] [[random variable]]s, and an identity of the form
* Only registered users will be able to execute this rendering mode.
* Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.


:<math>
Registered users will be able to choose between the following three rendering modes:  
\sum_{i=1}^n U_i^2=Q_1+\cdots + Q_k
</math>


can be written, where each ''Q''<sub>''i''</sub> is a sum of squares of linear combinations of the ''U''s.  Further suppose that
'''MathML'''
:<math forcemathmode="mathml">E=mc^2</math>


:<math>
<!--'''PNG'''  (currently default in production)
r_1+\cdots +r_k=n
:<math forcemathmode="png">E=mc^2</math>
</math>


where ''r''<sub>''i''</sub> is the [[rank (linear algebra)|rank]] of ''Q''<sub>''i''</sub>.  Cochran's theorem states that the ''Q''<sub>''i''</sub> are independent, and each ''Q''<sub>''i''</sub> has a  [[chi-squared distribution]] with ''r''<sub>''i''</sub> [[degrees of freedom (statistics)|degrees of freedom]].<ref name="Cochran"/> Here the rank of ''Q''<sub>''i''</sub> should be interpreted as meaning the rank of the matrix ''B''<sup>(''i'')</sup>, with elements ''B''<sub>''j,k''</sub><sup>(''i'')</sup>, in the representation of ''Q''<sub>''i''</sub> as a [[quadratic form]]:
'''source'''
:<math forcemathmode="source">E=mc^2</math> -->


:<math>Q_i=\sum_{j=1}^n\sum_{k=1}^n U_j B_{j,k}^{(i)} U_k .</math>
<span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples].


Less formally, it is the number of linear combinations included in the sum of squares defining ''Q''<sub>''i''</sub>, provided that these linear combinations are linearly independent.
==Demos==


===Proof===
Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]:


We first show that the matrices ''B''<sup>(''i'')</sup> can be [[Matrix_diagonalization#Simultaneous_diagonalization|simultaneously diagonalized]] and that their non-zero [[eigenvalue]]s are all equal to +1. We then use the [[Basis (linear algebra)|vector basis]] that diagonalize them to simplify their [[Characteristic function (probability theory)|characteristic function]] and show their independence and distribution.<ref>Craig A.T. (1938) On The Independence of Certain Estimates of Variances. Ann. Math. Statist. 9, pp. 48-55</ref>


Each of the matrices ''B''<sup>(''i'')</sup> has [[rank (linear algebra)|rank]] ''r''<sub>''i''</sub> and so has exactly ''r''<sub>''i''</sub> non-zero [[eigenvalue]]s. For each ''i'', the sum <math>C^{(i)} \equiv \sum_{j\ne i}B^{(j)}</math> has at most rank <math>\sum_{j\ne i}r_j = N-r_i</math>. Since <math>B^{(i)}+C^{(i)} = I_{NxN}</math>, it follows that ''C''<sup>(''i'')</sup> has exactly rank N-''r''<sub>''i''</sub>.
* accessibility:
** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]]
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)]
** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)]
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)]
** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]].
** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]].
** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode.


Therefore ''B''<sup>(''i'')</sup> and ''C''<sup>(''i'')</sup> can be [[Matrix_diagonalization#Simultaneous_diagonalization|simultaneously diagonalized]]. This can be shown by first diagonalizing ''B''<sup>(''i'')</sup>. In this basis, it is of the form:
==Test pages ==
:<math>\begin{bmatrix}
\lambda_1      & 0          & ... & ...          & ... &0 \\
0              & \lambda_2  & 0  & ...          & ... & 0 \\
0              &  ...      & ... & ...          & ... & 0\\
0              &  ...      &0    & \lambda_{r_i} & 0 &... \\
0 & ...      &    & 0            & 0...&0\\
0 & ...      &    & 0            & ...&...\\
0 & ...      &    & 0            & 0...&0
\end{bmatrix}.</math>


Thus the lower <math>(N-r_i)</math> rows are zero. Since <math>C^{(i)} = I - B^{(i)}</math>, it follows these rows in ''C''<sup>(''i'')</sup> in this basis contain a right block which is a <math>(N-r_i)\times(N-r_i)</math> unit matrix, with zeros in the rest of these rows. But since ''C''<sup>(''i'')</sup> has rank N-''r''<sub>''i''</sub>, it must be zero elsewhere. Thus it is diagonal in this basis as well. Moreover, it follows that all the non-zero [[eigenvalue]]s of both ''B''<sup>(''i'')</sup> and ''C''<sup>(''i'')</sup> are +1.
To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages:
*[[Displaystyle]]
*[[MathAxisAlignment]]
*[[Styling]]
*[[Linebreaking]]
*[[Unique Ids]]
*[[Help:Formula]]


It follows that the non-zero eigenvalues of all the ''B''-s are equal to +1. Moreover, the above analysis can be repeated in the diagonal basis for <math>C^{(1)} = B^{(2)} + \sum_{j>2}B^{(j)}</math>. In this basis <math>C^{(1)}</math> is the identity of an <math>(N-r_i)\times(N-r_i)</math> vector space, so it follows that both ''B''<sup>(''2'')</sup> and <math>\sum_{j>2}B^{(j)}</math> are simultaneously diagonalizable in this vector space (and hence also together ''B''<sup>(''1'')</sup>). By repeating this over and over it follows that all the ''B''-s are simultaneously diagonalizable.
*[[Inputtypes|Inputtypes (private Wikis only)]]
 
*[[Url2Image|Url2Image (private Wikis only)]]
Thus there exists an [[orthogonal matrix]] S such that for all i between 1 and ''k'': <math> S^\mathrm{T}Q_i S </math> is diagonal with the diagonal having 1-s at the places between <math>r_1 + ... r_{i-1} +1 </math> and <math>r_1 + ... r_i </math>.
==Bug reporting==
 
If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de .
Let <math>U_i^\prime</math> be the independent variables <math>U_i</math> after transformation by S.
 
The characteristic function of ''Q''<sub>''i''</sub> is:
:<math>\begin{align}
\varphi_i(t) =& (2\pi)^{-N/2} \int dU_1 \int dU_2 ... \int dU_N e^{i t Q_i} \cdot e^{-\frac{U_1^2}{2}}\cdot e^{-\frac{U_2^2}{2}}\cdot ...e^{-\frac{U_N^2}{2}} = (2\pi)^{-N/2} \left(\prod_{j=1}^N \int dU_j\right) e^{i t Q_i} \cdot e^{-\sum_{j=1}^N \frac{U_j^2}{2}} \\
=& (2\pi)^{-N/2} \left(\prod_{j=1}^N \int dU_j^\prime\right) e^{i t\cdot \sum_{m = r_1+...r_{i-1}+1}^{r_1+...r_i} (U_m^\prime)^2} \cdot e^{-\sum_{j=1}^N \frac{{U_j^\prime}^2}{2}}  \\
=& (1 - 2 i t)^{-r_i/2}
\end{align}</math>
 
This is the [[Fourier transform]] of the [[chi-squared distribution]] with ''r''<sub>''i''</sub> degrees of freedom. Therefore this is the distribution of ''Q''<sub>''i''</sub>.
 
Moreover, the characteristic function of the joint distribution of all the ''Q''<sub>''i''</sub>-s is:
:<math>\begin{align}
\varphi(t_1, t_2... t_k) =& (2\pi)^{-N/2} \left(\prod_{j=1}^N \int dU_j\right) e^{i \sum_{i=1}^k t_i \cdot Q_i} \cdot e^{-\sum_{j=1}^N \frac{U_j^2}{2}} \\
=& (2\pi)^{-N/2} \left(\prod_{j=1}^N \int dU_j^\prime\right) e^{i \cdot \sum_{i=1}^k t_i \sum_{k = r_1+...r_{i-1}+1}^{r_1+...r_i}  (U_k^\prime)^2} \cdot e^{-\sum_{j=1}^N \frac{{U_j^\prime}^2}{2}}  \\
=& \prod_{i=1}^k (1 - 2 i t_i)^{-r_i/2} = \prod_{i=1}^k \varphi_i(t_i)
\end{align}</math>
 
From which it follows that all the ''Q''<sub>''i''</sub>-s are statistically independent.
 
<!--
Cochran's theorem is the converse of [[Fisher's theorem]]. -->
 
== Examples ==
 
=== Sample mean and sample variance ===
If ''X''<sub>1</sub>, ..., ''X''<sub>''n''</sub> are independent normally distributed random variables with mean μ and standard deviation σ
then
 
:<math>U_i = \frac{X_i-\mu}{\sigma}</math>
 
is [[standard normal]] for each ''i''. It is possible to write
 
:<math>
\sum_{i=1}^n U_i^2=\sum_{i=1}^n\left(\frac{X_i-\overline{X}}{\sigma}\right)^2
+ n\left(\frac{\overline{X}-\mu}{\sigma}\right)^2
</math>
 
(here <math>\overline{X}</math> is the [[Arithmetic mean|sample mean]]). To see this identity, multiply throughout by <math>\sigma^2</math> and note that
 
:<math>
\sum(X_i-\mu)^2=
\sum(X_i-\overline{X}+\overline{X}-\mu)^2
</math>
 
and expand to give
 
:<math>
\sum(X_i-\mu)^2=
\sum(X_i-\overline{X})^2+\sum(\overline{X}-\mu)^2+
2\sum(X_i-\overline{X})(\overline{X}-\mu).
</math>
 
The third term is zero because it is equal to a constant times
 
:<math>\sum(\overline{X}-X_i)=0,</math>
 
and the second term has just ''n'' identical terms added together. Thus
:<math>
\sum(X_i-\mu)^2=
\sum(X_i-\overline{X})^2+n(\overline{X}-\mu)^2 ,
</math>
 
and hence
 
:<math>
\sum\left(\frac{X_i-\mu}{\sigma}\right)^2=
\sum\left(\frac{X_i-\overline{X}}{\sigma}\right)^2
+n\left(\frac{\overline{X}-\mu}{\sigma}\right)^2
=Q_1+Q_2.
</math>
 
Now the rank of ''Q''<sub>2</sub> is just 1 (it is the square of just one linear combination of the standard normal variables).  The rank of ''Q''<sub>1</sub> can be shown to be ''n'' &minus; 1, and thus the conditions for Cochran's theorem are met.
 
Cochran's theorem then states that ''Q''<sub>1</sub> and ''Q''<sub>2</sub> are independent, with chi-squared distributions with ''n'' &minus; 1 and 1 degree of freedom respectively. This shows that the sample mean and [[sample variance]] are independent.  This can also be shown by [[Basu's theorem]], and in fact this property ''characterizes'' the normal distribution – for no other distribution are the sample mean and sample variance independent.<ref>{{cite journal
|doi=10.2307/2983669
|first=R.C. |last=Geary |authorlink=Roy C. Geary
|year=1936
|title=The Distribution of the "Student's" Ratio for the Non-Normal Samples
|journal=Supplement to the Journal of the Royal Statistical Society
|volume=3 |issue=2 |pages=178–184
|jfm=63.1090.03
|jstor=2983669
}}</ref>
 
===Distributions===
 
The result for the distributions is written symbolically as
:<math>
\sum\left(X_i-\overline{X}\right)^2  \sim \sigma^2 \chi^2_{n-1}.
</math>
:<math>
n(\overline{X}-\mu)^2\sim \sigma^2 \chi^2_1,
</math>
 
Both these random variables are proportional to the true but unknown variance σ<sup>2</sup>. Thus their ratio does not depend on σ<sup>2</sup> and, because they are statistically independent. The distribution of their ratio is given by
 
:<math>
\frac{n\left(\overline{X}-\mu\right)^2}
{\frac{1}{n-1}\sum\left(X_i-\overline{X}\right)^2}\sim \frac{\chi^2_1}{\frac{1}{n-1}\chi^2_{n-1}}
  \sim F_{1,n-1}
</math>
 
where ''F''<sub>1,''n''&nbsp;&minus;&nbsp;1</sub> is the [[F-distribution]] with 1 and ''n''&nbsp;&minus;&nbsp;1 degrees of freedom (see also [[Student's t-distribution]]). The final step here is effectively the definition of a random variable having the F-distribution.
 
=== Estimation of variance ===
To estimate the variance σ<sup>2</sup>, one estimator that is sometimes used is the [[maximum likelihood]] estimator of the variance of a normal distribution
 
:<math>
\widehat{\sigma}^2=
\frac{1}{n}\sum\left(
X_i-\overline{X}\right)^2. </math>
 
Cochran's theorem shows that
 
:<math>
\frac{n\widehat{\sigma}^2}{\sigma^2}\sim\chi^2_{n-1}
</math>
 
and the properties of the chi-squared distribution show that the expected value of <math>\widehat{\sigma}^2</math> is σ<sup>2</sup>(''n'' &minus; 1)/''n''.
 
==Alternative formulation==
The following version is often seen when considering linear regression.{{Citation needed|date=July 2011}} Suppose that <math>Y\sim N_n(0,\sigma^2I_n)</math> is a standard [[Multivariate normal distribution|multivariate normal]] [[random vector]] (here <math>I_n</math> denotes the n-by-n [[identity matrix]]), and if <math>A_1,\ldots,A_k</math> are all n-by-n [[symmetric matrices]] with <math>\sum_{i=1}^kA_i=I_n</math>.  Then, on defining <math>r_i=Rank(A_i)</math>, any one of the following conditions implies the other two:
 
* <math>\sum_{i=1}^kr_i=n ,</math>
* <math>Y^TA_iY\sim\sigma^2\chi^2_{r_i}</math>  (thus the <math>A_i</math> are [[positive semidefinite]])
* <math>Y^TA_iY</math> is independent of <math>Y^TA_jY</math> for <math>i\neq j .</math>
 
== See also ==
* [[Cramér's theorem]], on decomposing normal distribution
* [[Infinite divisibility (probability)]]
 
{{refimprove|date=July 2011}}
 
==References==
<references/>
 
{{Experimental design|state=expanded}}
 
{{DEFAULTSORT:Cochran's Theorem}}
[[Category:Statistical theorems]]
[[Category:Characterization of probability distributions]]

Latest revision as of 22:52, 15 September 2019

This is a preview for the new MathML rendering mode (with SVG fallback), which is availble in production for registered users.

If you would like use the MathML rendering mode, you need a wikipedia user account that can be registered here [[1]]

  • Only registered users will be able to execute this rendering mode.
  • Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.

Registered users will be able to choose between the following three rendering modes:

MathML

E=mc2


Follow this link to change your Math rendering settings. You can also add a Custom CSS to force the MathML/SVG rendering or select different font families. See these examples.

Demos

Here are some demos:


Test pages

To test the MathML, PNG, and source rendering modes, please go to one of the following test pages:

Bug reporting

If you find any bugs, please report them at Bugzilla, or write an email to math_bugs (at) ckurs (dot) de .

Retrieved from "https://en.formulasearchengine.com/index.php?title=Main_Page&oldid=329754"