|
|
| Line 1: |
Line 1: |
| {{Multiple issues|
| | She is recognized by the title of Myrtle Shryock. One of the extremely best issues in the globe for me is to do aerobics and now I'm attempting to earn cash with it. For years he's been operating as a receptionist. My family life in Minnesota and my family members enjoys it.<br><br>my web site; [http://blogzaa.com/blogs/post/9199 http://blogzaa.com/] |
| {{Expert-subject|statistics|date=September 2009}}
| |
| {{morefootnotes|date=December 2010}}
| |
| }}
| |
| | |
| In [[statistics]], the '''explained sum of squares (ESS),''' alternatively known as the '''Model Sum of Squares''' or '''Sum of Squares due to Regression''' ('''"SSR"''' – not to be confused with the [[residual sum of squares]] when this writing is being used), is a quantity used in describing how well a model, often a [[regression analysis|regression model]], represents the data being modelled. In particular, the explained sum of squares measures how much variation there is in the modelled values and this is compared to the [[total sum of squares]], which measures how much variation there is in the observed data, and to the [[residual sum of squares]], which measures the variation in the modelling errors.
| |
| | |
| ==Definition==
| |
| The '''explained sum of squares (ESS)''' is the sum of the squares of the deviations of the predicted values from the mean value of a response variable, in a standard [[regression model]] — for example, {{nowrap|1=''y''<sub>''i''</sub> = ''a'' + ''b''<sub>1</sub>''x''<sub>1''i''</sub> + ''b''<sub>2</sub>''x''<sub>2''i''</sub> + ... + ''ε''<sub>''i''</sub>}}, where ''y''<sub>''i''</sub> is the ''i'' <sup>th</sup> observation of the [[response variable]], ''x''<sub>''ji''</sub> is the ''i'' <sup>th</sup> observation of the ''j'' <sup>th</sup> [[explanatory variable]], ''a'' and ''b''<sub>''i''</sub> are [[coefficient]]s, ''i'' indexes the observations from 1 to ''n'', and ''ε''<sub>''i''</sub> is the ''i'' <sup>th</sup> value of the [[error term]]. In general, the greater the ESS, the better the estimated model performs.
| |
| | |
| If <math>\hat{a}</math> and <math>\hat{b}_i</math> are the estimated [[coefficient]]s, then
| |
| | |
| :<math>\hat{y}_i=\hat{a}+\hat{b_1}x_{1i} + \hat{b_2}x_{2i} + \cdots \, </math>
| |
| | |
| is the ''i''<sup> th</sup> predicted value of the response variable. The ESS is the sum of the squares of the differences of the predicted values and the mean value of the response variable:
| |
| | |
| :<math>\text{ESS} = \sum_{i=1}^n \left(\hat{y}_i - \bar{y}\right)^2.</math>
| |
| | |
| In general: [[total sum of squares]] = '''explained sum of squares''' + [[residual sum of squares]].
| |
| | |
| ==Partitioning in simple linear regression==
| |
| The following equality, stating that the total sum of squares equals the residual sum of squares plus the explained sum of squares, is generally true in simple linear regression:
| |
| | |
| :<math>\sum_{i=1}^n \left(y_i - \bar{y}\right)^2 = \sum_{i=1}^n \left(y_i - \hat{y}_i\right)^2 + \sum_{i=1}^n \left(\hat{y}_i - \bar{y}\right)^2.</math>
| |
| | |
| ===Simple derivation===
| |
| | |
| :<math>
| |
| \begin{align}
| |
| (y_i - \bar{y}) = (y_{i}-\hat{y}_i)+(\hat{y}_i - \bar{y}).
| |
| \end{align}
| |
| </math>
| |
| | |
| Square both sides and sum over all ''i'':
| |
| | |
| :<math>
| |
| \sum_{i=1}^n (y_{i}-\bar{y})^2=\sum_{i=1}^n (y_i - \hat{y}_{i})^2+\sum_{i=1}^n (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n 2(\hat{y}_{i}-\bar{y})(y_i - \hat{y}_i).
| |
| </math>
| |
| | |
| [[Simple linear regression]] gives <math>\hat{a}=\bar{y}-\hat{b}\bar{x}</math>.<ref name=Mendenhall>{{cite book |last=Mendenhall |first=William |title=Introduction to Probability and Statistics |publisher=Brooks/Cole |year=2009 |location=Belmont, CA |page=507 |edition=13th |isbn=9780495389538 }}</ref> What follows depends on this.
| |
| | |
| :<math>
| |
| \begin{align}
| |
| \sum_{i=1}^n 2(\hat{y}_{i}-\bar{y})(y_{i}-\hat{y}_i) & = \sum_{i=1}^{n}2((\bar{y}-\hat{b}\bar{x}+\hat{b}x_{i})-\bar{y})(y_{i}-\hat{y}_{i}) \\
| |
| & = \sum_{i=1}^{n}2((\bar{y}+\hat{b}(x_{i}-\bar{x}))-\bar{y})(y_{i}-\hat{y}_{i}) \\
| |
| & = \sum_{i=1}^{n}2(\hat{b}(x_{i}-\bar{x}))(y_{i}-\hat{y}_{i}) \\
| |
| & = \sum_{i=1}^{n}2\hat{b}(x_{i}-\bar{x})(y_{i}-(\bar{y}+\hat{b}(x_{i}-\bar{x}))) \\
| |
| & = \sum_{i=1}^{n}2\hat{b}((y_{i}-\bar{y})(x_{i}-\bar{x})-\hat{b}(x_{i}-\bar{x})^2) .
| |
| \end{align}
| |
| </math>
| |
| | |
| Again [[simple linear regression]] gives<ref name=Mendenhall/>
| |
| :<math>\hat{b}=\left(\sum_{i=1}^{n}(x_{i}-\bar{x})(y_{i}-\bar{y})\right)/\left(\sum_{i=1}^{n}(x_{i}-\bar{x})^2\right), </math>
| |
| | |
| :<math>
| |
| \begin{align}
| |
| \sum_{i=1}^{n}2(\hat{y}_{i}-\bar{y})(y_{i}-\hat{y}_{i})
| |
| & = \sum_{i=1}^{n}2\hat{b}\left((y_{i}-\bar{y})(x_{i}-\bar{x})-\hat{b}(x_{i}-\bar{x})^2\right) \\
| |
| & = 2\hat{b}\left(\sum_{i=1}^{n}(y_{i}-\bar{y})(x_{i}-\bar{x})-\hat{b}\sum_{i=1}^{n}(x_{i}-\bar{x})^2\right) \\
| |
| & = 2\hat{b}\sum_{i=1}^{n}\left((y_{i}-\bar{y})(x_{i}-\bar{x})-(y_{i}-\bar{y})(x_{i}-\bar{x})\right) \\
| |
| & = 2\hat{b}\cdot 0 = 0.
| |
| \end{align}
| |
| </math>
| |
| | |
| ==Partitioning in the general OLS model==
| |
| | |
| The general regression model with ''n'' observations and ''k'' explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is
| |
| | |
| :<math> y = X \beta + e</math>
| |
| | |
| where ''y'' is an ''n'' × 1 vector of dependent variable observations, each column of the ''n'' × ''k'' matrix ''X'' is a vector of observations on one of the ''k'' explanators, <math>\beta </math> is a ''k'' × 1 vector of true coefficients, and ''e'' is an ''n''× 1 vector of the true underlying errors. The [[ordinary least squares]] estimator for <math>\beta</math> is
| |
| | |
| :<math> \hat \beta = (X^T X)^{-1}X^T y.</math>
| |
| | |
| The residual vector <math>\hat e</math> is <math>y - X \hat \beta = y - X (X^T X)^{-1}X^T y</math>, so the residual sum of squares <math>\hat e ^T \hat e</math> is, after simplification,
| |
| | |
| :<math> RSS = y^T y - y^T X(X^T X)^{-1} X^T y.</math>
| |
| | |
| Denote as <math>\bar y</math> the constant vector all of whose elements are the sample mean <math>y_m</math> of the dependent variable values in the vector ''y''. Then the total sum of squares is
| |
| | |
| :<math> TSS = (y - \bar y)^T(y - \bar y) = y^T y - 2y^T \bar y + \bar y ^T \bar y.</math>
| |
| | |
| The explained sum of squares, defined as the sum of squared deviations of the predicted values from the observed mean of ''y'', is
| |
| | |
| :<math> ESS = (\hat y - \bar y)^T(\hat y - \bar y) = \hat y^T \hat y - 2\hat y^T \bar y + \bar y ^T \bar y.</math>
| |
| | |
| Using <math> \hat y = X \hat \beta</math> in this, and simplifying to obtain <math>\hat y^T \hat y = y^TX(X^T X)^{-1}X^Ty </math>, gives the result that ''TSS'' = ''ESS'' + ''RSS'' if and only if <math>y^T \bar y = \hat y^T \bar y</math>. The left side of this is <math>y_m</math> times the sum of the elements of ''y'', and the right side is <math>y_m</math> times the sum of the elements of <math>\hat y</math>, so the condition is that the sum of the elements of ''y'' equals the sum of the elements of <math>\hat y</math>, or equivalently that the sum of the prediction errors (residuals) <math>y_i - \hat y_i</math> is zero. This can be seen to be true by noting the well-known OLS property that the ''k'' × 1 vector <math>X^T \hat e = X^T [I - X(X^T X)^{-1}X^T]y= 0</math>: since the first column of ''X'' is a vector of ones, the first element of this vector <math>X^T \hat e</math> is the sum of the residuals and is equal to zero. This proves that the condition holds for the result that ''TSS'' = ''ESS'' + ''RSS''.
| |
| | |
| In linear algebra terms, we have <math>RSS = ||y - {\hat y}||_2^2 </math>, <math> TSS = ||y - \bar y||_2^2</math>,
| |
| <math> ESS = ||{\hat y} - \bar y||_2^2 </math>.
| |
| The proof can be simplified by noting that <math> y^T {\hat y} = {\hat y}^T {\hat y} </math>. The proof is as follows:
| |
| :<math> {\hat y}^T {\hat y} =
| |
| y^T X (X^T X)^{-1} X^T X (X^T X)^{-1} X^T y = y^T X (X^T X)^{-1} X^T y = y^T {\hat y}, </math>
| |
| | |
| Thus,
| |
| :<math> TSS = ||y - \bar y||_2^2 = ||y - {\hat y} + {\hat y} - \bar y||_2^2 </math>
| |
| :<math> TSS = ||y - {\hat y}||_2^2 + ||{\hat y} - \bar y||_2^2 + 2 <y - {\hat y}, {\hat y} - {\bar y}> </math>
| |
| :<math> TSS = RSS + ESS + 2 y^T {\hat y} -2 {\hat y}^T {\hat y} - 2 y^T {\bar y} + 2 {\hat y}{\bar y} </math> | |
| :<math> TSS = RSS + ESS - 2 y^T {\bar y} + 2 {\hat y}{\bar y} </math>
| |
| which again gives the result that ''TSS'' = ''ESS'' + ''RSS'' if and only if <math>y^T \bar y = \hat y^T \bar y</math>.
| |
| | |
| ==See also==
| |
| *[[Sum of squares (statistics)]]
| |
| *[[Lack-of-fit sum of squares]]
| |
| | |
| ==Notes==
| |
| {{Reflist}}
| |
| | |
| ==References==
| |
| * S. E. Maxwell and H. D. Delaney (1990), "Designing experiments and analyzing data: A model comparison perspective". Wadsworth. pp. 289–290.
| |
| * G. A. Milliken and D. E. Johnson (1984), "Analysis of messy data", Vol. I: Designed experiments. Van Nostrand Reinhold. pp. 146–151.
| |
| * B. G. Tabachnick and L. S. Fidell (2007), "Experimental design using ANOVA". Duxbury. p. 220.
| |
| * B. G. Tabachnick and L. S. Fidell (2007), "Using multivariate statistics", 5th ed. Pearson Education. pp. 217–218.
| |
| | |
| {{DEFAULTSORT:Explained Sum Of Squares}}
| |
| [[Category:Regression analysis]]
| |
| [[Category:Least squares]]
| |
She is recognized by the title of Myrtle Shryock. One of the extremely best issues in the globe for me is to do aerobics and now I'm attempting to earn cash with it. For years he's been operating as a receptionist. My family life in Minnesota and my family members enjoys it.
my web site; http://blogzaa.com/