# Cook's distance

In statistics, Cook's distance or Cook's D is a commonly used estimate of the influence of a data point when performing least squares regression analysis.[1] In a practical ordinary least squares analysis, Cook's distance can be used in several ways: to indicate data points that are particularly worth checking for validity; to indicate regions of the design space where it would be good to be able to obtain more data points. It is named after the American statistician R. Dennis Cook, who introduced the concept in 1977.

## Definition

Cook's distance measures the effect of deleting a given observation. Data points with large residuals (outliers) and/or high leverage may distort the outcome and accuracy of a regression. Points with a large Cook's distance are considered to merit closer examination in the analysis. It is calculated as:

${\displaystyle D_{i}={\frac {\sum _{j=1}^{n}({\hat {Y}}_{j}\ -{\hat {Y}}_{j(i)})^{2}}{p\ \mathrm {MSE} }},}$

where:

${\displaystyle {\hat {Y}}_{j}\,}$ is the prediction from the full regression model for observation j;
${\displaystyle {\hat {Y}}_{j(i)}\,}$ is the prediction for observation j from a refitted regression model in which observation i has been omitted;
${\displaystyle p}$ is the number of fitted parameters in the model;
${\displaystyle {\mathrm {MSE} }\,}$ is the mean square error of the regression model.

The following are the algebraically equivalent expressions (in case of simple linear regression):

${\displaystyle D_{i}={\frac {e_{i}^{2}}{p\ {\mathrm {MSE} }}}\left[{\frac {h_{ii}}{(1-h_{ii})^{2}}}\right],}$
${\displaystyle D_{i}={\frac {({\hat {\beta }}-{\hat {\beta }}^{(-i)})^{T}(X^{T}X)({\hat {\beta }}-{\hat {\beta }}^{(-i)})}{(1+p)s^{2}}},}$

where:

${\displaystyle h_{ii}\,}$ is the leverage, i.e., the i-th diagonal element of the hat matrix ${\displaystyle {\mathbf {X} }\left({\mathbf {X} }^{T}{\mathbf {X} }\right)^{-1}{\mathbf {X} }^{T}}$;
${\displaystyle e_{i}\,}$ is the residual (i.e., the difference between the observed value and the value fitted by the proposed model).

## Detecting highly influential observations

There are different opinions regarding what cut-off values to use for spotting highly influential points. A simple operational guideline of ${\displaystyle D_{i}>1}$ has been suggested.[2] Others have indicated that ${\displaystyle D_{i}>4/n}$, where ${\displaystyle n}$ is the number of observations, might be used.[3]

A conservative approach relies on the fact that Cook's distance has the form W/p, where W is formally identical to the Wald statistic that one uses for testing that ${\displaystyle H_{0}:\beta _{i}=\beta _{0}}$ using some ${\displaystyle {\hat {\beta }}_{[-i]}}$.{{ safesubst:#invoke:Unsubst||date=__DATE__ |\$B= {{#invoke:Category handler|main}}{{#invoke:Category handler|main}}[citation needed] }} Recalling that W/p has an ${\displaystyle F_{p,n-p}}$ distribution (with p and n-p degrees of freedom), we see that Cook's distance is equivalent to the F statistic for testing this hypothesis, and we can thus use ${\displaystyle F_{p,n-p,1-\alpha }}$ as a threshold.

## Interpretation

Specifically ${\displaystyle D_{i}}$ can be interpreted as the distance one's estimates move within the confidence ellipsoid that represents a region of plausible values for the parameters.Template:Clarify This is shown by an alternative but equivalent representation of Cook's distance in terms of changes to the estimates of the regression parameters between the cases where the particular observation is either included or excluded from the regression analysis.

## References

1. {{#invoke:citation/CS1|citation |CitationClass=book }}
2. Cook, R. Dennis; and Weisberg, Sanford (1982); Residuals and influence in regression, New York, NY: Chapman & Hall
3. Bollen, Kenneth A.; and Jackman, Robert W. (1990); Regression diagnostics: An expository treatment of outliers and influential cases, in Fox, John; and Long, J. Scott (eds.); Modern Methods of Data Analysis (pp. 257-91). Newbury Park, CA: Sage
• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:Citation/CS1|citation

|CitationClass=journal }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }} Template:Refend