|
|
| Line 1: |
Line 1: |
| The '''prosecutor's fallacy''' is a [[fallacy]] of statistical reasoning, typically used by the prosecution to argue for the guilt of a defendant during a criminal trial. Although it is named after prosecutors it is not specific to them, and some variants of the fallacy can be utilized by defense lawyers arguing for the innocence of their client. At its heart the fallacy involves assuming that the prior probability of a random match is [[Equality (mathematics)|equal to]] the probability that the defendant is innocent. For instance, if a perpetrator is known to have the same blood type as a defendant and 10% of the population share that blood type; then to argue on that basis alone that the probability of the defendant being guilty is 90% makes the prosecutors's fallacy, in a very simple form.
| | Not much to tell about me really.<br>Yes! Im a member of this community.<br>I really wish I am useful at all<br><br>my webpage; [http://www.education2020.ca/Edupedia/All_You_Need_To_Know_To_Identify_A_Job www.education2020.ca] |
| | |
| ==Concept==
| |
| The terms "prosecutor's fallacy" and [[#Defense_attorney's_fallacy|"defense attorney's fallacy"]] were originated by William C. Thompson and Edward Schumann in 1987.<ref name="TS">{{cite journal |last1= Thompson |first= E.L. |last2= Shumann |year= 1987 |title= Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor's Fallacy and the Defense Attorney's Fallacy |journal= Law and Human Behavior |volume=2 |issue= 3 |page= 167 |jstor=1393631 |first2= E. L. |doi= 10.1007/BF01044641 }}</ref><ref>{{cite web |url=http://uctv.canterbury.ac.nz/viewfile.php/4/sharing/55/74/74/NZEPVersionofImpreciseProbabilitiespaperVersi.pdf |title= Ambiguity, the Certainty Illusion, and Gigerenzer's Natural Frequency Approach to Reasoning with Inverse Probabilities |first1= John |last1= Fountain |first2= Philip |last2= Gunby |date=February 2010 |publisher= [[University of Canterbury]] |page= 6}}</ref> The fallacy can arise from ''multiple testing'', such as when evidence is compared against a large database. The size of the database elevates the likelihood of finding a match by pure chance alone; i.e., DNA evidence is soundest when a match is found after a single directed comparison because the existence of matches against a large database where the test sample is of poor quality may be less unlikely by mere chance.
| |
| | |
| The basic fallacy results from misunderstanding [[conditional probability]] and neglecting the prior odds of a defendant being guilty before that evidence was introduced. When a [[prosecutor]] has collected some [[evidence (law)|evidence]] (for instance a [[DNA]] match) and has an expert testify that the [[probability]] of finding this evidence if the accused were innocent is tiny, the fallacy occurs if it is concluded that the probability of the accused being innocent must be comparably tiny. If the DNA match is used to confirm guilt which is otherwise suspected then it is indeed strong evidence. However if the DNA evidence is the sole evidence against the accused and the accused was picked out of a large database of DNA profiles, the odds of the match being made at random may be reduced, and less damaging to the defendant. The odds in this scenario do not relate to the odds of being guilty, they relate to the odds of being picked at random.
| |
| | |
| == Examples of prosecutor's fallacies ==
| |
| | |
| ===Conditional probability===
| |
| | |
| Argument from rarity – Consider this case: a lottery winner is accused of [[cheating]], based on the improbability of winning. At the trial, the prosecutor calculates the (very small) probability of winning the lottery without cheating and argues that this is the chance of innocence. The logical flaw is that the prosecutor has failed to account for the large number of people who play the lottery.
| |
| | |
| [[Berkson's paradox]] – mistaking [[conditional probability]] for unconditional – led to several [[Type I error|wrongful]] convictions of British mothers, accused of murdering two of their children in infancy, where the primary evidence against them was the statistical improbability of two children dying accidentally in the same household (under "[[Meadow's law]]"). Though multiple accidental ([[Sudden infant death syndrome|SIDS]]) deaths are rare, so are multiple murders; with only the facts of the deaths as evidence, it is the ratio of these (prior) improbabilities that gives the correct "[[posterior probability]]" of murder.<ref>{{cite web |url= http://www.guardian.co.uk/science/2006/oct/28/uknews1 |title= Prosecuting and defending by numbers |first= Ben |last= Goldacre |date= 2006-10-28 |work= [[The Guardian]] |accessdate= 2010-05-22 |quote= rarity is irrelevant, because double murder is rare too. An entire court process failed to spot the nuance of how the figure should be used. Twice. }}</ref>
| |
| | |
| ===Multiple testing===
| |
| | |
| In another scenario, a crime-scene [[DNA profiling|DNA sample]] is compared against a [[database]] of 20,000 men. A match is found, that man is accused and at his trial, it is testified that the probability that two DNA profiles match by chance is only 1 in 10,000. This does not mean the probability that the suspect is innocent is 1 in 10,000. Since 20,000 men were tested, there were 20,000 opportunities to find a match by chance.
| |
| | |
| Even if none of the men in the database left the crime-scene DNA, a match by chance to an innocent is more likely than not. The chance of getting at least one match among the records is:
| |
| | |
| : <math>1 - \left(1-\frac{1}{10000}\right)^{20000} \approx 86\%</math>
| |
| | |
| So, this evidence alone is an uncompelling [[data dredging]] result. If the culprit ''was'' in the database then he and one or more other men would probably be matched; in either case, it would be a fallacy to ignore the number of records searched when weighing the evidence. [[DNA_profiling#DNA_databases|"Cold hits"]] like this on DNA databanks are now understood to require careful presentation as trial evidence.
| |
| | |
| == Mathematical analysis ==
| |
| | |
| Finding a person innocent or guilty can be viewed in mathematical terms as a form of [[binary classification]].
| |
| If ''E'' is the observed evidence, and ''I'' stands for "accused is innocent" then consider the [[conditional probability|conditional probabilities]]:
| |
| * ''P''(''E''|''I'') is the probability that the "damning evidence" would be observed even when the accused is innocent (a [[Type I and type II errors#Usage_examples|"false positive"]]).
| |
| * ''P''(''I''|''E'') is the probability that the accused is innocent, despite the evidence ''E''.
| |
| | |
| With [[forensic evidence]], ''P''(''E''|''I'') is tiny. The prosecutor wrongly concludes that ''P''(''I''|''E'') is comparatively tiny. (The [[Lucia de Berk]] prosecution is accused of exactly this error,<ref>
| |
| {{cite journal |last1= Meester |first1= R. |last2= Collins |first2= M. |last3= Gill |first3= R. |last4= van Lambalgen |first4= M. |date= 2007-05-05 |title= On the (ab)use of statistics in the legal case against the nurse Lucia de B |journal= Law, Probability & Risk |doi= 10.1093/lpr/mgm003 |arxiv= math/0607340 |quote= [page 11] Writing E for the observed event, and H for the [[null hypothesis|hypothesis of chance]], Elffers calculated P('''E''' | H) < 0.0342%, while the court seems to have concluded that P('''H''' | E) < 0.0342% |volume= 5 |issue= 3–4 |pages= 233–250 }}
| |
| </ref> for example.) In [[Bayesian inference|fact]], ''P''(''E''|''I'') and ''P''(''I''|''E'') are quite different; using [[Bayes' theorem]]:
| |
| : <math> P(I|E) = P(E|I) \cdot \frac{P(I) }{P(E)} </math>
| |
| Where:
| |
| * ''P''(''I'') is the probability of innocence ''independent of the test result'' (i.e. from all other [[Admissible evidence|evidence]]) and
| |
| * ''P''(''E'') is the prior probability that the evidence would be observed (regardless of innocence):
| |
| | |
| : <math> P(E) = P(E|I) \cdot P(I) + P(E|\sim I) \cdot [1 - P(I)] </math>
| |
| * ''P''(''E''|~''I'') is the probability that the evidence would identify a [[wiktionary:guilty|guilty]] suspect (not give a [[Type I and type II errors#False_negative_rate|false negative]]). This is usually close to 100%, slightly increasing the inference of innocence over a test without false negatives. That inequality is concisely expressed in terms of [[odds]]:
| |
| | |
| : <math> \operatorname{Odds}(I|E) \ge \operatorname{Odds}(I)\cdot P(E|I) </math>
| |
| The prosecutor is claiming a negligible chance of innocence, given the evidence, implying ''Odds''(''I''|''E'') -> ''P''(''I''|''E''), or that:
| |
| | |
| : <math> P(I|E) \approx P(E|I)\cdot \operatorname{Odds}(I) </math>
| |
| | |
| A prosecutor conflating ''P''(''I''|''E'') with ''P''(''E''|''I'') makes a technical error whenever ''Odds''(''I'') >> 1. This may be a [[harmless error]] if ''P''(''I''|''E'') is still negligible, but it is especially [[mislead]]ing otherwise (mistaking low [[statistical significance]] for high [[Confidence_(statistics)#Meaning_of_the_term_.22confidence.22|confidence]]).
| |
| | |
| == Legal impact ==
| |
| | |
| In the courtroom, the prosecutor's fallacy typically happens by mistake,<ref>
| |
| {{cite journal |url= http://policechiefmagazine.org/magazine/index.cfm?fuseaction=display_arch&article_id=1922&issue_id=102009 |title= Failures in Criminal Investigation: Errors of Thinking |last1= Rossmo |first1= D.K. |date=October 2009 |publisher= |accessdate=2010-05-21 |journal= The Police Chief |volume= LXXVI |issue= 10 | quote= The prosecutor's fallacy is more insidious because it typically happens by mistake. }}
| |
| </ref> but deliberate use of the prosecutor's fallacy is [[prosecutorial misconduct]] and can subject the prosecutor to official reprimand, disbarment or criminal punishment.
| |
| | |
| In the [[adversarial system]], [[lawyer]]s are usually free to present statistical evidence as best suits their case; retrials are more commonly the result of the prosecutor's fallacy in [[expert witness]] testimony or in the [[Jury instructions|judge's summation]].<ref>{{cite web |url= http://www.denverda.org/DNA_Documents/DNA%20in%20Australia.pdf |title= DNA Identification in the Criminal Justice System |date= 2002-05-01 |publisher= Australian Institute of Criminology |accessdate=2010-05-21}}</ref>
| |
| | |
| ==Defense attorney's fallacy==
| |
| {{Expert-subject|Philosophy/Logic|section|date=September 2009}}
| |
| Suppose there is a one-in-a-million chance of a match given that the accused is innocent. The prosecutor says this means there is only a one-in-a-million chance of innocence. But if everyone in a community of 10 million people is tested, one expects 10 matches even if all are innocent. The defense fallacy would be to reason that "10 matches were expected, so the accused is no more likely to be guilty than any of the other matches, thus the evidence suggests a 90% chance that the accused is innocent." and "As such, this evidence is irrelevant." The first part of the reasoning would be correct only in the case where there is no further evidence pointing to the defendant. On the second part, Thompson & Schumann wrote that the evidence should still be highly [[relevant]] because it "drastically narrows the group of people who are or could have been suspects, while failing to exclude the defendant" (page 171).<ref name="TS" /><ref>{{cite journal |ssrn=1539107 |title= Interpretative Arguments of Forensic Match Evidence: An Evidentiary Analysis |author= N. Scurich |year=2010 |journal=The Darmouth Law Journal |volume=8 |issue=2 |pages=31–47 | quote = The idea is that each piece of evidence need not conclusively establish a proposition, but that all the evidence can be used as a [[mosaic]] to establish the proposition}}</ref>
| |
| | |
| ===Possible examples of fallacious defense arguments===
| |
| Some authors have cited defense arguments in the [[O. J. Simpson murder case|O. J. Simpson murder trial]] as an example of this fallacy regarding the context in which the accused had been brought to court: crime scene blood matched [[O. J. Simpson|Simpson]]'s with characteristics shared by 1 in 400 people. The defense argued that a football stadium could be filled with [[Los Angeles|Angelenos]] matching the sample and that the figure of 1 in 400 was useless.<ref>Robertson, B., & Vignaux, G. A. (1995). '' Interpreting evidence: Evaluating forensic evidence in the courtroom. '' Chichester: John Wiley and Sons.</ref><ref>Rossmo, D. Kim (2009). '' Criminal Investigative Failures. '' CRC Press Taylor & Francis Group.</ref>
| |
| | |
| Also at the [[O. J. Simpson murder case|O. J. Simpson murder trial]], the prosecution presented evidence that Simpson had been violent toward his wife, while the defense argued that there was only one woman murdered for every 2500 women who were subjected to spousal abuse, and that any history of Simpson being violent toward his wife was irrelevant to the trial. However, some regard the reasoning behind the defense's calculation as fallacious. According to author [[Gerd Gigerenzer]], the correct probability requires the context—that Simpson's wife had not only been subjected to domestic violence, but subjected to domestic violence ''and'' murdered—to be taken into account. Gigerenzer writes "the chances that a batterer actually murdered his partner, given that she has been killed, is about 8 in 9 or approximately 90%".<ref>Gigerenzer, G., Reckoning with Risk: Learning to Live with Uncertainty, Penguin, (2003)</ref>
| |
| | |
| == The Sally Clark case ==
| |
| | |
| [[Sally Clark]], a British woman who was accused in 1998 of having killed her first child at 11 weeks of age, then conceived another child and allegedly killed it at 8 weeks of age. The prosecution had [[expert witness]] [[Roy Meadow|Sir Roy Meadow]] testify that the probability of two children in the same family dying from [[Sudden infant death syndrome|SIDS]] is about 1 in 73 million. That was much less frequent than the actual rate measured in [[Epidemiology|historical data]] – Meadow estimated it from single-SIDS death data, and the assumption that the probability of such deaths should be [[uncorrelated]] between infants.<ref>The population-wide probability of a SIDS fatality was about 1 in 1,303; Meadow generated his 1-in-73 million estimate from the lesser probability of SIDS death in the Clark household, which had lower risk factors (e.g. non-smoking). In this sub-population he estimated the probability of a single death at 1 in 8,500. See: {{cite web
| |
| |url= http://plus.maths.org/issue21/features/clark/
| |
| |title= Beyond reasonable doubt |format= pdf
| |
| |publisher=plus.maths.org
| |
| |accessdate=2010-06-12
| |
| |last= Joyce |date=September 2002
| |
| |first= H.
| |
| }}. Professor Ray Hill questioned even this first step (1/8,500 vs 1/1,300) in two ways: firstly, on the grounds that it was [[bias]]ed, excluding those factors that increased risk (especially that both children were boys) and (more importantly) because reductions in SIDS risk factors will proportionately reduce murder risk factors, so that the relative frequencies of [[Münchausen syndrome by proxy]] and SIDS will remain in the same ratio as in the general population: {{cite web |url= http://www.mrbartonmaths.com/resources/a%20level/s1/Beyond%20reasonable%20doubt.doc
| |
| |first= Ray |last= Hill |quote= it is patently unfair to use the characteristics which basically make her a good, clean-living, mother as factors which count against her. Yes, we can agree that such factors make a natural death less likely – but those same characteristics also make murder less likely. |year= 2002 |title= Cot Death or Murder? – Weighing the Probabilities }}
| |
| </ref>
| |
|
| |
| Meadow acknowledged that 1-in-73 million is not an impossibility, but argued that such accidents would happen "once every hundred years" and that, in a country of 15 million 2-child families, it is vastly more likely that the double-deaths are due to [[Münchausen syndrome by proxy]] than to such a rare accident. However, there is good reason to suppose that the likelihood of a death from SIDS in a family is significantly greater if a previous child has already died in these circumstances (a [[genetic predisposition]] to SIDS is likely to invalidate that assumed [[statistical independence]]<ref>[http://observer.guardian.co.uk/print/0,,4221973-102285,00.html Gene find casts doubt on double 'cot death' murders]. The Observer; July 15, 2001</ref>) making some families more susceptible to SIDS and the error an outcome of the [[ecological fallacy]].<ref>{{cite web |url= http://understandinguncertainty.org/node/545#notes |title= Convicted on Statistics? |author= Vincent Scheurer |accessdate=2010-05-21}}</ref> The likelihood of two SIDS deaths in the same family cannot be soundly [[Estimation theory|estimated]] by squaring the likelihood of a single such death in all otherwise similar families.<ref>{{cite journal |title= Multiple sudden infant deaths – coincidence or beyond coincidence? |url= http://www.cse.salford.ac.uk/staff/RHill/ppe_5601.pdf |first= R. |last= Hill |journal= Paediatric and Perinatal Epidemiology |year= 2004 |volume= 18 |page= 321 |doi= 10.1111/j.1365-3016.2004.00560.x |issue= 5}}</ref>
| |
|
| |
| 1-in-73 million greatly underestimated the chance of two successive accidents, but, even if that assessment were accurate, the court seems to have missed the fact that the 1-in-73 million number meant nothing on its own. As an ''[[a priori and a posteriori|a priori]]'' probability, it should have been weighed against the ''a priori'' probabilities of the alternatives. [[A priori and a posteriori|Given that]] two deaths had occurred, one of the following explanations must be true, and all of them are ''a priori'' extremely improbable:
| |
| # Two successive deaths in the same family, both by SIDS
| |
| # Double homicide (the prosecution's case)
| |
| # Other possibilities (including one homicide and one case of SIDS)
| |
| | |
| It's unclear that an estimate for the second possibility was ever proposed during the trial, or that the comparison of the first two probabilities was understood to be the key estimate to make in the statistical analysis assessing the prosecution's case against the case for innocence.
| |
| | |
| Mrs. Clark was convicted in 1999, resulting in a press release by the [[Royal Statistical Society]] which pointed out the mistakes.<ref>{{cite web |url= http://www.rss.org.uk/uploadedfiles/documentlibrary/744.pdf |title = Royal Statistical Society concerned by issues raised in Sally Clark case |date= 23 October 2001 |quote= Society does not tolerate doctors making serious clinical errors because it is widely understood that such errors could mean the difference between life and death. The case of R v. Sally Clark is one example of a medical expert witness making a serious statistical error, one which may have had a profound effect on the outcome of the case }}</ref>
| |
| | |
| In 2002, Ray Hill (Mathematics professor at [[University of Salford|Salford]]) attempted to accurately compare the chances of these two possible explanations; he concluded that successive accidents are between 4.5 and 9 times more likely than are successive murders, so that the ''a priori'' [[odds]] of Clark's guilt were between 4.5 to 1 and 9 to 1 against.<ref>The uncertainty in this range is mainly driven by uncertainty in the likelihood of [[Recidivism|killing a second child]], having killed a first, see: {{cite journal |title= Multiple sudden infant deaths – coincidence or beyond coincidence? |url= http://www.cse.salford.ac.uk/staff/RHill/ppe_5601.pdf |first= R. |last= Hill |journal= Paediatric and Perinatal Epidemiology |year= 2004 |volume= 18 |pages= 322–323 |doi= 10.1111/j.1365-3016.2004.00560.x |issue= 5}}</ref>
| |
| | |
| A higher court later quashed Sally Clark's conviction, on other grounds, on 29 January 2003.
| |
| | |
| Sally Clark, a practising solicitor before the conviction, developed a number of serious psychiatric problems including serious alcohol dependency and died in 2007 from alcohol poisoning.<ref>{{cite news |url=http://www.guardian.co.uk/society/2007/mar/17/childrensservices.uknews |title=Sally Clark, mother wrongly convicted of killing her sons, found dead at home |publisher=The Guardian |date=March 17, 2007 |accessdate=2008-09-25 | location=London | first=Thair | last=Shaikh}}</ref>
| |
| | |
| == See also ==
| |
| * [[Base rate fallacy]]
| |
| * [[Representativeness heuristic]]
| |
| * [[False positive]]
| |
| * [[False positive paradox]]
| |
| * [[Likelihood function]]
| |
| * [[Howland will forgery trial]]
| |
| * [[People v. Collins]]
| |
| * [[Conditional probability#The conditional probability fallacy|Conditional probability fallacy]]
| |
| * [[Lucia de Berk]]
| |
| * [[Simpson's paradox]]
| |
| * [[Data dredging]]
| |
| | |
| ==References==
| |
| {{More footnotes|date=November 2008}}
| |
| {{Reflist|30em}}
| |
| | |
| == External links ==
| |
| * [http://www.colchsfc.ac.uk/maths/dna/discuss.htm Discussion of the prosecutor's fallacy]{{Dead link|date=April 2013}}
| |
| * [http://dna-view.com/profile.htm Forensic mathematics of DNA matching]
| |
| * [http://www.ted.com/index.php/talks/peter_donnelly_shows_how_stats_fool_juries.html A British statistician on the fallacy]
| |
| | |
| {{DEFAULTSORT:Prosecutor's Fallacy}}
| |
| [[Category:Bayesian statistics]]
| |
| [[Category:Logical fallacies]]
| |
| [[Category:Forensic statistics]]
| |
| [[Category:Misuse of statistics]]
| |