Professor Larry Laudan
Lecturer in Law and Philosophy
University of Texas at Austin
“When the ‘Not-Guilty’ Falsely Pass for Innocent” by Larry Laudan
While it is a belief deeply ingrained in the legal community (and among the public) that false negatives are much more common than false positives (a 10:1 ratio being the preferred guess), empirical studies of that question are very few and far between. While false convictions have been carefully investigated in more than two dozen studies, there are virtually no well-designed studies of the frequency of false acquittals. The disinterest in the latter question is dramatically borne out by looking at discussions among intellectuals of the two sorts of errors. (A search of Google Books identifies some 6.3k discussions of the former and only 144 treatments of the latter in the period from 1800 to now.) I’m persuaded that it is time we brought false negatives out of the shadows, not least because each such mistake carries significant potential harms, typically inflicted by falsely-acquitted recidivists who are on the streets instead of in prison.
In criminal law, false negatives occur under two circumstances: when a guilty defendant is acquitted at trial and when an arrested, guilty defendant has the charges against him dropped or dismissed by the judge or prosecutor. Almost no one tries to measure how often either type of false negative occurs. That is partly understandable, given the fact that the legal system prohibits a judicial investigation into the correctness of an acquittal at trial; the double jeopardy principle guarantees that such acquittals are fixed in stone. Thanks in no small part to the general societal indifference to false negatives, there have been virtually no efforts to design empirical studies that would yield reliable figures on false acquittals. That means that my efforts here to estimate how often they occur must depend on a plethora of indirect indicators. With a bit of ingenuity, it is possible to find data that provide strong clues as to approximately how often a truly guilty defendant is acquitted at trial and in the pre-trial process. The resulting inferences are not precise and I will try to explain why as we go along. As we look at various data sources not initially designed to measure false negatives, we will see that they nonetheless provide salient information about when and why false acquittals occur, thereby enabling us to make an approximate estimate of their frequency.
My discussion of how to estimate the frequency of false negatives will fall into two parts, reflecting the stark differences between the sources of errors in pleas and the sources of error in trials. (All the data to be cited here deal entirely with cases of crimes of violence.)
i). Estimating the frequency of false negatives at trials. Trial acquittals represent a very small subset of overall acquittals. Specifically, of the 232k defendants who were arrested in 2008 for, but not convicted of, a violent crime, only 6% (15k) of the freed defendants were products of a trial. Conventional wisdom has it that most defendants acquitted at trial are probably factually guilty. After all, so the usual argument goes, these defendants wouldn’t even be going to trial unless the prosecutor believed that he had a strong chance of persuading jurors that these defendants were guilty beyond a reasonable doubt.
While this argument does not rest on any solid data (and we will soon be looking at one that does), it enjoys a prima facie plausibility. Even if the prosecutor sometimes overestimates the strength of his case against the defendant, it seems reasonable to suppose that most defendants winning an acquittal at trial have an apparent guilt in the range from about 70% to 90%. One’s initial inclination in such circumstances is to suppose that at least half of those who are acquitted at trial actually committed the crime(s) they are charged with but the evidence allowed room for rational doubt about defendant’s guilt. Accordingly, one might assume that about half of those acquitted at trial are guilty, giving us some 7.5k false negatives, even though my strong suspicion is that the true figure is higher than that. There are two powerful reasons for thinking that this simplistic assumption understates the frequency of guilt among those acquitted at trial. They are as follows:
a). One potential source for corroborating my hunch involves looking at some interesting data from Scotland. There, the justice system uses BARD as the standard, as in the United States, and trial by jury. However, the Scottish system consists of three verdicts rather than the usual two: ‘guilty’, ‘guilt not proven’ and ‘not guilty’. The intermediate verdict gives us a point of entry for trying to pin down the rate of false acquittals. A guilt-not-proven verdict is called for when i). the jury is persuaded that the defendant is factually guilty (that is, p(guilt)≧0.5) but ii). the jury is not convinced of that guilt beyond a reasonable doubt. Both the not-guilty and the guilt-not-proven verdicts count as official acquittals but they send decidedly different messages. In a study of criminal prosecutions in 2005 and 2006 done by the Scottish government, it turned out that 71% of those defendants tried for homicide and acquitted received a ‘guilt-not-proven’ verdict. That means that about 7-in-10 acquittals in Scotland involve defendants regarded by the jurors as having probably committed the crime.
b). A different way of estimating the frequency of false acquittals at trials emerges from the monumental study by Kalven and Zeisel (The American Jury) of some 3,500+ jury trials in the US. The researchers asked judges in each of the trials that resulted in an acquittal whether, in the opinion of the judge, the case was ‘close’ (meaning the apparent guilt of the acquitted defendant verged on proof beyond a reasonable doubt) or whether it was a ‘clear’ acquittal (meaning that defendant’s apparent guilt was well below the BARD standard). According to the responses to this question (dealing with 1,191 acquittals), judges indicated that, in their opinions, only 5% of the trials resulted in ‘clear’ acquittals; by contrast, 52% of the cases were, in the view of judges, ‘clear for conviction’.
Since about one-third of trials for violent crimes result in an acquittal, the Kalven-Zeisel data would seem to entail that only about 15% of the acquittals are ‘clearly’ acquittals, while some 85% are, in the opinion of the presiding judge, close cases. If, as in our example from 2008, there are some 15k acquittals, more than 12k of them are close enough to warrant an assumption that these are probably factually guilty defendants, even if their apparent guilt fails to eliminate all reasonable doubts.
Putting the two data sets together, it is fair to say that significantly more than half of those acquitted at trial of a violent crime were nonetheless regarded by the jurors and judges as probably guilty and thereby are reasonably assumed to be false negatives. Accordingly, I shall hereafter assume that, among those 15k acquittals that emerged in trials for violent crimes in the US in 2008, some 11.2k of them were false negatives.
ii). False negatives in the dropping of charges (pre-trial acquittals). The much more intriguing question concerns the true guilt or innocence not of those 15k defendants acquitted at trial but of those 217k arrestees against whom charges were dropped or dismissed. Such decisions obviously came prior to trial, usually at the initiative of a prosecutor, sometimes at the initiative of a judge. We know that of those arrested by the police and charged with violent crimes in 2008, some 37% never make it to a trial or a plea bargain; the prosecutor or the pre-trial judge, in effect, acquits them. But how many of them so acquitted were truly innocent? Fortunately, there are two very large studies that shed substantial light on the answer to that crucial question. Both depend on the responses of thousands of prosecutors who were quizzed about the reasons why they dropped the charges that they did. One such study, analyzing FBI-initiated prosecutions nationwide, provides annual data about the reasons why federal prosecutors have dropped (and judges have dismissed) charges against those accused of a violent crime. The second study, undertaken by the Bureau of Justice Statistics, looked at the same issue in state cases, where of course most violent crime adjudications take place.
What emerge from both studies are many cases that were dropped for reasons that may indicate defendant’s innocence, or at least the relative weakness of the prosecutor’s case against the defendant. I shall call these factors innocence-indicators. Both studies show that prosecutors have multiple reasons for the dismissal or dropping of charges against persons charged with a violent crime. Still, both data sets about prosecutorial decisions indicate that the dominant motive for dropping outstanding charges is not, as you might expect, a belief that the defendant is actually innocent.
Sometimes, charges are dropped because of a defendant’s willingness to testify for the state in the separate trial of an accomplice. Occasionally, charges are dropped because the prosecutor discovers that the statute of limitations expires before the trial can be scheduled or he discovers that the defendant, when the alleged crime occurred, was a minor and should be tried in juvenile court. Prosecutors will also often drop charges if the rulings in the pre-trial evidence hearing indicate that the judge will exclude what the prosecutors deem to be highly inculpatory evidence of defendants’ guilt. When that occurs, the case against the defendant obviously becomes less compelling than it would have been if the relevant evidence were admitted. In fact, this was reported as the most frequent problem that prosecutors’ offices ran into. Commonly, prosecutors cite limitations of personnel and financial resources to cope with all the cases on their docket as another reason for dropping charges. (So much for the common idea that prosecutors have virtually unlimited resources!) Charges are also likely to be dropped if a key witness for the state vanishes or changes her testimony (as the Bureau of Justice Statistics puts it: “the reason for this reluctance [to testify] was usually fear of reprisal, followed by actual threats against the victim or witness.”), or if the defendant was awarded bail awaiting trial and vanished, thereby becoming a fugitive at large. Clearly, none of these reasons for dropping a case is, in any sense, an indicator of the defendant’s innocence.
Oftentimes, of course, charges are dropped for reasons that imply the weakness of the case against the defendant. A detailed report about the many decisions made in 2010 by federal prosecutors – in deciding whether to drop charges against some 7.3k detainees arrested by the FBI– claims that in 20.5% of dismissals, there appeared to be a ‘lack of criminal intent’; 7% of dropped charges were a result of the prosecutor’s decision that ‘no crime was committed’; and in another quarter of the dropped cases there were signs of ‘weak or insufficient evidence.’ That boils down to saying that, in federal trials for violent crimes, slightly less than half of all dismissals (48%) are motivated by factors other than a worry that defendant’s guilt might not be provable at trial. (Recall, too, that ‘insufficient evidence’ does not mean lack of substantial evidence that defendant committed the crime but rather evidence the prosecution believes is probably insufficient to establish defendant’s guilt beyond a reasonable doubt.)
This already gives us reason to suspect that about half of the cases where charges are dropped involve the abandonment of charges against defendants whom the prosecutor thought were probably factually guilty but was not at all sure that he could prove that guilt beyond a reasonable doubt. That argument becomes much more convincing when we remind ourselves of how defendants came to the prosecutors’ attention in the first place. Typically, a person becomes the object of police investigations initially as nothing more than a suspect, perhaps among several others who strike the police as possible culprits. If, after further inquiries and the analysis of more evidence, police decide to file charges (thereby ‘clearing’ the case as far as the police are concerned), they are required to have grounds to believe that it is more likely than not that defendant committed the crime. To make the arrest official, the police must persuade either a judge or a grand jury (or both) that a rational person, confronted with the available evidence, would conclude that defendant probably committed the crime.
Accordingly, by the time the prosecutor typically gets deeply into the act, he is dealing with a host of arrestees, each of whom is considered by the police, a grand jury and the arraigning judge to be more likely than not to be guilty on the available evidence. As the prosecutor begins assembling his case, some new evidence will often come in or be actively sought. Sometimes, that evidence will be exculpatory, and persuade the prosecutor that defendant really did not commit the crime. Much more often, though, the decision point for the prosecutor arrives when, after having reviewed the evidence, he must decide whether the case against the defendant is strong enough to persuade a trial jury that the defendant is guilty beyond a reasonable doubt. Supposing, with many scholars, that this standard represents roughly a 90+% likelihood of guilt, this means that most of those now charged with a crime have an apparent guilt that falls in the very broad range from 50+% to something close to 100%. The prosecutor will generally cull those defendants in the range of 50-80% apparent guilt out of the class of those he intends to take to trial or to negotiate a plea bargain with.
Why would he do that? When apparent guilt is in that range, the prosecutor knows that it is unlikely that he will be able to persuade the defendant to accept a plea bargain and he also knows that, if he takes the defendant to trial, it will probably result in an acquittal. There are moral reasons as well that lead to the dropping of charges, even against those whom the prosecutor believes to be factually guilty.
The second pertinent study on this vexing issue of the frequency of guilt among those dropped out of the system prior to trial was published in 1992. Unlike the FBI study, this one investigated state (rather than federal) criminal trials. It included some 40k cases. The researchers asked prosecutors why they had dropped charges in the cases (or why judges had dismissed charges) when they did. Three of the reasons given appear to be innocence-indicators: ‘evidence issues’, ‘witness problems’ and ‘the interests of justice’. Some 35% of the dropped/dismissed cases were attributed to these reasons. That left 65% of the abandoned cases involving reasons implying nothing about guilt or innocence. An earlier study of 17,500 arrests in Washington, D.C. federal courts indicates that the prosecutor dropped 3.6k cases but only a third of those dismissals (34%) were attributed to ‘insufficiency of evidence’.
Taking the mean between the FBI probably-guilty rate of 47% and the BJS value of 65%, we arrive at the estimate that about 56% of the dismissed and dropped arrestees were probably factually guilty. Even so, that figure doesn’t take us fully where we want to go. We’re after a reasonable estimate of the number of truly guilty who have the charges against them either dropped or dismissed. The fact that the 56% of arrestees against whom charges were dropped are probably guilty does not yet give us a definite way of determining how many of them were actually guilty.
There is, however, a way of generating the result we seek. Remember that the defendants in this group were dropped or dismissed because of reasons that had nothing to do with signs of their innocence. Hence, we can reasonably suppose that the proportion of guilty among them would be about the same as the proportion of guilty among those who go to trial. (After all, there is no perceived evidential weakness in the case against them that distinguished them from those who do go to trial.) Exactly two-thirds of those who went to trial for a violent crime were convicted. We have already explained why we assume that that 75% of those acquitted at trial are probably truly guilty.
That seems to provide a plausible rationale for saying that, among those defendants who had the charges against them dropped for non-evidentiary reasons, approximately two-out-of-three (and probably more) are highly likely to be guilty. Hence, we shall assume that about 37% (that is two-thirds of the 56% of those whom were booted out of the trial system for non-evidentiary reasons) are factually guilty (and, if they had gone to trial, would have been convicted). This amounts to 81k false negatives. When added to the estimate of 12k probably guilty defendants among those acquitted at trial, this figure entails that, at a minimum, some 93k of the 595k arrestees are acquitted although truly guilty. This suggests a false negative rate of ~40% (viz., 93k guilty out of 232k acquitted). The false positive rate in this example is 3% (some 11k falsely convicted defendants out of 360k convicted).
It remains to be seen whether this pattern of error distribution serves the interests of society. That is the subject of my next book. For now, I will simply note that recidivism data show unambiguously that the 88k false negatives do vastly more harm to innocent citizens than the 11k false positives do. Quite clearly, the current standard of proof needs drastic re-adjustment.
 For a lengthy discussion of the Scottish verdict system, see my “Need Verdicts Come in Pairs?” International Journal of Evidence and Proof, vol. 14 (2010), 1-24.
 Scottish Government Statistical Bulletin, Crim/2006/Part 11. The data come from the years 2004-2005. (http://www.scotland.gov.uk/Publications/2006/04/25104019/11.)
 Kalven & Zeisel, op. cit., Table 32.
 Given the Scottish estimate of ~70% false negatives at trial and the Kalven-Zeisel estimate of an 85% false negative rate in trials, I shall assume a false negative rate of ~75% in acquittals at trial.
 See especially US Dept. of Justice, United States Attorneys’ Annual Statistical Report, 2010.
 Ibid., Table 6.
 BJS, Prosecutors in State Courts, 1994 (1996), p.5.
 5% of those on bail awaiting trial on a murder charge become fugitives. BJS, Felony Defendants in Large Urban Counties, 2009 –Statistical Tables, Table 18.
 The detailed breakdown of the relevant data can be found in Table 14 of US Dept. of Justice, United States Attorneys’ Annual Statistical Report, 2010. In that year, the FBI declined to prosecute some 7,252 cases of arrested defendants (794 of these cases were violent crimes) (ibid., Table 3).
 The ethics manual of the American Bar Association, the ABA Standards for Criminal Justice: Prosecution and Defense Function, insists that prosecutors “should not institute, or cause to be instituted, or permit the continued pendency of criminal charges when the prosecutor knows that the charges are not supported by probable cause.” (Standard 3-3.9) It goes on to say that the prosecutor should drop charges against the defendant if there is “reasonable doubt that the accused is in fact guilty.” (ibid.)
 Barbara Boland et al., The Prosecution of Felony Arrests, 1988. Bureau of Justice Statistics, 1992.
 Here were the data for some of the cities in their study: Denver (46% dropped because of innocence issue); Los Angeles (50%); Manhattan (43%); St. Louis (20%); San Diego (27%); Seattle (25%); and Washington, D.C. (37%). Id., Table 5.
 Brian Forst et al., What Happens after Arrest? Institute for Law and Social Research (1977), Exhibit 5.1.
Earlier guest post by Laudan:
Larry Laudan. Why Presuming Innocence is Not a Bayesian Prior