The Wrongful Conviction of Forensic Science
by Moderator on Sep.04, 2009, under Articles for discussion
Authors: John M. Collins and Jay Jarvis
The purpose of this study was to examine the accuracy of claims that faulty forensic science is a leading cause of wrongful convictions. This sentiment has been reported at length by major news outlets across the United States. It has also been a matter of great concern to a group of activists in what is known as the innocence network and other individuals having varying degrees of interest in the formulation of public policies related to forensic science.
To meet the objectives of this study, its authors reviewed past research and public information pertaining to the first 200 DNA exonerations that occurred between 1989 and 2007. The frequencies of systemic failures extracted from case profiles published by the Innocence Project were tabulated and analyzed with due consideration given to media reports that summarized individual cases. Authoritative texts were also consulted to help put the issue of wrongful convictions in proper context.
As a result of this study, forensic science malpractice, whether intentionally or accidentally committed, was shown to be a comparatively small risk to the criminal justice system—accounting statistically for less than 11 percent of all cases studied. As the authors will explain, the true percentage is likely much lower. But just as compelling were the number of wrongful convictions (18 percent) where forensic evidence reportedly favored the defendant.
In this regard, the available data strongly indicate that complaints about the overall quality of forensic science in American jurisprudence are mainly rhetorical in nature and are not based on valid statistical analyses of the role of forensic science in overturned convictions. While the profession of forensic science, like all critical professions, has a responsibility to continuously improve itself, its portrayal in the media as generally having an adverse impact on our criminal justice system is an injustice.
Click here to visit the article page on journal homepage (full article only available to subscribers).
October 2nd, 2009 on 7:20 am
I think that a good illustration of what Collins and Jarvis are pointing out in the wrongful conviction of forensic science, is the situation with the morphological analysis of human hairs. This issue came up during testimony before the National Academy of Sciences Committee on Forensic Science, that issued its report in February of 2009.
Assume that a competent hair examiner compared hairs from a crime scene with those from a known source by studying the morphology of the hairs in detail under low and high power microscopes. This was and is still considered to be a valid method for hair comparision and analysis. Assume that the examiner found many characteristics in common among the known and unknown hairs and no unexplainable differences. She then concludes that the owner of the known hairs could have been the source of the unknowns. She qualifies this conclusion that these characteristics could have also been shared by other people and that the number of other people is unknown. Fast forward to the present, when nuclear DNA typing can be done on hairs when sufficient root is present. Assume that the DNA typing in this case excludes the suspect. The question is: did the original examiner err in her conclusion? The answer is a resounding NO. This is simply a matter of resoultion: DNA typing has higher resolution than morphological analysis of hairs. The conclusion of the original examiner was correct – the suspect could have been the source of the hairs. Yet, if the suspect is wrongfully convicted, then forensic science will be blamed because the suspect was ultimately shown to not be the owner of the hair. Before forensic science is blamed for wrongful convictions, the whole context needs to be known. It is clear from the analysis by Collins and Jarvis, that this was not done in the analysis of cases of wrongful conviction.
October 2nd, 2009 on 8:22 am
With toxicology testing and solid dose drug testing, preliminary (or screening) tests are used to narrow down the types of substances and assist the examiner to decide on the confirmatory tests. Often, preliminary results will not reported unless the results are negative and the lab report can state that no drug is present. Is the microscopic examination of hair more like a screening test then a confirmatory test and if so should microscopic hair examinations be reported out? Or is there sufficient information to report out that such information may be reported as long as there are adequate caveats in the report alerting the police investigator, prosecutor and defender that DNA testing is indicated to ascribe identity to the donor?
October 6th, 2009 on 8:13 am
The presumptive/confirmatory dichotomy is one that haunts hair microscopy and mtDNA analysis. Presumptive and confirmatory tests are a construct of drug testing and chemistry in general (to include serology, which is really the chemistry of body fluids). The concept works there.
Microscopy, however, is no more a “screening” test for mtDNA than the latter is a confirmation of the former. Different modes of information are offered by hair microscopy (phenotypic) and mtDNA (genotypic) and to consider that one “trumps” the other is to misunderstand the nature of these two tests and what they offer.
A good example is the case offered in my chapter with Scott Ryland in “Trace Evidence Analysis” regarding a pubic-area hair found on the body of a nine-year old murder victim. The hair was a human hair with characteristics that identified it as coming from a post-pubertal person–de facto not the victim. Without that specific body area determination, it could have been “any” kind of hair and, although the mtDNA sequenced matched, could have been explained away by even the most flimsy alibi.
Along with Jay’s comment about resolution (and I am a footnote in the NRC report on this issue!), it is key for forensic scientists to understand their methods and not try to dump all tests into a limited set of baskets.
January 11th, 2010 on 1:39 pm
In a follow-on article, “Contextual Contamination of Forensic Evidence by Post-Conviction Litigators” (May 12, 2009) Collins and Jarvis complain of the anecdotal evidence used to “convict” forensic science, then they use anecdotes to support their argument. They argue:
Forensic science is incapable of determining guilt or innocence. The term DNA exoneration, used so frequently by journalists who report on overturned convictions, is a misnomer. DNA does not exonerate innocent prisoners – people do. As the Ronald Cotton and Steven Barnes cases demonstrate, very critical and sensitive leaps of logic are needed to cross the line that divides a DNA test result from the confirmation of innocence. Even though DNA results may seem intuitively exculpatory, extreme caution must be exercised. For this reason, the use of DNA evidence to overturn previous convictions is a profoundly serious matter that should be left to the devices of equally serious professionals.
How disingenuous! The semen found in Robin Thompson was that of another rapist, one who had no connection with Mr. Cotton. The DNA evidence showed conclusively that some other dude did it, and he was convicted for it.
If the idea is to trade anecdotes, consider the wrongful conviction of Mr. James Bain, freed after 35 years in a Florida prison in December 2009. Forensic serology was used to bolster a mistaken eyewitness identification. The assailant was type B. Mr. Bain was type AB, but a forensic “scientist” characterized the “A” component of Mr. Bain’s blood type to be “weak,” thereby allowing the jury to ignore the contradictory evidence. The defense serologist characterized the “A” component as “strong,” but what of it? He was obviously some defense whore.
Collins and Jarvis argue that, “All of these complex nuances illustrate that post-conviction forensic evidence must be treated with the same degree of care and caution as evidence used during trial.”
In fact, they have to know that the consideration of evidence after conviction is always “more careful.” The usual case is that appellate courts do not care one bit about evidence adduced after a conviction. The defendant has the burden of proof, and bears the additional burden of explaining why any “new” evidence was not presented in the original trial.
The issue here is not whether forensic science is “wrong.” The issue is whether forensic scientists should just go along with advocates who will present weak associations as being meaningful, then overstate the meaning in closing arguments. Juries are not sophisticated enough to understand “limitations,” particularly if they are glossed over or not presented at all. (Let the defense attorney bring that up on cross—if he is sophisticated enough).
It is easy to defend a “consistent” finding in a hair comparison as being correct, even though later testing proves the hair is not really the defendant’s. All the scientist said was the hair COULD have come from the defendant, not that it did. “Piling on” one nearly meaningless association after another will “corroborate” a flawed eyewitness identification. Presenting five different “CSI” experts will surely pretty up an otherwise unpersuasive case. Forensic scientists have an obligation to recognize when their science is being misused, and to object.
The California Association of Criminalists Code of Ethics codifies this principle when it states, “Where circumstances indicate it to be proper, the expert will not hesitate to indicate that while he has an opinion, derived of study, and judgment within his field, the opinion may lack the certainty of other opinions he might offer. By this or other means, he takes care to leave no false impressions in the minds of the jurors or the court.”
Giving a “technically correct” opinion that leaves a false impression is unethical. Some “scientists” may feel that perhaps this kind of ethical slovenliness is acceptable in order to “protect society from an obviously guilty defendant.” It is this kind of contextual contamination that we should (but probably will not) worry about.
The responsibility of forensic science in wrongful convictions has certainly been overstated, and it is forensic science, after all, that allowed the wrongness of the convictions to be recognized. But arguments to the effect that forensic science is blameless are also overstated. To the extent that we recognize problems in any forensic science discipline, our responsibility is not to deny those problems, but to address them.
January 27th, 2010 on 12:58 pm
I share the frustration with the public association of forensic science with miscarriages of justice. However we need mechanisms to deal with it and getting concrete information as the authors have is essential.
I believe we also need to do more. There is a need to promote the idea that the clinical model does not suit forensic science and that reporting results whether as numerical output or descriptions of “matching” without some estimate as to the meaning is a risky business.
I think the essence of this problem is the interaction of science and the law and the mechanism by which science is used. There is little value in us complaining that the scientific evidence is misinterpreted if we as a community do not know what we mean ourselves. Words such as “consistent” and “could have” do not help.
A group I am a member of – Association of Forensic Science Providers (AFSP) have produced standards for the formulation of evaluative forensic science expert opinion. They are available in the Sept edition of Science and Justice. (Standards for the formulation of evaluative forensic science expert opinion ; Science and Justice 49 (2009) 161-164). I bring them to your attention because they are produced in response to the realisation that the risk of miscarriages of justice is high unless we have some agreement as to what the evidence actually means. We need not only to be in a position to assist as to the significance of the findings but also to interpret them in the context in which they occur.
The standards separate the need for robust science from the need to set expectations and assess the outcomes.
DNA is used a lot to illustrate the ideal in forensic science because of the availability of match probabilities and I will use it to try to make my point. If female DNA matching a victim is detected on the underpants of a suspect, do we have sufficient data to assist the court as to whether that can be explained as a result of social contact or by sexual intercourse? Background studies of transfer and persistence are needed in DNA as they were and continue to be for various trace evidence materials. We should be passed the point where the matching of the DNA is the issue to one where how the finding can be explained is the issue.
Many people are uncomfortable with this approach because of the fear that it is usurping the role of the jury. On the contrary, this clearly identifies the role of the forensic scientist and how they can assist the court. The book by Robertson and Vignaux published some time ago argues this point well.(Robertson, B. and Vignaux, G.A. (1995). Interpreting Evidence: Evaluating Forensic Science in the Courtroom. Chichester: John Wiley and Sons.)
None of the above will avoid the media making allegations about misrepresentations of the role of forensic science in the justice system but the more clarity and transparency we can provide the better.
February 18th, 2010 on 7:07 am
This article does not help the cause of forensic science evidence in court. First, I would not charatcterize it as a “statistical analysis”. It is simply a review of online case descriptions from the Innocence Project website, primarily in anecdotal fashion. If anything it is an exercize in the critique of a web site.
More importantly, the authors chose to redefine the category of forensic science errors as “forensic science malpractice”. That definition is not used in the context of criminal cases for good reason. I suspect the authors chose it because “malpractice” in the civil case context means that the standard of practice is determined by looking at what other specialists in the same area do. In effect, it allows the standard of practice to be set at the lowest common denominator among those colleagues who practice in the same field. It is, in our context, the rejected Frye standard of “general acceptance” in a disguised form. It is precisely that approach to forensic science evidence that was rejected by Daubert and criticized strongly in the NRC Report.
Further, even if we accept the numbers presented by the authors, an 11% error rate for forensic science evidence in criminal cases is not acceptable. An 11% error rate would not be accpetable in any scientific setting, especially when liberty and even lives are at stake. Would we have heart surgery by a cardiologist with an 11% error rate? The authors’ response to this error rate is simply to point the finger at other errors that are involved in wrongful convictions, in effect saying “we might not be very good but other errors, like bad lawyering, are worse than we are.” How callous. I am reminded of a song used to build self esteem in little children that begins “Everyone makes mistakes so why can’t I?”
The approach in this article is precisely the problem when we leave what is supposed to be a scientific analysis to mere self-examination. There is an inherent bias that results in a defensive posturing ill fitting to scientists.
September 23rd, 2010 on 8:31 am
I would argue that the use of DNA to exonerate wrongly convicted persons is an example of how forensic science is an asset to the justice system. Most of these cases involved sexual assault and many convictions were based on questionable identifications or other considerations. The fact that a forensic science technique for identifying perpetrators is also used to correct injustices speaks highly for the forensic science field.
As I told my students on the first day of class, while they are now in a community college forensics program, within 15-20 years they may hold a PhD and be the one to make a significant breakthorugh for the field such as Sir jeffreys did with DNA.
The possibilities are endless in forensic science and as long as practitioners remember their duty to the truth and not to a conviction then forensic science will and should continue to be well utilized as a serious aspect of the justice system.