Both Jeff Gamso and Scott Greenfield have written about Brooklyn Supreme Court Justice Mark Dwyer’s opinion in People v. Collins discussing the admissibility of a certain type of DNA analysis in a criminal case. Since I have an amateur’s interest in both science and criminal law, I thought it might be interesting to read the actual opinion.
Surprisingly, despite the complexity of the subject and the amount of detail, it’s actually pretty readable. Dwyer takes the time to explain a lot of what the reader needs to know to understand what’s going on. And if you’re interested in this kind of thing, it’s fascinating. I think I understand it, so let me see if I can explain some of the science…
The basic issue is whether the court should allow the prosecution to present a certain type of DNA evidence from the New York City Office of the Chief Medical Examiner. DNA evidence itself is generally considered very reliable, but in this case the evidence was based on a very small sample of genetic material discovered at the crime scene and processed using a technique called high sensitivity analysis. The judge had to decide if this specialized technique was good enough to allow it to be admitted as evidence.
To understand the basic problem, imagine that you get a voicemail message in which the caller is speaking so quietly you can’t understand what he’s saying. So you try turning up the volume. This makes the caller’s voice louder, but it also amplifies the crackle and pop of the phone line and the background noises behind the caller. The amplifier circuit even introduces some noise of its own. So even though the caller’s voice is louder, you still might not be able to make out what he’s saying.
Photographers working in low light see a similar phenomenon when they try to crank up their camera’s ISO sensitivity. It makes the picture brighter, but it also adds a lot of speckled noise to the image. This happens to the image for roughly the same reason turning up the volume doesn’t make the voice easier to understand: In trying to amplify the signal, you’ve also amplified the noise — and introduced some as well.
The same thing happens with high sensitivity DNA analysis. One of the steps involves using a process called PCR to vastly multiply the number of copies of the DNA markers in the sample, because the process for measuring the markers requires a lot of material to work with. This process is called “amplification,” and just as with the audio and photographic examples, it will amplify the noise and introduce some of its own.
When the starting DNA sample is fairly large, it’s like a voicemail caller who speaks loudly or a photograph taken in bright sunlight: The signal stands out clearly from the noise in the system. When the sample is small — so-called “touch DNA” or a very degraded sample — the noise can overwhelm the signal.
The ME’s office came up with an interesting way to deal with the noise problem: Basically, they fed samples of known DNA through the high-amplification process and measured the statistical likelihood of the different types of noise. This allowed them to develop mathematical models of the effects of the various types of noise.
That wasn’t enough to filter out the noise completely, but it did allow them to make probabilistic determinations about the original DNA sample. Given a particular amplification result, they could now estimate the probability of getting that result from a particular suspect’s DNA fed through the noisy amplification process. They could also produce an estimate of the probability of getting that result from a random member of the population for comparison. Finally, they could even do this for mixtures of DNA from several people. The end result is that they could say things like, “It is 100,000 times more likely that this is a mixture of the defendant’s DNA and that of two other random people than that it is a mixture of the DNA of three random people.”
That’s not the sort of trillions-to-one odds you get with pristine DNA samples, but it is pretty good. If the science is for real. Which was kind of the question in front of Judge Dwyer.
In New York, the standard for admissibility of scientific evidence is Frye, which basically boils down to saying that the methodology behind the evidence must be widely acceptable to the relevant scientific community.
The high sensitivity analysis technique in question had recently been invented by the New York ME’s office and was not used much elsewhere, so Judge Dwyer could not conclude that it was widely accepted on that basis. His analysis didn’t end there, however, because the technique might still be considered widely accepted if it consisted only of components that were widely accepted. That is, doing several widely accepted things together is probably still widely accepted.
Judge Dwyer found that many parts of the high sensitivity analysis technique were indeed widely accepted — polymerase chain reaction amplification, electrophoresis, Bayesian statistics — but not all of them. Therefore he found that high sensitivity analysis was not widely accepted.
The problem that struck me as most significant was the experiments conducted by the New York ME’s office to determine how high-amplification noise affected the known DNA samples. This was a unique experiment, and as such it was a great piece of science, but until other labs have reproduced the results, it can’t really be considered widely accepted. Furthermore, the custom software that does the statistical analysis has not been made available for review, so it’s not clear how other labs could ever reach a conclusion about its acceptability.
Judge Dwyer’s opinion goes into a great deal more detail about the potential causes for concern and why he’s ruled that the technique is not widely accepted. It’s clear he’s given this a lot of careful thought, and as an observer of our justice system, it’s great to see judges take such care to get the science right. It makes me feel good about the system.
Trust Jeff to ruin that:
…he did something that almost never happened with expert evidence: he did it properly. That’s worthy of some note. Frye is readily abused.
And Scott:
To the uninitiated, what happened in Collins may seem, well, normal, expected even, since it’s perfectly reasonable to expect judges to do their job. But what Justice Dwyer did was remarkably bold, maybe even exceptional. He said “no” to a new wrinkle in DNA analysis, and he did so after other judges said, “sure, why not?”
With pathetic regularity, the by-product of bad forensic science is revealed when defendants convicted with the utmost certainty are later exonerated.
Oh well. At least it felt good while I was reading it.
Leave a Reply