Forensic Laboratory Independence, Control, and … Laboratory Independence, Control, and...
Transcript of Forensic Laboratory Independence, Control, and … Laboratory Independence, Control, and...
Forensic Laboratory Independence, Control, and
Exonerations
Patrick L. Warren
May 20, 2015
Abstract
The relationship between forensic laboratories and the other institutions of
law enforcement varies widely over space and time in the United States. Some
jurisdictions have their own local lab within the police or sheriff’s department,
others depend on a statewide lab system either independent or under the state
police, and others still contract with a private lab to process their forensic
evidence. These different organizational forms may shift the incentives lab
technicians and managers have to provide timely and accurate analysis and
testimony. In this paper, I investigate the relationship between one particu-
lar institutional variant, local police/sheriff control of the crime lab, and one
particular outcome in the criminal-justice chain: the conviction of innocent de-
fendants due, in part, to faulty or misleading forensic testimony. Among the
200 largest counties in the U.S., counties with locally controlled labs have lower
rates of exonerations in which faulty forensics were implicated in the original
trial than similarly-situated counties without locally controlled labs that have
similar rates of exonerations where forensic problems were not implicated. This
difference is robust to state fixed-effects, a variety of observable characteristics,
and nearest-neighbor matching. The difference seems to be driven by control,
per se, as matched counties containing state-controlled labs have significantly
higher exoneration rates.
JEL Classification:
Keywords: Forensics, Law Enforcement, Exoneration,
1
1 Introduction
In the past decade, the forensic science community has been buffeted by a number
of scandals. In Massachusetts in 2013, state chemist Sonja Farak pled guilty to tam-
pering with drug evidence, potentially affecting 60,000 samples in 34,000 cases. In
2009, an audit of the Houston Police Department crime lab’s fingerprint unit found
irregularities in over half of the 548 cases reviewed.1 In 2010, lab workers in the North
Carolina state forensic lab were found to have failed to turn over potentially exculpa-
tory evidence, including in death penalty cases. The investigation of the State Bureau
of Investigation blood serology unit yielded a total of 229 cases of misrepresentation
of blood serology. Of the 229 cases, seven persons had been executed, others were on
death row, and some had died in jail.2
In the modern courtroom, forensic evidence plays a very important role. Trials
for defendants facing probative forensic evidence are more likely to end in a con-
viction and result in longer sentences (McEwen 2011). But this impact depends on
reliable testimony, and the anecdotes above suggest the high levels of reliability are
not guaranteed.
Almost simultaneous to the scandals, the National Research Council of the Na-
tional Academy of Science issued a report entitled “Strengthening Forensic Science in
the United States: A Path Forward”, in which a distinguished panel of scientists lays
out a number of recommendations to improve the accuracy and reliability of forensic
science in the United States (Committee on Identifying the Needs of the Forensic
Sciences Community, National Research Council 2009). The first of these recommen-
dations was to foster independent forensic organizations. In their words “[the current
system] leads to significant concerns related to the independence of the laboratory
and its budget. Ideally, public forensic science laboratories should be independent
of or autonomous within law enforcement agencies. In these contexts, the director
would have an equal voice with others in the justice system on matters involving the
laboratory and other agencies.”
The relationship between forensic laboratories and the other institutions of law
enforcement varies widely over space and time in the United States, with wide dif-
ferences in the levels of independence. Some jurisdictions have their own local lab
1“Ex-Chemist Pleads Guilty in Drug Evidence Theft,” Milton J. Valencia, Boston Globe, January7, 2014
2“Scathing SBI Audit Says 230 Cases Tainted by Shoddy Investigations,” Mandy Locke, JosephNeff, and J. Andrew Curliss, Raleigh News and Observer, August 19, 2010.
2
within the police department, sheriff’s department, or the county attorney’s office,
others depend on a statewide lab system either independent of or under the state
police, and others still contract with a private lab to process their forensic evidence.
These differences in organizational form may shift the incentives lab technicians and
managers have to provide timely and accurate analysis and to invest in the resources
necessary to do so. That shift could affect outcomes throughout the criminal-justice
process: crime rates, arrest rates, conviction rates, and exoneration rates.
The theoretical relationship between independence and performance is ambiguous.
Pressures placed on a lab by the police and prosecutor in an adversarial criminal-
justice system can potentially bias the outcomes of their investigations. Concerns
about this effect have dominated the discussion of lab independence in the law and
economics literature (Giannelli 1997, Koppl 2005, Koppl 2010). But there are forces
in the other direction that are, perhaps, under-appreciated in the policy discussion.
When good performance is difficult to specify contractually and/or coordination is
important, direct control can improve outcomes. In fact, many of our theories of
why firms exist, at all, turn on some variant of this idea (Gibbons 2005), and there
is vast empirical support for it (Lafontaine and Slade 2007). In brief, direct control
can provide stronger incentives and improve coordination, but strong incentives to
respond to the principal’s demands can increase both good and bad behavior and
not all coordination is consonant with the pursuit of justice. On net, police control
could either improve or degrade the performance of the crime lab. An empirical
investigation is required, and such an investigation will require variation in the level
of lab independence.
Forensic labs that are entirely independent of law enforcement are rare in the his-
tory of the United States, but even among labs that are related to law enforcement,
there are degrees of independence. Since most police investigations in the U.S. are
handled by city and county agencies, and most prosecutions are handled by city or
county attorneys, a crime lab that is under the control of local authorities might
be the least independent of all. Contrasting their behavior with that of more dis-
tantly connected labs may provide a window into the more general question of the
relationship between independence and performance. In this paper, I investigate the
relationship between one particular institutional variant, local police/sheriff control of
the crime lab, and one particular outcome in the criminal-justice chain: the conviction
of innocent defendants due, in part, to faulty or misleading forensics.
In the main analysis of this paper, I compare the rate of eventual exoneration
3
in counties with a locally-controlled crime lab to that in similarly-situated counties
without a locally controlled lab (although the county may include a lab controlled by
some other entity, such as the state government). In particular, I study the rate of
what I call forensic exonerations, exonerations in which faulty forensic were implicated
as a significant contributor to the initial false conviction, in the 200 largest counties
in the U.S., using the rate of non-forensic problems as a control to proxy for all
non-forensic factors that might relate to exoneration. In this analysis, counties with
locally-controlled labs seem to experience MUCH lower rates of forensic exonerations
than those without. Counties with local labs experience between 0.6 and 1 fewer
exonerations per million residents in 1980, on a mean about about 1.25 exonerations.
The analysis on the extensive margin is similar. About 40 percent of these large
counties have experienced a forensic exoneration recorded in the National Registry
of Exonerations, but counties with locally controlled labs are about 20 percentage
points less likely to have had one. All these results are robust to various county-level
controls and state fixed effects.
These basic results in hand, I explore what is different about the counties with
locally-controlled labs that might be contributing to the difference in exoneration
outcomes. Turning, first, to a more contemporary data set containing judicial out-
comes for felony defendants in large urban counties, I document that the exoneration
results do not seem to arise mechanically from fewer convictions, as might be the
case if locally controlled labs simply provided less evidence or hedged in favor of the
defendant. On the contrary, counties with locally controlled labs have higher felony
convictions rates, fewer dismissals, and more jail sentences than those without. These
results are robust to controlling for case mix and load and spending on police.
Next, from a survey of forensic professionals conducted in 1977 (roughly the be-
ginning of the set of cases that were at risk to appear in the exoneration list) and from
a census of publicly funded crime labs in 2008 (roughly the end of the exoneration
period), I obtain some proxies for the human capital of the people working in the
crime labs: education and salary levels. For all measures, those working in county
crime labs reported higher levels of human capital than those working for state crime
labs. This is evidence that locally-controlled crime labs invest more, as organizations,
in the quality of their staff. Such investments might reduce both type-1 and type-2
errors.
4
2 Empirical Analysis of Error and Forensic Lab
Control
The unit of observation for this paper will be the county. Looking at the 200 most
populous counties in the U.S., over the past three decades, I explore whether the
presence of a locally controlled crime lab is related to the quality of forensic ser-
vice3. The basic idea is to compare rates of exonerations where forensic problems
were implicated in the original trial, while controlling for a proxy for the quality of
convictions, in general. The basic result is that counties with locally controlled crime
labs have lower rates of forensic exonerations than their counterparts who must send
their forensic work to an outside lab (mostly to state-controlled labs).
2.1 Errors in the Criminal Justice Process
People have worried about errors in the criminal justice process for a long time,
including false convictions (Borchard 1932). But with the advent of DNA testing,
errant convictions have been easier to identify and their underlying causes have been
subjected to more careful investigation (Jacoby et al. 2005, Gross 2008, Hampikian,
West and Akselrod 2011, Gross, O’Brien, Hu and Kennedy 2014). Although mis-
leading forensic evidence has appeared in those investigations, its role has not been
pronounced. Historically, more attention has been paid to other sources of error,
such as perjury, witness misidentification, or false confessions. In one striking excep-
tion, Garrett and Neufeld (2009) performed a comprehensive analysis of the forensic
evidence presented in the trials of defendants who were later exonerated by the Inno-
cence Project through DNA evidence, documenting errors is a large fraction (≈ 40%)
of these cases.
Of course, false convictions are only one side of the coin in terms of the errors.
Errors can also lead to false acquittals (or, more generally, to general failures to catch
and punish the perpetrator appropriately). The literature on this side of the error
calculations has focussed on conviction rates, and how variations in budgets, career
incentives, and political incentives affected them (Rasmusen, Raghav and Ramseyer
2009, Gordon and Huber 2009).
Compared to exonerations, conviction rates are a relatively poor measure of errors,
since higher conviction rates (or crime closure rates, more generally) do not necessarily
3All results are robust to limiting to the top-100 counties.
5
mean fewer people getting away with crimes, if many of convictions are in error. For
this reason, I will look first at exonerations as the primary measure of error. Only
with those results in hand can we properly evaluate any effects on conviction rates.
2.2 Control of The Lab
The relationship between crime labs and police has varied considerably over space
and time in the United States. According to a 2008 census by the Bureau of Justice
Statistics, there are 411 publicly funded forensic-science labs in the United States,
217 of which are controlled by state or regional authorities, 90 of which are controlled
by county governments, and 66 of which are controlled by municipal authorities. The
remaining 38 labs are part of the federal government. Getting a handle on the number
of privately-controlled forensic science labs is more difficult, as there is no official
count conducted, but there are 30 private labs that are accredited by the American
Society of Crime Laboratory Directors, the largest and best-known accrediting body
for forensic labs.4
2.3 Data and Econometric Approach
The data on exonerations are drawn from the National Registry of Exonerations,
which is a continually updated list of every known exoneration in the United States
since 1989, where an exoneration is defined as a case in which a person was wrongly
convicted of a crime and later cleared of all the charges based on new evidence of
innocence. It details the year and county of each original trial, the most serious crime
of which the defendant was convicted, and the factors identified as contributing to the
original false convictions: false confession, mistaken witness identification, perjury,
official misconduct, inadequate defense, and false or misleading forensic evidence.
After limiting the sample to violent crimes5 and dropping federal cases, there are 1193
exonerations, of which just over half (605) have murder as the most serious conviction
and about a third are sexual assaults or child sex abuse cases (413). Forensic problems
are identified by the Registry as contributing to the original conviction in 23 percent
of the cases, the second-least common contributor (false confessions are least common
at 13 percent). The most common problems are perjury (57) and official misconduct
4As of July, 2014, there were 184 ASCLD accredited state labs, 135 county or municipal labs,and 31 federal lab.
5Violent crimes include murder, manslaughter, sexual assault, child abuse, arson, assault, at-tempted murder, and robbery.
6
(46), with mistaken witness identification contributing in 40 percent of the cases.
The median year of the initial conviction was 1991, with 95 percent of convictions
occurring after 1978. From these data, I derive the two most important variables in
the empirical analysis. Forensic exonerations, those where faulty forensics is identified
as a key contributor to the original false convictions, will be the measure of forensic
quality, while the number of non-forensic problems in the full set of exonerations
will be the most important control for factors beyond forensic control that might be
driving exonerations.
Many of these exonerations arose from cases pursued by a collection of non-profit
legal clinics known as Innocence Projects. These groups are patterned after the
original Innocence Project, which was founded in 1992 by Barry C. Scheck and Peter
J. Neufeld at the Benjamin N. Cardozo School of Law at Yeshiva University to assist
prisoners who could be proven innocent through DNA testing. Although some of
these organizations accept clients nationally, they tend to be regionally specialized,
and cases that are close to “home” might be prioritized. Since the chance that a given
false conviction ends in exoneration can depends on whether it attracts the attention
of an innocence project, I collected the location of each member organization from
the innocence project and will include whether a county has an innocence project as
an additional control in some specifications.6 Between this control, and the control
for non-forensic problems, above, I hope to control for the potential influence of case
selection.
The only unusual data on county-level institutions included is information about
forensic science labs present in the county. A contemporaneous census of world foren-
sic labs conducted by the Forensic Science Society (Forensic Science Society 1977)
in 1977 provides a comprehensive list of public and private forensic labs. I coded
the presence of forensic labs in each county, as well as who controls the lab, using
historical newspaper and court accounts in cases where control was not clear from
the census. In 1977, 35.5 percent of the top-200 U.S. counties by 1980 population
had some sort of locally-controlled public forensic lab: 15 percent with a county lab
only, 18 percent with a municipal lab, only, and 3.5 percent with both. In the same
period, 31 percent of these counties contained state-controlled labs, with 9 percent of
counties having both locally-controlled and state-controlled labs.
6There are some intermediate-outcome concerns with this specification, if we think that inno-cence projects are more likely to arise in areas with more false-convictions, but I think the bias fromcase-section is the greater worry, so I do include it. Specifications without local Innocence-Projectindicators yield nearly identical estimates
7
More prosaic county-level data on law-enforcement resources, demographics, and
geography are gathered from a variety of common sources. Informational about spend-
ing on police and law enforcement spending comes from the 1982 census of govern-
ments, while population and demographic information come from the 1980 census.
Crime data are drawn from the county-level Department of Justice Uniform Crime
Reports from 1980 through 1983, which are averaged.
Exonerations are rare phenomena and exonerations where forensics were impli-
cated are particularly so. Of the top-200 U.S. counties by 1980 population, 64 percent
have had no exonerations in which forensic errors were identified in the initial con-
victions. So rather than using a panel approach I will instead conduct a cross-section
analysis using the entire time period, examining both the probability of having a
forensic exoneration and the forensic exoneration rate (per million people in 1980).
Thus, I present regressions of the form
FExoni = βLocalLabi + δNonForenProblemsi + ΓXi + εi, (1)
where the outcome variable is either a dummy equal to one if the county i has had
any forensic-related exonerations or the number of forensic exonerations per million
residents. The key variable of interest is an indicator for a locally controlled crime
lab in the county, which I will sometimes divide into two dummies, one representing
the presence of a county-wide local lab, almost always controlled by the sheriff, and
one representing the presence of a public lab in a municipality within the county, but
which does not span the entire geography of the county. If the city and county are
co-terminus, any locally-controlled lab is coded as a county lab.
Every regression will include the key control variable, the number of non-forensic
problems per capita. The purpose of this variable is to try to control for all factors that
contribute to exoneration rates, in general, and which might be correlated in a non-
causal way with the presence of a local crime lab. Thus if, for instance, counties with
local crime labs happen to, say, have higher quality and more professional police force,
and professional police lead to fewer exonerations, then we want to control for that
general lower rate of exonerations. Otherwise, we would improperly attribute that
effect to the locally-controlled crime lab. Any omitted variable that is correlated with
locally-controlled crime labs and forensic exoneration is a risk for biasing estimates,
but if the correlation with forensic exonerations is zero, conditional on the rate of
non-forensic problems, there will be no omitted-variable bias. Essentially, this is the
key identification assumption–conditional on the rate of non-forensic problems (and
8
other control variables, as appropriate), the presence of a locally controlled forensic
lab is uncorrelated with unmodelled factors that causes forensic exonerations (the
error term in regression 1).
The variables included in Xi will vary with specifications, but they include some
subset of: murder and rape rates (crimes per 100k population), an indicator for
a county including the state capital, county-level spending on police (log(spending
M$)), fraction black, fraction hispanic, indicators for census region, and/or state-level
fixed-effects. Finally, these regressions are run on two samples of counties. First, the
full top-200 counties by 1980 population, excluding the counties making up New York
City. New York City is omitted because allocating crime rates and police resources
among the borroughs/counties is impossible. Including them as a single county or
as five identical counties has no substantive effect on the result. Second, a restricted
set of counties containing only those top-200 counties which contain either a locally-
controlled lab or a state-controlled lab. This subset is of interest if we think that the
presence of a convenient lab might have a direct effect, and we want to just contrast
control per se.
Table 1 presents the sample means for the key variables, separated by the presence
of a municipal or county crime lab. From these descriptive statistics, alone, there are a
number of differences between the counties that have local crime labs and those that
do not. First, counties with county labs have lower rates of forensic exonerations,
despite counties with any sort of local labs having much higher rates of non-forensic
exonerations and non-forensic problems. Second, counties with local labs are much
more populous than those without local labs, even in this selected sample of big
counties, and the difference is even bigger for counties with city labs or both. Despite
being twice as populous, counties with only county labs are equally likely to have
experienced a forensic exonerations as those without a lab. But the counties with
municipal labs are so much bigger than those with no local labs that they are much
more likely to have experienced any forensic exonerations. Crime rates are higher
for counties with local labs, and again higher still in those with city labs. They also
spend more on police. It is important to remember that these crime rates come from
the peak of the 80s urban crack epidemic, so they are much higher than current rates.
Counties with local labs have higher fractions black and hispanic. Counties with local
labs are more likely to be located in the midwest and west and less likely to be in the
northeast than counties without local labs. Finally, counties with county labs are less
likely to have a state lab, or to contain the state capital than counties with no local
9
labs, while counties with a city lab but no county lab are more likely to contain the
capital and a state lab.
A consistent story arises just from these means. Large, populous counties in the
midwest and west are likely to have high crime rates, large minority populations,
large police budgets, and their own crime labs. They are also likely to have high rates
of exoneration, in general, but without a proportionately higher rates of forensic
exoneration. The rates for non-forensic problems provides a natural control for these
underlying differences that might be accidentally correlated with the presence of a
local lab. Something like a differences-in-differences estimate is available: do the
differences in forensic exoneration rates between counties with local labs and those
without mirror the differences in non-forensic problems? The answer seems to be no.
If anything, the gap in forensic exonerations rates between those with local labs and
those without seems to go in the opposite direction and is certainly smaller than the
concomitant gap in non-forensic problems, suggesting a positive causal effect of local
control on the quality of forensic analysis and testimony.
Figure 1 illustrates this difference, by calculating the fraction of total exonera-
tions in which faulty forensics were implicated and comparing the mean rate of this
fraction among counties into those including labs controlled by county sheriffs, those
including labs controlled by municipal police, and those including labs of both types.
The blocks indicate the number of counties of each type. They do not add up to 200,
since 53 counties had no exonerations of any type. The dot indicates the mean, while
the whiskers indicate 1 standard error on the mean. The pattern is straightforward.
Counties with more local control of the forensic labs had a forensic problems impli-
cated in a lower fraction of their overall exonerations. Of course, further investigation
is warranted, which the next section pursues.
2.4 Local Control and Exoneration: Regression Estimates
The results in the OLS estimations of variants of equation (1) are presented in Table 2.
The first panel includes only dummies for county- and city-controlled labs, together
with the non-forensic problem rate. The second panel adds a variety of county-
level controls as well as census-region fixed-effects, and the third panel replaces those
census-region dummies with state-specific fixed-effects. Columns (1) and (3) have the
rate of forensic exonerations as the dependant variable, while columns (2) and (4) are
linear probability models with an indicator for the presence of a forensic exoneration
as the dependent variable. Finally, the first two columns are run on the full sample,
10
Figure 1: Forensic Exonerations as Fraction of Total Exonerations
11
while the last two columns restrict the sample to only those counties which have
either a locally-controlled lab or a state-controlled lab (or both).
Throughout all specifications, there is a negative relationship between the presence
of a county-controlled forensic lab in 1977 and the rate of eventual forensic exonera-
tions arising in that county (columns 1 and 3). Counties with county-controlled labs
have between 0.6 and 0.9 fewer forensic exonerations per million residents than we
would we would predict given their rate of non-forensic problems and other observ-
ables. With the mean forensic exoneration rate of about 1.25 per million this is a very
substantial effect. These results are robust to the inclusion of county controls and
state-fixed effects. The relationship for counties containing a city with a lab is also
consistently negative, although rarely statistically significant. Restricting the sample
to counties having a lab, as is done in regression (3), does nothing to upend the result.
In fact, the point estimates are quite consistent, suggesting that the key contrast is
not between counties with and without labs but rather between labs controlled by
the state and those controlled locally.
An alternative way to look at the problem is to ignore the intensive margin and
simply ask whether counties with local crime labs are more or less likely to have
experienced a forensic exoneration, regardless of the number. Since exonerations are
quite rare (128 counties had no forensic exonerations, 42 had exactly one, and only
six counties had five or more), much of the variance in the rate measure is actually
driven by population differences. Since population, and even forensic exonerations,
are likely measured with error there is a chance that the intensive margin is adding
more noise than real information. It is also potentially more sensitive to outliers,
such as Chicago, with 24 forensic exonerations–by far the most.7 Columns (2) and
(4) present the results of a linear-probability model of this type. The results in the
first panel are clearly biased, since there is no control for scale and more populous
counties are clearly more likely to have local crime labs and have experienced at least
one forensic exoneration, if only because they have experience many more crimes and
convictions. The other two panels, however, give quite consistent results. Counties
with locally controlled crime labs are less likely to have experienced a forensic-related
exonoration. Again, county-controlled crime labs have the most robust effect– with
counties having local labs being about 20 percentage points less likely to have had
a forensic exoneration, on a mean of about 40 percent. Counties with cities with
7Chicago is not driving the main results. Dropping it from the analysis moves the point estimatesinsignificantly.
12
labs are also consistently less likely to experience exonerations, although the results
are never statistically significant. Again, the results when restricting the analysis to
counties containing crime labs are quite consistent with the more general analysis,
suggesting that local control is the key factor, rather than simply having a lab close
by.
Although the regressions in Table 2 attempt to control for all factors that might
be related to local control and forensic exonerations, there remain some chance of
bias. The sample means suggested that there were a number of important differences
in average observable characteristics between counties with local control and those
without. Perhaps the linearity assumptions of OLS have failed to capture these differ-
ences adequately and some bias remains. An alternative approach is to concentrate
on one type of local control, say county-controlled labs, and select a quasi-control
group by choosing a subsample of the counties without that type of control which is
“close” to treated group on a variety of other observable characteristics. This kind of
“nearest-neighbor” matching eschews the linearity assumption at the cost of discount-
ing the information in the observations that are “far” from any observations with the
opposite treatment. Table 3 contains the results of this analysis, where the matching
variables in each panel are the exact set used as controls in the OLS analysis, three
matches are chosen for each treated observation, and the dependent variables in the
columns match those in the OLS analysis.
The matching results for the exoneration rates are nearly identical to the OLS
results, both on the lab-containing counties and on the full set. The linear-probability
results are of the same sign as the OLS results, although only statistically significant
in the within-state matches. There are two possible interpretations of this result. In
sum, the results do not seem to be driven by the linear projection assumptions in
OLS.
3 Why the Difference?
Across specifications, samples, and econometric approaches, counties with locally con-
trolled labs have lower rates of exonerations in which faulty forensics were implicated
in the original trial than similarly-situated counties without locally controlled labs.
In this section, I explore what might be different about the criminal justice process in
counties with locally-controlled labs. The available data varies enormously for each
potential difference, maintaining a consistent approach throughout these comparisons
13
is infeasible. Instead, I will present the best data available, even as the time frame
and unit of analysis varies, while being clear, throughout, as to the origin of the data
and my econometric approach. I briefly summarize the results, here, before turning
to each detailed analysis, below.
The first potential difference I explore is whether there are differential conviction
rates in counties with local labs. Perhaps counties with local labs had fewer exon-
erations, controlling for crime rates, because they actually had fewer convictions–
either they were ineffective at catching criminals or in providing good evidence to
prosecutors, so there were, mechanically, fewer false convictions. Or, perhaps, they
are simply trading off in a different way on “close calls” and choosing to have fewer
“close” convictions and, therefore, fewer exonerations. On the contrary, between 2002
and 2006, counties with locally-controlled crime labs had much higher felony convic-
tion rates and much lower dismissal rates than those without, controlling for case mix
and load, population, and policing resources. Thus, neither of these proposed mecha-
nisms seems correct. Counties with locally controlled labs have both more convictions
and fewer exonerations.
A second potential difference is whether the lab personnel were better trained or
more experienced in local labs than in state labs. That seems correct. In two surveys,
one before the period of the bulk of the trials and one after, crime lab employees
at county labs have high levels of human capital than those working in state labs.
They are more likely to have graduate degrees, more experience, and are paid higher
salaries.
3.1 Conviction Rates
The only reliable data on conviction rates at the local level come from the Department
of Justice’s State Court Processing Statistics: Felony Defendants in Large Urban
Counties series, which includes an unbalanced panel of a sample of the 75 largest
urban counties in the U.S. from 1990 to 2006 (United State Department of Justice–
Bureau of Justice Statistics 2002-2006). They track the eventual outcomes of every
defendant ever charged with a felony for up to two years, recording the most serious
crime charged faced, the gender and race of the defendant, and the final outcome of
the charges. The public- use data are aggregated to the county level, so these variables
become data on the mix of initial charges, demographics of defendants, number of
cases, and fraction of cases resulting in various judicial outcomes.
The best contemporaneous measure of the presence of a locally-controlled crime
14
lab comes from the Census of Publicly Funded Forensic Crime Labs (United State
Department of Justice–Bureau of Justice Statistics 2009).8 Since the presence and
control of crime labs did not change much over this decade, I will ignore the panel
nature of the data and instead treat it as a cross-section over the last three sample
years (2002, 2004, 2006), accounting for the correlation in conviction rates if a county
is sampled multiple times by clustering the standard errors at the county level. I
pair this series with data from standard sources on police spending, crime rates, and
population.
The basic differences in outcomes are presented in Figure 2, where the four sub-
graphs present the four potential case outcomes for defendants initially charged with a
felony: felony conviction, misdemeanor conviction, dismissal and acquittal. Each dot
represents the average share of cases resulting in each outcome among counties with
the indicated presence of a locally-controlled crime lab, while the whiskers represent
a standard error from the mean in each direction. The bars represent the number
of observations (right axis). As is immediately apparent from the first sub-graph,
counties with no locally controlled crime labs have felony conviction rates right at
50%, while those with a county or city crime lab, and especially those with both,
have much higher felony conviction rates.
Of course, this difference could be driven by other factors about those counties,
so Table 4 presents the results of a regression that controls for various factors that
might be related to local control of the crime lab and conviction rates, such as the
mix of crimes, case load, and police spending. The regression results reinforce those
from the figure. Defendants in counties with locally-controlled crime labs have higher
felony conviction rates, lower dismissal rates, and a greater likelihood to end up in
jail than those in counties without a locally controlled crime lab.
8Censuses were also conducted in 2002 and 2005, but they did not identify the agency withadministrative control over the labs.
15
Figure 2: Case Outcomes by Control of Lab
(a) Felony Conviction (b) Misdim. Conviction
(c) Dismissal (d) Acquital
16
Table 1: Sample Means by Presence of Local Crime Lab- 200 Largest Counties in1980 (sd)
No Local City Lab County Lab BothForen. Exon. Rate 1.317 1.362 0.969 0.790
(2.716) (1.854) (2.321) (0.986)Forensic Exoneration 0.326 0.500 0.324 0.571
(0.470) (0.509) (0.475) (0.535)Non-F. Problem Rate 6.854 11.51 12.19 9.838
(8.826) (12.17) (22.96) (3.816)Non-F. Exon. Rate 2.960 5.399 5.815 5.213
(4.197) (5.326) (9.239) (3.147)State Lab 0.341 0.367 0.176 0.143
(0.476) (0.490) (0.387) (0.378)Pop. 1980 (M) 0.398 0.902 0.742 2.317
(0.204) (0.961) (0.356) (2.361)Capital County 0.132 0.267 0.0882 0
(0.340) (0.450) (0.288) (0)Inno. Project 0.101 0.500 0.176 0.286
(0.302) (0.509) (0.387) (0.488)Rape per 100k 35.24 52.91 48.21 51.97
(21.87) (16.87) (24.46) (19.72)Murder per 100k 7.199 13.00 10.56 12.25
(6.499) (7.375) (8.228) (8.125)Frac. Black 0.102 0.168 0.124 0.125
(0.111) (0.135) (0.114) (0.118)Frac. Hisp. 0.0519 0.0665 0.0789 0.133
(0.108) (0.103) (0.0902) (0.0961)Census Northeast 0.341 0.200 0.206 0
(0.476) (0.407) (0.410) (0)Census South 0.295 0.267 0.324 0
(0.458) (0.450) (0.475) (0)Census West 0.163 0.133 0.324 0.714
(0.371) (0.346) (0.475) (0.488)Census Midwest 0.194 0.400 0.147 0.286
(0.397) (0.498) (0.359) (0.488)Police Budget (Mil. 1982) 1.418 4.858 3.206 12.37
(1.226) (8.041) (2.190) (15.96)n 130 30 33 7
17
Table 2: Eventual Exonerations by Availability of Local Crime Lab
Full Sample Counties with Labs(1) (2) (3) (4)
F. Exon. Rate Any F. Exon F. Exon. Rate Any F. Exon
Panel 1: Non-Forensic Problems ControlCounty Lab −0.86∗∗∗ −0.05 −0.97∗∗ −0.06
(0.30) (0.08) (0.42) (0.09)City Lab −0.38 0.14 −0.49 0.13
(0.32) (0.09) (0.41) (0.10)Non-F. Problem Rate 0.12∗∗∗ 0.01∗∗∗ 0.11∗∗∗ 0.01∗∗∗
(0.02) (0.00) (0.02) (0.00)Panel 2: County Controls
County Lab −0.69∗∗ −0.21∗∗ −0.63∗ −0.20∗
(0.29) (0.09) (0.35) (0.11)City Lab −0.37 −0.07 −0.24 −0.06
(0.32) (0.10) (0.37) (0.11)Non-F. Problem Rate 0.12∗∗∗ 0.01∗∗∗ 0.12∗∗∗ 0.01∗∗∗
(0.02) (0.00) (0.02) (0.00)Pop. (1980) 1.16∗ 0.39∗∗ 1.32 0.31
(0.65) (0.16) (0.93) (0.23)Capital County 1.66∗∗ 0.20∗ 2.04∗∗ 0.27∗∗
(0.81) (0.10) (0.85) (0.12)Rape per 100k 0.00 −0.00 −0.01 −0.00
(0.01) (0.00) (0.01) (0.00)Murder per 100k 0.03 0.01 0.00 −0.00
(0.05) (0.01) (0.06) (0.01)Frac. Black −6.76∗∗ −0.53 −7.13∗ −0.50
(2.82) (0.54) (3.66) (0.72)Frac. Hisp. −2.27 −0.21 −2.29 0.04
(1.42) (0.35) (2.34) (0.58)Inno Project 0.06 0.04 0.06 0.06
(0.47) (0.11) (0.54) (0.12)log(Police Budget) −0.78 −0.08 −0.75 0.04
(0.49) (0.12) (0.73) (0.17)Panel 3: + State FE
County Lab −0.88∗∗ −0.25∗∗ −0.91∗ −0.29∗∗
(0.36) (0.11) (0.50) (0.14)City Lab −0.64∗ −0.09 −0.71 −0.17
(0.36) (0.13) (0.48) (0.14)Non-F. Problem Rate 0.11∗∗∗ 0.01∗∗∗ 0.11∗∗∗ 0.01∗∗∗
(0.01) (0.00) (0.01) (0.00)Dependant Variable Mean
Sample Mean 1.25 0.36 1.39 0.41n 200 200 123 123
Notes: Regressions on top-200 U.S. counties in 1980 population. Dependent variables are, in (1)and (3), the number of forensic exonerations per million residents and , in (2) and (4), a dummywith 1 indicating the presence of a forensic exoneration. Robust standard errors, with statisticalsignificance of test against zero null indicated by ***: 0.01, **:0.05, and *:0.10.18
Table 3: Eventual Exonerations by Availability of Local Crime Lab (Nearest-NeighborMatch)
F. Exon. Rate Any F. Exon F. Exon. Rate Any F. Exon
Panel 1: Match on Non-Forensic Problem RateCounty Lab −0.77∗∗∗ −0.07 −0.88∗∗ −0.08
(0.27) (0.08) (0.37) (0.09)Panel 2: Also Match on County Controls
County Lab −0.62∗ −0.11 −0.75∗∗ −0.14(0.34) (0.11) (0.30) (0.10)Panel 3: Also Match on State FE
County Lab −0.65∗∗ −0.14∗ −0.88∗∗∗ −0.22∗∗∗
(0.28) (0.08) (0.26) (0.08)n 200 200 123 123
Notes: Nearest-neighbor matching on top-200 U.S. counties in 1980 population. Dependent variablesare, in (1) and (3), the number of forensic exonerations per million residents and , in (2) and (4),a dummy with 1 indicating the presence of a forensic exoneration. Robust standard errors, withstatistical significance of test against zero null indicated by ***: 0.01, **:0.05, and *:0.10.
19
Table 4: Judicial Proceedings Outcomes by Control of Crime lab
Felony Convict Dismissal Prison Sent. Jail Sent.Panel 1: Simple Mean Differences
County Lab 8.43∗ −7.19∗ −1.83 14.81∗∗∗
(4.63) (3.86) (3.63) (4.29)Muni. Lab 11.70∗∗ −7.46∗∗ 5.26 −4.79
(4.43) (3.63) (4.73) (4.26)Panel 2: + Case Mix and Counts, County Pop, and Year FE
County Lab 10.48∗∗ −10.48∗∗∗ −1.01 13.97∗∗∗
(4.60) (3.67) (4.20) (4.99)Muni. Lab 5.42 −4.13 4.63 −6.18
(4.75) (4.26) (4.55) (5.71)% Public −0.10 −0.51∗ −0.12 0.21
(0.41) (0.28) (0.42) (0.40)% Violent −0.87∗∗∗ 0.61∗∗ −0.21 0.32
(0.26) (0.26) (0.26) (0.45)% Property −0.20 0.12 −0.05 0.09
(0.29) (0.24) (0.17) (0.25)Log(Cases) −6.67 13.96∗∗∗ 1.32 −1.98
(4.91) (4.59) (3.05) (4.24)Log(Pop.) 8.36∗∗ −6.96∗∗ −0.54 4.66
(3.80) (3.07) (3.36) (4.45)Police Spend/Cap −0.35∗∗ 0.17∗∗ −0.26 −0.10
(0.14) (0.07) (0.33) (0.30)Sample Mean of Dependent Variable
Sample Mean 56.79 22.61 36.44 34.63n 111 111 111 111
Notes: Dependent Variable is percent of felony defendants with indicated case outcome. Observa-tions from unbalanced panel of a subset of 75 large urban counties from 2002, 2004, and 2006.
3.2 Human Capital of Lab Personnel
Ideal data on the human capital of the laboratory personnel would include information
about the individuals who were working at the lab at the time of the original trial.
Two surveys, one at the beginning of the period and one near the present provide the
best available data on this question. The evidence of human capital in these surveys
differs significantly, but they tell a very consistent story. Federal lab employees have
the highest level of human capital, as evidenced by training, experience, and salary,
while state lab employees have the lowest levels. County and and city employees fall
in between, with county employees outperforming state and city employees in nearly
20
every dimensions and at nearly every level of the hierarchy of responsibility.
A 1976 survey conducted by the Forensic Science Foundation on behalf of the
U.S. Department of Justice investigated the characteristics of every role in the foren-
sic science process: Coroner/Medical Examiners, criminalists, anthropologists, odon-
tologists, psychiatrists, toxicologists, evidence technicians, and questioned-document
examiners (Field et al. 1977). The original data of the survey have been lost, but the
report prepared by the surveyors offers some useful evidence of the level of human
capital in various forensic labs in that era. For criminalists, the forensic scientists
who analyze the bulk of the forensic evidence and provide most trial testimony, the
survey report provides counts of respondents by level of education/experience and
control of the forensic lab. The first two columns of first panel of Table 5 present
means for two of these measures, the fraction with graduate degrees and the fraction
with at least 10 years of experience. For each measure, criminalists working in county
labs outscore those in state labs, while municipal lab employees are less educated
but more experienced than both. No salary information is available for criminalists
broken out by lab control, but the survey does break down the salary of evidence tech-
nicians. Again, county technicians are much more likely to be paid over $15k/year
than state technicians, which are in turn more likely to be highly paid than municipal
technicians.
The same 2009 Department of Justice census of publicly funded forensic crime labs
used to identify local labs for the conviction analysis, above, provides similar evidence
for the end of the sample period. No data on education or experience is provided,
but each lab is asked to report the salary range for employees at various levels of the
organization. In the second panel of Table 5 reports the average of the midpoint of
these salary ranges for each lab control type and occupation. Throughout, employees
at county labs are paid more than their counterparts in state-controlled labs, and
the difference is statistically significant at the 5-percent level. In three of four cases
(all but the director), municipal lab employees are also paid more than state-lab
employees, although the difference is only statistically significant for the analyst and
supervisor positions.
4 Conclusions
The calls for forensic laboratory independence come from the natural suspicion of
excessive coziness between the police and those who provide scientific evidence at
21
Table 5: Education, Experience, and Pay by Control of Crime lab
Panel A: 1976 Forensic Science Foundation SurveyGrad. Degree 10+ Yrs. Exp. Evid. Tech. Sal.> $15k
Federal 42.3 27.0 n/aState 23.0 16.8 28.6County 34.3 26.7 47.4Municipal 14.5 29.6 10.0
Panel B: 2009 Census of Publicly Funded Crime LabsSalary Midpoint
Director Supervisor Senior Analyst AnalystFederal $132,911 $119,569 $94,723 $58,081State $90,878 $72,759 $62,536 $48,594County $100,698 $76,721 $65,871 $53,340Municipal $90,685 $77,148 $64,235 $52,175
trial. The most natural way to induce this separation would be to move the forensic
services from the control of the local officials, who handle much of the investigation
and prosecution of crimes, to a state office. Goldstein (2009), for example, argues that
the state, in particular, has strong incentives to improve forensic science outcomes.
But basic theory of the firm suggests that the decision to separate related activities
comes with both costs and benefits. This paper has provided the first systematic
evidence of the net consequences of combining or separating forensic services from the
local criminal justice organizations and that evidence suggests that calls for increasing
forensic lab independence might be misguided. In particular, I find that counties with
locally controlled crime labs have fewer exonerations and higher conviction rates.
The mechanism is not entirely clear, but there is some evidence that local labs invest
more in their personnel than state-controlled labs do–paying higher salaries and hiring
higher human
Beyond the specifics of crime labs, these results corroborate well with the broader
literature on decentralization and governmental responsiveness (Besley and Coate
2003, Faguet 2004, Rubinchik-Pessach 2005). If most of the gains from good con-
victions and losses from bad convictions are borne locally, the broader evidence on
public good provision squares well with our results–local provision dominates global
provision, even if local government is generally seen as “corrupt, institutionally weak,
or prone to interest-group capture.” Local labs invest more, since they capture more
22
of the benefits, and any pressure to bias results on close calls are offset by fewer close
calls.
References
Besley, Tim and Stephen Coate, “Centralized versus decentralized provision of
local public goods: a political economy approach,” Journal of Public Economics,
2003, pp. 2611–37.
Borchard, Edwin M., Convicting The Innocent, Yale University Press, 1932.
Committee on Identifying the Needs of the Forensic Sciences Community,
National Research Council, “Strengthening Forensic Science in the United
States: A Path Forward,” Technical Report, National Academy of Science 2009.
Faguet, Jean-Paul, “Does Decentralization Increase Government Responsiveness
to Local Needs? Evidence from Bolivia,” Journal of Public Economics, 2004,
pp. 867–93.
Field, Kenneth S. et al., “Assessment of the.Personnel of the Forensic Sciences
Profession,” Technical Report, National Institute of Law Enforcement and Crim-
inal Justice, Law Enforcement Assistance Agency, Department of Justice 1977.
Forensic Science Society, “World List of Forensic Science Laboratories,” Technical
Report 3rd Edition, Harrogate, North Yorkshite, England 1977.
Garrett, Brandon L. and Peter J. Neufeld, “Invalid Forensic Science Testimony
and Wrongfuul Convictions,” Virginia Law Review, 2009.
Giannelli, Paul C, “The Abuse of Evidence in Criminal Cases: The Need for In-
dependent Crime Laboratories,” Virginia Journal of Social Policy and the Law,
1997.
Gibbons, Robert, “Four Formal(izable) Theories of the Firm,” Journal of Economic
Behavior and Organization, 2005.
Goldstein, Ryan M., “Improving Forensic Science through State Oversight,” Texas
Law Review, 2009, pp. 225–58.
23
Gordon, Sanford and Gregory Huber, “The Political Economy of Prosecution,”
Annual Review of Law and Social Science, 2009.
Gross, S., “Convicting the Innocent,” Annual Review of Law and Social Science,
2008, 4, 173–92.
Gross, S, B O’Brien, C. Hu, and E. H. Kennedy, “Rate of false convic-
tion of criminal defendants who are sentenced to death,” Proceedings - National
Academy of Science, 2014, 111 (20), 7230–5.
Hampikian, Greg, Emily West, and Ogla Akselrod, “The Genetics of Inno-
cence: Analysis of 194 U.S. DNA Exonerations,” Annual Review of Genomics
and Human Genetics, 2011, 12, 97–120.
Jacoby, K et al., “Exonerations in the United States 1989 Through 2003.,” Journal
of Criminal Law and Criminology, 2005, 95 (2), 523–53.
Koppl, Roger, “How to Improve Forensic Science,” European Journal of Law and
Economics, 2005.
, “Organization Economics Explains Many Forensic Science Errors,” Journal of
Institutional Economics, 2010.
Lafontaine, Francine and Margaret Slade, “Vertical Integration and Firm
Boundaries: The Evidence,” Journal of Economic Literature, 2007.
McEwen, Tom, “The Role and Impact of Forensic Evidence in the Criminal Justice
System, Final Report,” Technical Report, Institute for Law and Justice, Inc.
2011.
Rasmusen, Eric Bennett, Manu Raghav, and J. Mark Ramseyer, “Convic-
tions versus Conviction Rates, The Prosecutor’s Choice,” American Law and
Economics Review, 2009.
Rubinchik-Pessach, Anna, “Can Decentralization Be Beneficial?,” Journal of Pub-
lic Economics, 2005.
United State Department of Justice–Bureau of Justice Statistics, “State
Court Processing Statistics: Felony Defendants in Large Urban Counties,” 2002-
2006.
24
, “Census of Publicly Funded Forensic Crime Laboratories,” 2009.
25