A Tale of (the Tales Told About) Two Expert Consensus Reports: Death Penalty & Gun Control - Cultural Cognition of Health

What is the expert consensus on whether the death penalty deters murders—or instead increases them through a cultural “brutalization effect”?

What is the expert consensus on whether permitting citizens to carry concealed handguns in the public increases homicide—or instead decreases it by discouraging violent predation?

According to the National Research Council, the research arm of the National Academy of Sciences, the expert consensus answer to these two questions is the same:

It’s just not possible to say, one way or the other .

Last April (way back in 2012), an expert NRC panel charged with determining whether the “available evidence provide[s] a reasonable basis for drawing conclusions” about the impact of the death penalty

concluded that research to date on the effect of capital punishment on homicide is not informative about whether capital punishment decreases, increases, or has no effect on homicide rates. Therefore, the committee recommends that these studies not be used to inform deliberations requiring judgments about the effect of the death penalty on homicide. Consequently, claims that research demonstrates that capital punishment decreases or increases the homicide rate by a specified amount or has no effect on the homicide rate should not influence policy judgments.

concluded that research to date on the effect of capital punishment on homicide is not informative about whether capital punishment decreases, increases, or has no effect on homicide rates. Therefore, the committee recommends that these studies not be used to inform deliberations requiring judgments about the effect of the death penalty on homicide. Consequently, claims that research demonstrates that capital punishment decreases or increases the homicide rate by a specified amount or has no effect on the homicide rate should not influence policy judgments.

Way way back in 2004 (surely new studies have come out since, right?), the expert panel assigned to assess the “strengths and limitations of the existing research and data on gun violence,”

found no credible evidence that the passage of right-to-carry laws decreases or increases violent crime, and there is almost no empirical evidence that the more than 80 prevention programs focused on gun-related violence have had any effect on children’s behavior, knowledge, attitudes, or beliefs about firearms. The committee found that the data available on these questions are too weak to support unambiguous conclusions or strong policy statements.

found no credible evidence that the passage of right-to-carry laws decreases or increases violent crime, and there is almost no empirical evidence that the more than 80 prevention programs focused on gun-related violence have had any effect on children’s behavior, knowledge, attitudes, or beliefs about firearms. The committee found that the data available on these questions are too weak to support unambiguous conclusions or strong policy statements.

The expert panels’ determinations, moreover, were based not primarily on the volume of data available on these questions but rather on what both panels saw as limitations inherent in the methods that criminologists have relied on in analyzing this evidence.

In both areas, this literature consists of multivariate regression models. As applied in this context, multivariate regression seeks to extract the causal impact of criminal laws by correlating differences in law with differences in crime rates “controlling for” the myriad other influences that could conceivably be contributing to variation in homicide across different places or within a single place over time.

Inevitably, such analyses involve judgment calls. They are models that, like  many statistical models, must make use of imprecise indicators of unobserved and unobservable influences, the relationship of which to one another must be specified based on a theory that is itself independent of any evidence in the model.

The problem, for both the death penalty and concealed-carry law regression studies, is that results come out differently depending on how one constructs the models.

“The specification of the death penalty variables in the panel models varies widely across the research and has been the focus of much debate,” the NRC capital punishment panel observed. “The research has demonstrated that different death penalty sanction variables, and different specifications of these variables, lead to very different deterrence estimates—negative and positive, large and small, both statistically significant and not statistically significant.”

That’s exactly the same problem that the panel charged with investigating concealed-carrry laws focused on:

The committee concludes that it is not possible to reach any scientifically supported conclusion because of (a) the sensitivity of the empirical results to seemingly minor changes in model specification, (b) a lack of robustness of the results to the inclusion of more recent years of data (during which there were many more law changes than in the earlier period), and (c) the statistical imprecision of the results.

The committee concludes that it is not possible to reach any scientifically supported conclusion because of (a) the sensitivity of the empirical results to seemingly minor changes in model specification, (b) a lack of robustness of the results to the inclusion of more recent years of data (during which there were many more law changes than in the earlier period), and (c) the statistical imprecision of the results.

This problem, both panels concluded, is intrinsic to the mode of analysis being employed. It can’t be cured with more data; it can only be made worse as one multiplies the number of choices that can be made about what to put in and what to leave out of the necessarily complex models that must be constructed to account for the interplay of all the potential influences involved.

“There is no empirical basis for choosing among these [model] specifications,” the NRC death penalty panel wrote.