follow CCP

Recent blog entries

Cultural vs. ideological extremists: the case of gun control

Look what those nut job socialists & libertarians are saying now: that if we really want to  reduce gun homicides—including the regular shooting of children on street corners in cities like Chicago—we should select one of the myriad sensible alternatives to our current "war on drugs," which predictably spawns violent competition to control a lucrative black market without doing much of anything to reduce either the supply or the demand for banned substances.

They just don’t get it!

So what if an expert consensus report from the National Academy of Sciences “found no credible evidence that the passage of right-to-carry laws decreases or increases violent crime.” Big deal that a Center for Disease Control task force “found insufficient evidence to determine the effectiveness of any of the firearms laws reviewed”—including waiting periods, ammunition bans, child access prevention laws, and “gun free school zones”—“for preventing violence.” 

Who cares that the best available evidence clearly suggests, in contrast, that there are myriad steps we could take (“wholesale legalization” vs. “wholesale criminalization” is a specious dichotomy) that would very appreciably reduce the number of homicides associated with the criminogenic property of our own drug-law enforcement policies?

The point isn’t to save lives! It’s to capture the expressive capital of the law.

Their role (real and fabled) in American history—in overthrowing tyranny and in perpetuating conditions of slavery and apartheid; in taming the frontier and in assassinating Presidents—have imbued guns with a rich surfeit of social meanings. Wholly apart, then, from the effect gun laws have (or don’t) on homicide, they convey messages that symbolically affirm and denigrate opposing cultural styles.

We are a liberal democratic society, comprising a plurality of diverse moral communities. The individual liberty provisions of our Constitution forbid the State to “enforce … on the whole society” standards of “private conduct” reflecting any one community’s “conceptions of right and acceptable behavior.”

So for crying out loud, how will we possibly be able to use State power to resolve whose way of life is virtuous and honorable and whose vicious and depraved if we don’t fixate on laws that have ambiguous public-welfare consequences but express unambiguously partisan cultural meanings?

What’s that? You say that the “war on drugs” should also be viewed as an exercise of expressive power aimed at enforcing a cultural orthodoxy?

Of course. But the partisan meanings that are expressed by those laws are ones that only “ideological extremists”—libertarians, socialists, et al.—would object to.


Center for Disease Control.First Reports Evaluating the Effectiveness of Strategies for Preventing Violence: Firearms Laws, Findings from the Task Force on Community Preventive Services (2003). 

Jacobs, J.B. Can gun control work? (Oxford University Press, Oxford ; New York; 2002).

Kahan, D.M.Cognitive Bias and the Constitution of the Liberal Republic of Science, working paper, available at

Kahan, D.M. The Cognitively Illiberal State. Stan. L. Rev. 60, 115-154 (2007).

Kahan, D.M. & Braman, D. More Statistics, Less Persuasion: A Cultural Theory of Gun-Risk Perceptions. U. Pa. L. Rev. 151, 1291-1327 (2003).

Kleiman, M. Marijuana : costs of abuse, costs of control. (Greenwood Press, New York; 1989).

Kleiman, M., Caulkins, J.P. & Hawken, A. Drugs and drug policy : what everyone needs to know. (Oxford University Press, Oxford ; New York; 2011).

MacCoun, R.J. & Reuter, P. Drug war heresies : learning from other vices, times, and places. (Cambridge University Press, Cambridge, U.K. ; New York; 2001).

Musto, D. F. (1987). The American Disease: Origins of Narcotic Control (Expanded ed.). New York: Oxford University Press.

National Research Council (U.S.). Committee to Improve Research Information and Data on Firearms., Wellford, C.F., Pepper, J., Petrie, C. & National Research Council (U.S.). Committee on Law and Justice. Firearms and violence : a critical review. (National Academies Press, Washington, DC; 2004). 




Are *positions* on the deterrent effect of the death penalty & gun control possible & justifiable? Of course!

So I started to answer one of the interesting comments in response to the last post & found myself convinced that the issues involved warranted their own post. So this one "supplements" & "adjusts" the last.

And by the way, I anticipate "supplementing" & "adjusting" everything I have ever said and ever will say.  If you don't see why that's the right attitude to have, then probably you aren't engaged in the same activity I am (which isn't to say that I plan to supplement & adjust every blog post w/ another; that's not the "activity" I mean to be involved in, but rather a symptom of something that perhaps I should worry about, and you too since you seem to be reading this).

Here's the question (from JB):

I'm puzzled about how the NRC dealt with Figure 2 in this paper, the "Canada graph" of Donohue and Wolfers. This is not multiple regression. (I agree that multiple regression is vastly over-used and that statistical control of the sort it attempts to do is much more difficult, if not impossible in many situations). But this graph settled the issue for me. It is not a regression analysis. . . .

Here's my answer:

@JB: The answer (to the question, what did NRC say about Fig. 2 in D&W) is . . . nothing  virtually nothing!

As you note, this is not the sort of multivariate regression analysis that the NRC's expert panel on the death penalty had in mind when it “recommend[ed] that these studies not be used to inform deliberations requiring judgments about the effect of death penalty on homicide.”

Your calling attention to this cool Figure furnishes me with an opportunity to supplement my post in a manner that (a) corrects a misimpression that it could easily have invited; and (b) makes a point that is just plain important, one I know you know but I want to be sure others who read my post do too.

The NRC reports are saying that a certain kind of analysis – the one that is afforded the highest level of respect by economists; that’s an issue that they really should talk about—is not valid in this context. In this context – deterrence of homicide by criminal law (whether gun control or capital punishment) --  these studies don’t give us any more or less reason to believe one thing or the other.

But that doesn’t mean that it is pointless to think about deterrence, or unjustifiable for us to have positions on it, when we are deliberating about criminal laws, including gun control & capital punishment! 

Two points:

First, just because one empirical method turns out to have a likelihood ratio of 1 doesn’t mean all forms of evidence have LR = 1!

You say, “hey, look at this simple comparison: our homicide rate & Candada’s are highly correlated notwithstanding how radically they differ in the use of the death penalty over time. That's pretty compelling!”

I think you would agree with me that that this evidence doesn’t literally “settle the issue.”  We know what people who would stand by their regression analyses (and others who merely wish those sorts of analyses could actually help) would say. Thinks like ... 

  • maybe the use of the death penalty is what kept the homicide rate in the US in “synch” with the Canadian one (i.e., w/o it, the U.S. rate would have accelerated relative to Canada, due to exogenous influences that differ in the 2 nations);
  • maybe when the death penalty isn’t or can’t be (b/c of constitutional probhition) used, legislators "make up the difference" by increasing the certainty of other, less severe punishments, and it is still the case that we can deter for "less" by adding capital punishment to the mix (after getting rid of all the cost-inflating, obstructionist litigation, of course);
  • maybe the death penalty work as James Fitzjames Stephen imagines – as a preference shaping device – and Canadians, b/c they watch so much U.S. TV are morally moulded by our culture (in effect, they are free riding on all our work to shape preferences through executing our citizens--outrageous);
  • variation in US homicide rates in response to the death penalty is too fine-grained to be picked  up by these data, which don’t rule out that the U.S. homicide rate would have decelerated in relation to Canada if the US had used capital punishment more frequently after Gregg;
  • the Donohue and Wolfers chart excludes hockey-related deaths resulting from player brawls and errant slapshots that careen lethally into the stands, and thus grossly understates the homicide rate in Canada (compare how few players and fans have been killed by baseball since Gregg!);
  • etc. etc. etc.   

These are perfectly legitimate points, I’d say. But what is the upshot?

They certainly don’t mean that evidence of the sort reflected in Fig. 2 is entitled to no weight – that its "Likelihood Ratio = 1."  If someone thinks that that’s how empirical proof works – that evidence either “proves” something “conclusively,” or “proves nothing, because it hasn’t ruled out all alternative explanations”—is “empirical-science illiterate” (we need a measure for this!).

These points just present us with reasons to understand why the data in Fig. 2 don't mean LR ≠ ε (if the hypothesis is “death penalty deter”; if hypothesis is “death penalty doesn’t,” then why LR ≠ ∞).

I agree with you that Fig 2 has a pretty healthy LR – say, 0.2, if the hypothesis is “the death penalty deters” – which is to say, that that I believe the correlation between U.S. and Canadian homicide rates is “5 times more consistent with” the alternative hypothesis (“doesn’t deter”).

And , of course, this way of talking is all just a stylized way of representing how to think about this—I’m using the statistical concept of “likelihood ratio” & Bayesianism as a heuristic. I have no idea what the LR really is, and I haven’t just multiplied my “priors” by it.

But I do have an idea (a conviction, in fact) about the sensible way to make sense of empirical evidence. It's that it should be evaluated not as "proving" things but as supplying more or less reason to believe one thing or another. So when one is presented with empirical evidence, one shouldn't say either "yes, game over!" or "pfffff ... what about this that & the other thing..." but rather should supplement & adjust what one believes, and how confidently, after reflecting on the evidence for a long enough time to truly understand why it supports a particular infernece and how strongly.

Second, even when we recognize that an empirical proposition relevant to a policy matter admits of competing, plausible  conjectures (they don't have to be “equally plausible”; only an idiot says that the “most plausible thing must be true!”), and that it would be really really nice to have more evidence w/ LR ≠ 1, we still have to do something.  And we can and should use our best judgment about what the truth is, informed by all the “valid” evidence (LR ≠ 1) we can lay our hands on.

I think people can have justifiable beliefs about the impact (or lack thereof) of gun control laws & the death penalty on homicide rates!

They just shouldn't abuse reason. 

They do that when they insist that bad statistical proofs -- simplistic ones ones that just toss out arbitrary bits of raw data; or arbitrarily complex yet grossly undertheorized ones like "y =b1*x1+ b2*x2 +b3*x3 ... +b75*x34^3 + ..." – “conclusively refute” or “demonstrably establish” blah blah blah.

And they do that and something even worse when they mischaracterize the best scientific evidence we do have.


A Tale of (the Tales Told About) Two Expert Consensus Reports: Death Penalty & Gun Control

What is the expert consensus on whether the death penalty deters murders—or instead increases them through a cultural “brutalization effect”?

What is the expert consensus on whether permitting citizens to carry concealed handguns in the public increases homicide—or instead decreases it by discouraging violent predation?

According to the National Research Council, the research arm of the National Academy of Sciences, the expert consensus answer to these two questions is the same:

It’s just not possible to say, one way or the other.

Last April (way back in 2012), an expert NRC panel charged with determining whether the “available evidence provide[s] a reasonable basis for drawing conclusions” about the impact of the death penalty

concluded that research to date on the effect of capital punishment on homicide is not informative about whether capital punishment decreases, increases, or has no effect on homicide rates. Therefore, the committee recommends that these studies not be used to inform deliberations requiring judgments about the effect of the death penalty on homicide. Consequently, claims that research demonstrates that capital punishment decreases or increases the homicide rate by a specified amount or has no effect on the homicide rate should not influence policy judgments.

Way way back in 2004 (surely new studies have come out since, right?), the expert panel assigned to assess the “strengths and limitations of the existing research and data on gun violence,”

found no credible evidence that the passage of right-to-carry laws decreases or increases violent crime, and there is almost no empirical evidence that the more than 80 prevention programs focused on gun-related violence have had any effect on children’s behavior, knowledge, attitudes, or beliefs about firearms. The committee found that the data available on these questions are too weak to support unambiguous conclusions or strong policy statements.

The expert panels’ determinations, moreover, were based not primarily on the volume of data available on these questions but rather on what both panels saw as limitations inherent in the methods that criminologists have relied on in analyzing this evidence. 

In both areas, this literature consists of multivariate regression models. As applied in this context, multivariate regression seeks to extract the causal impact of criminal laws by correlating differences in law with differences in crime rates “controlling for” the myriad other influences that could conceivably be contributing to variation in homicide across different places or within a single place over time. 

Inevitably, such analyses involve judgment calls. They are models that, like  many statistical models, must make use of imprecise indicators of unobserved and unobservable influences, the relationship of which to one another must be specified based on a theory that is itself independent of any evidence in the model.

The problem, for both the death penalty and concealed-carry law regression studies, is that results come out differently depending on how one constructs the models.

“The specification of the death penalty variables in the panel models varies widely across the research and has been the focus of much debate,” the NRC capital punishment panel observed. “The research has demonstrated that different death penalty sanction variables, and different specifications of these variables, lead to very different deterrence estimates—negative and positive, large and small, both statistically significant and not statistically significant."

That’s exactly the same problem that the panel charged with investigating concealed-carrry laws focused on:

The committee concludes that it is not possible to reach any scientifically supported conclusion because of (a) the sensitivity of the empirical results to seemingly minor changes in model specification, (b) a lack of robustness of the results to the inclusion of more recent years of data (during which there were many more law changes than in the earlier period), and (c) the statistical imprecision of the results.

This problem, both panels concluded, is intrinsic to the mode of analysis being employed. It can’t be cured with more data; it can only be made worse as one multiplies the number of choices that can be made about what to put in and what to leave out of the necessarily complex models that must be constructed to account for the interplay of all the potential influences involved.

“There is no empirical basis for choosing among these [model] specifications,” the NRC death penalty panel wrote.

[T]here has been heated debate among researchers about them.... This debate, however, is not based on clear and principled arguments as to why the probability timing that is used corresponds to the objective probability of execution, or, even more importantly, to criminal perceptions of that probability. Instead, researchers have constructed ad hoc measures of criminal perceptions. . . .

Even if the research and data collection initiatives discussed in this chapter are ultimately successful, research in both literatures share a common characteristic of invoking strong, often unverifiable, assumptions in order to provide point estimates of the effect of capital punishment on homicides.

The NRC gun panel said the same thing:

It is also the committee’s view that additional analysis along the lines of the current literature is unlikely to yield results that will persuasively demonstrate a causal link between right-to-carry laws and crime rates (unless substantial numbers of states were to adopt or repeal right-to-carry laws), because of the sensitivity of the results to model specification. Furthermore, the usefulness of future crime data for studying the effects of right-to-carry laws will decrease as the time elapsed since enactment of the laws increases. If further headway is to be made on this question, new analytical approaches and data sets will need to be used.

So to be sure, the NRC  reached its “no credible evidence" conclusion  on right-to-carry laws way back in 2004. But its conclusion was based on “the complex methodological problems inherent in” regression analysis--the same methodological problem that were the basis of the NRC’s 2012 conclusion that death penalty studies are "not informative" and "should not influence policy judgments."

Nothing's changed on that score. The experts at the National Academy of Sciences either are right or they are wrong to treat multivariate regression analysis as an invalid basis for inference about the effects of criminal law.

The reasoning here is all pretty basic, pretty simple, something that any educated, motivated person could figure out by sitting down with the reports for a few hours (& who wouldn't want to do that?!).

Yet all of this has clearly evaded the understanding of many extremely intelligent, extremely influential participants in our national political conversation.

I’ll pick on the New York Times, not because it is worse than anyone else but because it’s the newspaper I happen to read everyday.

Just the day before yesterday, it said this in an editorial about the NRC’s capital punishment report:

A distinguished committee of scholars convened by the National Research Council found that there is no useful evidence to determine if the death penalty deters serious crimes. Many first-rate scholars have tried to prove the theory of deterrence, but that research “is not informative about whether capital punishment increases, decreases, or has no effect on homicide rates,” the committee said.

Okay, that’s right. 

But here is what the Times’ editorial page editor said the week before last about concealed carry laws:

Of the many specious arguments against gun control, perhaps the most ridiculous is that what we really need is the opposite: more guns, in the hands of more people, in more places. If people were packing heat in the movies, at workplaces, in shopping malls and in schools, they could just pop up and shoot the assailant. . . . I see it differently: About the only thing more terrifying than a lone gunman firing into a classroom or a crowded movie theater is a half a dozen more gunmen leaping around firing their pistols at the killer, which is to say really at each other and every bystander. It’s a police officer’s nightmare. . . . While other advanced countries have imposed gun control laws, America has conducted a natural experiment in what happens when a society has as many guns as people. The results are in, and they’re not counterintuitive.

Wait a sec.... What about the NRC report? Didn’t it tell us that the “results are in” and that "it is not possible to reach any scientifically supported conclusionon whether concealed carry laws increase or decrease crime?

I know the New York Times is aware of the NRC’s expert consensus report on gun violence. It referred to the report in an editorial just a couple days earlier.

In that one, it called on Congress to enact a national law that would require the 35 states that now have permissive “shall issue” laws—ones that mandate officials approve the application of any person who doesn’t have a criminal record or history of mental illness—to “set higher standards for granting permits for concealed weapons.”  “Among the arguments advanced for these irresponsible statutes,” it observed,

is the claim that ‘shall issue’ laws have played a major role in reducing violent crime. But the National Research Council has thoroughly discredited this argument for analytical errors. In fact, the legal scholar John Donohue III and others have found that from 1977 to 2006, ‘shall issue’ laws increased aggravated assaults by “roughly 3 to 5 percent each year.


Yes, the NRC concluded that there was “no credible evidence” that concealed carry laws reduce crime.

But as I pointed out, what it said was that it “found no credible evidence that the passage of right-to-carry laws decreases or increases violent crime.” So why shouldn't we view the Report as also “thoroughly discrediting” the Times editorial’s conclusion that those laws“seem almost designed to encourage violence?”

And, yes, the NRC can be said (allowing for loose translation of more precise and measured language) to have found “analytical errors” in the studies that purported to show shall issue laws reduce crime. 

But those “analytical errors,” as I’ve pointed out, involve the use of multivariate regression analysis to try to figure out the impact of concealed carry laws. That’s precisely the sort of analysis used in the Donohue study that the Times identifies as finding shall issue laws increased violent crime. 

The “analytical errors” that the Times refers to are inherent in the use of multivariate regression analysis to try to understand the impact criminal laws on homicide rates. 

That’s why the NRC’s 2012 death penalty report said that findings based on this methodology are “not informative” and “should be ignored for policy analysis.”

The Times, as I said, got that point. But only when it was being made about studies that show the death penalty deters murder, and not when it was being made about studies that find concealed carry laws increase crime....

This post is not about concealed carry laws (my state has one; I wish it didn’t) or the death penalty (I think it is awful).

It is about the obligation of opinion leaders not to degrade the value of scientific evidence as a form of currency in our public deliberations.

In an experimental study, the CCP found that citizens of diverse cultural outlooks all believe that “scientific consensus” is consistent with the position that predominates within their group on climate change, concealed carry laws, and nuclear power.  Members of all groups were correct – 33% of the time.

How do ordinary people (ones like you & me, included) become so thoroughly confused about these things?

The answer, in part, is that they are putting their trust in authoritative sources of information—opinion leaders—who furnish them with a distorted, misleading picture of what the best available scientific evidence really is.

The Times, very appropriately, has published articles that attack the NRA for seeking to block federal funding of the scientific study of firearms and homicide.  Let’s not mince words: obstructing scientific investigation aimed at promoting society’s collective well-being is a crime in the Liberal Republic of Science.

But so is presenting an opportunistically distorted picture of what the state of that evidence really is.

The harm that such behavior causes, moreover, isn’t limited to the confusion that such a practice creates in people who (like me!) rely on opinion leaders to tell us what scientists really believe.

It includes as well the cynicism it breeds about whether claims about scientific consensus mean anything at all.  One day someone is bashing his or her opponents over the head for disputing or distorting “scientific consensus”—and the next day that same someone can be shown (incontrovertibly and easily) to be ignoring or distorting it too.

By the way, John Donohue is a great scholar, one of the greatest empirical analysts of public policy ever.

Both of the NRC expert consensus reports that I’ve cited conclude that studies he and other econometricians have done are “not informative” for policy because of what those reports view as insuperable methodological problems with multivariate analysis as a tool for understanding the impact of law on crime.

Donohue disagrees, and continues to write papers reanalyzing the data that the NRC (in its firearms study) said are inherently inconclusive because of "complex methodological problems" inherent in the statistical techniques that Donohue used, and continues to use, to analyze them.

But that’s okay.

You know what one calls a scientist who disputes “scientific consensus”?

A scientist.

But that’s for another day. 


Chewing the fat, so to speak...

I've already exhausted my allotted time for blogging in answering interesting comments related to the post on Silver's climate change wisdom. I invite others to weigh in (but not on whether Mann is a great climate scientist; see my post update on that).

In particular, I'd like help (Larry has provided a ton, but I'm greedy) on what is right/wrong/incisive/incomplete/provocative/troubling/paradoxical/inherently contradictory etc. about my statement, "Gaps between prediction and reality are not evidence of a deficiency in method. They are just evidence--information that is reprocessed as part of the method of generating increasingly precise and accurate probabilistic estimates." Also the questions of (a) how forecasting model imprecision or imperfection should affect policymaking proposals & even more interesting (given the orientation of this blog) (b) how to communicate or talk about this practical dilemma. (Contributions should be added to that comment thread.)

Two more things to think about, complements of Maggie Wittlin:

1. Who is afraid of obesity & why?  Maggie notes "new meta-analysis finds that overweight people (and, with less confidence, people with grade 1 obesity) have a lower risk of mortality than people with BMIs in the 'normal' range" and wonders, as do I, how cultural outlooks or other sources of motivated reasoning affect reactions to evidence like this -- or of the health consequences of obesity generally.

2. Forget terrorism; we're all going to die from an asteroid. Maggie also puts my anxiety about magnitude 7-8-9 terrorism into context by pointing out that the size/energy-releasing-potential of asteroid impacts on earth also follow a power-law distribution.  Given the impact (so to speak) of civilization-destroying asteroid collision, isn't preparing to protect earth from such a fate (however improbable) yet another thing that we need to do but are being distracted from doing by OHS's rules on removing shoes at airport security-screening stations?! I could do some research but Aaron Clauset's spontaneous & generous supply of references for the likelihood of "large" terrorism attacks makes me hope that some other generous person who knows the literature here will point us to useful sources.


Wisdom from Silver’s Signal & Noise, part 2: Climate change & the political perils of forecasting maturation

This is post 2 in my three part series on Silver’s Signal & Noise, which tied for first (with  Sharon Bertsch McGrayne’s The Theory That Would Not Die) in my “personal book of the year” contest (I’ve already mailed them both the quantity of gold bullion that I always award to the winner—I didn’t even divide it in half; or maybe I did, or possibly I even doubled or tripled it).

It turns out that Silver is not only amazingly good at statistical modeling & pretty decent at story telling. He also happens to be pretty wise (obviously this is a limited sample & I’ll update based on new information etc).

The nugget of wisdom I mined out of the book in the first post had to do with Silver’s idea that we should treat terrorist attacks a bit more like earthquakes.

This time I want to make a report on what Silver had to say about climate-change forecasting. One way to understand his assessment is that the practitioners of it are being punished for their methodological virtue. 

Silver essentially structures the book around prototypes. There’s baseball, which is to forecasting what Saudi Arabia is to oil drilling. There are elections, another data-rich field but one that gets screwed up by a combination of bad traits in those who prognosticate (they are full of themselves) and those who are consuming their prognostications (too many of them want to be told only what they want to hear).

And earthquakes—can’t be forecast, but can still yield lots of info.

And economics--a bastion of bad statistics hygiene.

Then there’s meteorology, which is the archetype prototype of forecasting excellence because it is super hard and yet has made measureable progress (that’s much higher praise than “immeasurable,” in this context) due to the purity and discipline of its practitioners. (I’m eager to see who gets to play Richard Loft, the director of Technology Development at NCAR in the upcoming movie adaptation of Signal; I’m guessing Pierce Brosnan, unless he is cast as Silver himself).

In Silver’s account, climate forecasting is traveling the path of meteorology. The problem is that emulating the meteorologists obliges climate forecasters to become unwitting manufacturers of the ammunition being directed against them in the political flack storm surrounding climate change.

One of the things that meteorology forecasters did that makes them the superheroes of Signal was calibration. They not only made prodigious predictions but then revisited and retooled their models in light of how close they came to their targets, thereby progressively improving their aim.

When climate forecasters do this—as they must—they leave themselves wide open to guerilla attack by those seeking to repel the advance of science. The reason is that error is an inevitable and indeed vitally productive element of the Bayesian-evolutionary process that characterizes the maturation of valid forecasting.

Gaps between prediction and reality are not evidence of a deficiency in method. They are just evidence, information that is reprocessed as part of the method of generating increasingly precise and accurate probabilistic estimates.

This is a subtle point to get across even if one is trying to help someone to actually understand how science works. But for those who are trying to confuse, the foreseeable generation of incorrect predictions furnishes a steady supply of resources with which to harass and embarrass and discredit earnest scientists.

Silver recounts this dilemma in explicating the plight of James Hansen, whose forecasts from 30 and 25 years were in many respects impressively good but just as importantly instructively wrong. Ditto for the IPCC’s 1990 predictions.

Another thing that the superhero meteorologists did right was, in effect, theorize. They enriched their data with scientific knowledge that enabled them to do things like create amazing simulations of the dynamics they were trying to make predictive sense of. As a result, they got a lot further than they would have if they had used brute statistical force alone.

Climate forecasters are doing this too, and as a result necessarily enlarging the target that they offer for political sniping.  The reason is that theory-informed modeling of dynamic systems is hard work, the payoffs of which are unlikely to accumulate steadily in a linear fashion but rather to accrue in incremental breakthroughs punctuated by periods of nothing.

Indeed, those who travel this path might well seem to be make slower progress at least temporarily than those who settle for simpler, undertheorized number-crunching strategies, which make fewer assumptions and thus expose themselves to fewer sources of error, which tend to compound within dynamic models. Silver notes, for example, that some of Hansen’s earlier predictions—which were in the nature of simple multivariate regressions—in some respects outperformed some of his subsequent, dynamic-simulation driven ones.

Again, then, the virtuous forecaster will, precisely as a result of being virtuous, find him- or herself vulnerable to opportunistic hectoring, particularly by anti-science, lawyerly critics who will adroitly collect and construct number-crunching models that generated more conservative predictions and thereby outperformed the more theoretically dynamic ones over particular periods of time (including ones defined by happenstance or design to capitalize on inevitable and inevitably noisy short-term fluctuations in things like global temperatures).

Silver mentions the work of Scott Armstrong, a serious forecaster who nevertheless confines himself to simple number-crunching and consciously eschews the sort of theory-driven enrichment that was the signature of meteorology’s advancement. “I actually try not to learn a lot about climate change,” Armstrong, who is famous for his “no change” forecast with respect to global temperatures, boasts. “I am a forecasting guy” (Signal, p. 403).

“This book advises you to be wary of forecasters who say that science is not very important to their jobs,” Silver writes, just as it advises us to be skeptical toward “scientists who say that forecasting is not important to their[s] . . . . What distinguishes science, and what makes a forecast scientific, is that it is concerned with the objective world. What makes forecasts fail is when our concern only extends as far as the method, maxim, or model” (p. 403).

For Silver, the basic reason to “believe” in—and be plenty concerned about—climate change is the basic scientific fact, disputed by no one of any seriousness, that increasing concentrations of atmospheric CO2 (also not doubted by anyone) conduce to increasing global temperatures, which in turn have a significant impact on the environment. Forecasting is less a test of that than a vital tool to help us understand the consequences of this fact, and to gauge the efficacy (including costs and benefits) of potential responses.

Seems right to me. Indeed, seems wise.

* * * *

Okay, here’s something else that I feel I ought to say.

One reason I was actually pretty excited to get to the climate forecasting chapter was to verify an extremely critical review of the book (issued well before the release date of it) by Michael Mann, climate scientist of “hockey stick” fame.

Frankly, I find the gap between Mann’s depiction and the reality of what Silver said disturbing. You’d get the impression from reading Mann’s review that Silver is a “Chicago School” “free market fundamentalist” who dogmatically attacks the assumptions and methods of climate forecasters.

Just not so. I’m mean really really really untrue.

Mann figures very briefly at the end of the chapter, where Silver reports Mann’s reaction to what is in fact the chapter's central theme—that climate forecasting is exposed to political perils precisely because those engaged in it are taking an uncompromisingly scientific approach.

Mann is obviously—understandably and justifiably!—frustrated and filled with anger.

He describes climate scientists themselves as being involved in a “street fight with these people”—i.e., the professional “skeptics” who hector and harass, distort and mislead (p. 409).

Of course, that’s a response that sees fighting as something climate scientists ought to be doing.

“It would be irresponsible for us as a community to not be speaking out,” Mann explains.

“Where you have to draw the line is to be very clear about where the uncertainties are,” he allows, but it would be a mistake to “have our statements so laden in uncertainty that no one even listens to what we’re saying.”

Silver doesn’t say this—indeed, had no reason to at the time he wrote the book—but I have to wonder whether Mann’s savage reaction to Silver is part of Mann’s “street fighting” posture, which apparently includes attacking even intellectually and emotionally sympathetic commentators whose excessive reflection on climate forecasting “uncertainty”  threatens to prevent the public from even “listen[ing] to what we’re saying.”

Mann is a great climate scientist. He is not a scientist of science communication.

For those who do study and reflect on science communication, whether simplifying things or dispensing with qualifications (not to mention outright effacing of complexity) will promote open-minded public engagement with climate science are matters characterized by uncertainties analogous to the ones that climate change forecasters deal with.

But I think one thing that admits of no uncertainty is that neither climate scientists nor scientists of science communication nor any other scientifically minded person should resort to simplification, effacement of complexity, and disregard for intellectual subtlety in describing the thoughtful reflections of a scholarly minded person who is trying to engage openly and candidly with complicated issues for the benefit of curious people.

That’s a moral issue, not an empirical one, and it goes to the nature of what the enterprise of scholarly discussion is all about.


Nature Climate Change study on science literacy & cultural polarization can now be downloaded from CCP site

We not only post "preprints" & "working papers, but also the published versions of our studies too so long as consistent with copyright & like agreements.

Nature journals ("mother Nature" plus its brood of baby Natures, such as Nature Climate Change, Nature Nanotechnology, Nature Biotechnolgy, and Nature Natural Law Review) usually permit authors to post in online repositories final versions of their published articles 6 mos. after they appear but only in a non-typeset form (you know, not a simple mimeograph or xerox of the article as it appeared in the journal).

Well, it is now possible to download a non-typeset version of the CCP study on science literacy (numeracy, too) & cultural polarization of perceived climate change risks.

Just click here!


Wisdom from Silver’s Signal & Noise, part 1: “Predicting” magnitude 7-8-9 terrorist attacks

So I finished Nate Silver’s Signal & the Noise a couple days ago.  I loved it! 

n fact, it managed, very unexpectedly, to sneak in as a late entrant and catch what looked like the sure winner for “my personal favorite book of the year,” Sharon Bertsch McGrayne’s The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy. The two ended up in a dead heat, I’d say  (super honorable mention to George Dyson’s Turing’s Cathedral & James Gleick’s The Information).

McGrayne is the better story teller. Silver's not bad in that regard but the reason he managed to score so high is that his book contains some genuine elements of wisdom. Or at least I thought so!

Over the course of three posts (I’m guessing), I’m going to feature three of the nuggets of insight that I spotted in the book.

The first has to do with the important parallels between earthquakes and terrorist attacks.

As likely everyone—whether or not they’ve read the book—knows, Signal is about the simultaneous indispensability and fallibility of data-driven forecasting. The book essentially uses certain cases as prototypes for what forecasting can and can’t do, when it does and doesn’t work, and why.

Baseball, e.g., works well because there are so much data and the outcomes of interest so straightforward.

Economic forecasting is absurdly bad, in large part because of the bad statistical hygiene of economists, who are theoretically dogmatic, enamored of post-hoc story-telling (they use a statistical tool for this: “overfitting”), and uninterested in iterative calibration of their models.

Another prototype is forecasting of earthquake occurrence.  Pretty much can’t be done: there aren’t enough events to form and test models; the underlying dynamics are likely chaotic—if not technically so, then practically, given the high number of interacting mechanisms; and measurement of those mechanisms is confined to exceedingly crude proxies.

But the kind of forecasting that can’t be done has to do with predicting the timing of earthquakes, particularly ones of significant magnitude.

What can be predicted, however, is the likely scale and frequency of earthquakes in particular locales.

The reason that can be done (and actually, one of the reasons that precise forecasting of timing probably can’t be) is that earthquake magnitude and frequency reflect a “power law” distribution.

In a power-law distribution, there will be a predictable incidence of extreme values—ones many more standard deviations from the mean than you’d ever expect to see in a normal, Gaussian or “bell-shaped” distribution. The classic illustration is height, which is normally distributed, vs. wealth, which reflects a power-law distribution: you’ll never find even one person 20 SDs taller than average (10.5', or so), whereas people who are 20 SDs wealthier than average (net worth of, oh, $2 billion,) or even 100 SDs ($10 billion, more or less), while not commonplace, definitely are out there (for this reason, it is not very meaningful to talk about "standard deviations" in wealth).

Well, if you have enough data to estimate the key parameters of a power law, you can form a reasonably precise estimate of the frequency of extreme values relative to more typical, more common-place ones.

One can do this for earthquakes, it turns out. The Gutenberg-Richter law characterizes the power-law distribution for earthquakes and makes it possible, based on the frequency of relatively “mild” earthquakes to conclude that San Francisco can be expected to have a “major” one (> 6.75 on Richter scale) once ever 30 years, whereas New York will have a major one on average, once every 12,000 years (Signal, p. 150).

This is useful information. It doesn’t tell you when you are likely to have a major earth quake. But it tells you that if you live in San Francisco you are an idiot not to plan for one, whereas if you live in New York, you’d likely be out of your mind to insist that every building be able to able to handle the same impact.

Okay, so Silver goes through all of this. But later in the book he uses this information to say something pretty insightful (to me!) about terrorism.

Terrorist incidents, Silver observes, reflect a power law distribution, too (pp. 430-31). From the number of relatively “small” incidents in a given locale, one can estimate, with a reasonable degree of precision, the expected frequency of progressively “larger” ones.

Could 9/11 have been “predicted”?  As to when & where—probably not (Silver hedges on this; I think he is guilty of a bit of hindsight bias). But as to whether? For sure.

Applying a power-law model with data compiled by Aaron Clauset, Silver concludes that, as of Sept. 10, 2001,  “a September 11-scale attack” could have been expected to occur on average “about once every eighty years in a NATO country” (p. 432), which (by my math) comes out to over a 10% chance in a given decade.

Updating the power-law estimate to incorporate this the new information associated with the occurrence of 9/11 itself, Silver reports that we now have reason to expect a 9/11-scale attack in a NATO country once every 40 years. That’s close to a 25% chance every decade (p. 432).

He also now estimates that there is a “10 percent chance of an attack that would kill at least 10,000 people in a NATO country,” a “3 percent change of an attack that would kill 100,000, and a 0.6 percent chance of one that would kill one million or more” in the next decade (p. 437).

The estimates, like any ones generated by Bayesian techniques, are provisional, and here are appropriately qualified in light of the relative spareness of the data and also various judgment calls about how to define the relevant class of events.

But the point is we know more with this information than we would without it.  Just as the Gutenberg-Richter law supplies information that can help determine the appropriate level of preparedness for earthquakes, this Clauset-Silver terrorism power law supplies us with information we can use to engage in more rational planning for terrorism.

There will (predictably) be many many many times more smaller terrorist incidents than “big” ones—just as there will be many many times more minor than major earthquakes.

But for both quakes as for terrorist attacks, the destructive energy of progressively “larger” ones increases exponentially relative to their diminishing frequency.

We might expect, say, 10 magnitude 6 quakes for every magnitude 7, but the magnitude 7 quake (according to Silver, pp. 437-38) releases 1,000x more energy than each of the 6’s and in the absence of appropriate preparation cause much more damage than all the 6’s combined.

Similarly, in NATO countries, there were scores of terrorist acts smaller than 9/11—a “magnitude 7” attack--in the three decades that led up to it, but the 9/11 death toll was higher than that for all of those combined (p. 438). 

If you have limited resources, it makes sense to invest them in the way that minimizes the expected harm by as great an amount as possible.

San Francisco should be devoting much more resources to bracing for the magnitude 7 quake than it does trying to prepare for the damage associated with the much more frequent but much less damaging smaller ones.

We devote a huge amount of resources to trying to deter or preempt small-scale attacks. Fine. But a blown up commuter-rail train or subway station will likely kill only a few dozen people, and even the downing of a commercial airliner only a few hundred. 

A “magnitude 8” or even “9”-- the release of a chemical or biological agent or the detonation of a nuclear device in an urban area--  has the potential to kill 10,000s or even 100,000s and, while much less likely seems well within the range of power-law range of possibility.

Certainly we should—and I’m sure are-- doing a lot to try to avert the latter sort of attack.  But trying to detect the precursors of such a thing—so that we can intervene and avert it—is very difficult; it is a lot like trying to predict when a magnitude 7 or 8 or 9 earthquake is going to occur.

Nevertheless, we should be preparing for a magnitude 7 or 8 or 9 terrorist attack, in the same way that we do prepare for magnitude 7 and above earthquakes in the places where they can be expected to occur. Cities like San Francisco mandate that structures be built to withstand major quakes and as a result only scores of people die in ones that kill 10,000s in countries like Iran and Armenia.

A civil preparedness capacity to respond to a magnitude 8 or 9 terrorist attack could likewise make a difference in orders of magnitude in the number of people killed by it.

We don’t have such a regime.  Relatively little of the money that the Department of Homeland Security has doled out since its inception have been used for large-scale crisis-response planning.  Moreover, major cities like Los Angeles and New York make what many would regard as only half-hearted efforts to prepare their emergency-response capacities, in part because things like mock evacuations and similar drills tend to be viewed as disruptive and also anxiety-provoking.

Instead, as Silver notes, the post-911 world is pervaded by various forms of “security theatre”— cumbersome screening procedures and goofy alert systems that gratify a public demand for action but are unlikely to contribute anything to public safety.

After the (completely inept) “underwear bomber” was foiled in Christmas 2009, President Obama responded to the media frenzy by upbraiding our intelligence services for “failing to connect the dots,” and “order[ed] an immediate effort to strengthen the criteria used to add individuals to our terrorist watchlists, especially the  ‘no-fly’ list”—a device that experts view as too bloated to be effective and as imposing needless burdens on travelers and security personnel alike.

A guy as smart as Obama doesn't actually need Nate Silver to tell him that you don’t try to improve your ability to pick up a signal by adding noise.

Rather, what he and the rest of us need is a science of science communication that would make as as smart about managing the political-psychology dynamics of risk perception as Silver is about data-driven forecasting.  If we lack the former, the substantial contribution the latter can make to securing the public welfare is wasted.


The black and white -- but mainly gray -- of gun control and drug prohibition, part 2

So ... this is actually part two of a two-part series on race, gun control, and drug criminalization.

Last time I went into the motivation behind the series. The more proximate cause was a question posed in the discussion of my post on legalizing drugs. 

The more remote—but more fundamental—was the unsettling sense I had upon reflecting on my feelings on the Newtown shooting.  The shooting upset me (as it did many, of course). But it upset me, too, to realize that I’m not that upset more or less continuously, because in fact young kids are being shot more or less continuously—not in elementary schools in communities like Newtown, but on street corners & playgrounds in cities like Chicago, Los Angeles and Detroit.

They are essentially part of the War on Drug’s “collateral damage.” And I guess in the same way we don’t worry overmuch about “collateral damage” in the form of deaths to civilians in our other wars, we don’t really get distracted by it here at home. . . .

However much progress one thinks can be made by laws restricting firearms, orders of magnitude more can be made by ending drug Prohibition, which like alcohol Prohibition combines opportunities for monopoly profits with the necessity for violent, extralegal enforcement of commercial obligations, creating a hyper-homicide cocktail potent enough to bend the historical curve of pacifism that is the signature of liberal markety societies.

There are some unconscious emotional dynamics at work here that ought to be exposed and critically examined. They include the partisan cultural meanings of recreational drugs that account for their being treated differently from alcohol.  They include, too, the predictably parochial nature of our empathic sensibilities, which lead us to attend selectively to forms of suffering and loss that are in fact universal in their nature and concentrated disproportionately on certain members of our society.

But as always, matters are complicated.

One of the complexities is the profound moral ambivalence that members of inner-city, largely minority communities themselves have toward drug criminalization.  That was the subject of the my last post, which described the powerful case that members of those communities often make for their entitlement to use the expressive power of the law as a countervailing force to oppose social influences—including commercial advertising of lawful products--that they believe unfairly diminish their power to instill in their children the dispositions, tastes, and habits that they (like all parents everywhere) perceive to be essential to their children’s’ prospects for living happy, flourishing lives.

Precisely because the impact of our policies on guns and drugs so profoundly and distinctively affect these communities, the power of their members to make law in a manner that relfects their judgments about the morally complicated, empirically uncertain issues involves necessarily goes straight to the core of their entitlement to enjoy meaningful self-government.

I don’t know what this element of the dilemma means for how political disputes over guns and over drugs should be resolved. If I said I did, I’d be contradicting myself (likely I already have in some place and in some way connected to this topic; only someone who doesn’t actually get the complexity involved will manage to get all the pieces to fit together in an analytically neat and tidy little pattern).  My point is just that this is an element of the dilemma that has to be taken into account; and for that reason, I want to take it into account by acknowledging that it complicates the argument I have been making about the mistaken overemphasis on regulating guns relative to deregulating drugs as a response to our Nation’s unacceptable gun-homicide rate. . . .

So, finally, some data (I’m sorry for going on; I should just accept that it is impossible ever to say enough and not keep trying).

It’s not a perfect measure of the ambivalence that the communities I’m describing feel, but what I’ve done, combining data from several CCP studies conducted over the last year, is break down attitudes on gun control and marijuana legalization by race. 

The sample consists of about 2500 individuals drawn from a nationally representative on-line sample of individuals recruited to participate in CCP studies of one sort or another.  I wanted to cobble together several data sets in order to get enough African-American sample to enable reasonably precise estimates; African-Americans make up 11% of the sample overall.

Start with this:

It shows that African Americans and whites in the sample held comparable attitudes overall toward marijuana legalization—about 60% support, actually—but were divided sharply on gun control, with African Americans decidedly more supportive. 

I’ve put Hispanics and Asians in, too, but realize that they (especially Asians) comprised an even smaller part of the sample, diminishing the precision of any estimate of the attitudes of these groups in the general population (the error bars are standard errors; double the interval, essentially, if you want 0.95 levels of confidence).  But we are on relatively solid ground for African Americans and whites.

Now to do a meaningful comparison of African American and whites, it is also useful to take account of ideology.  Support for gun control and for legalizing marijuana both are associated with being liberal (as that term is used in ordinary political discourse). African Americans are, on average, much more liberal than whites.  Can we see the racial difference, then, as reflecting the greater ideological diversity in the white population?

To answer that question, I think it is helpful to start by getting a sense of just how much more liberal African Americans are than whites.  Consider:

What I’ve done, essentially, is plot the distribution (with a kernel density estimate) of whites and African Americans respectively over a continuous measure of right-left political orientation formed by aggregating the subjects’ self-reported identification with the Democratic and Republican parties and their ideology on a “liberal-conservative” scale.

You can “see” how much more “liberal Democratic” African-Americans are. Whereas the mean score on the scale for whites puts them just shy of the middle of the “Liberal Democratic” scale, the mean for African-Americans is closer to +1 Standard Deviation toward “Liberal Democratic. Minus 1 SD in the distribution for African-Americans equates to being around the middle of the scale; minus 1 SD for whites is a little more than one SD toward "Conservative Republican." 

Basically, “normalizing” the scale for race, one would say that a “middle of the road” African American is as far right within his or her group as a “moderately conservative Republican” white is for his or hers. 

Now consider this:


These are “fitted values” based on regression analyses (click on the thumbnail if you'd like to see them) that treat race, “liberal Democratic” political orientation, and the interaction of those two as predictors for support for gun control and support for marijuana legalization, respectively.  They tell you, in effect, how a progressively more “liberal Democratic” orientation influences positions on these issues for both groups.  I’ve started plotting the African-American line at the middle of the “liberal Democratic” scale because the proportion of African Americans more “conservative Republican” than that is so small—and it is visually misleading, I think, to include them if one is trying to assess how important ideology is in explaining the difference in white & African-American views on drugs & on guns.

What you can see, in effect, is that African Americans are more supportive of gun control, and less supportive of marijuana legalization, than whites of comparable political outlook. 

The effect is most pronounced for marijuana legalization. There, political outlooks and race interact, signifying that the impact of becoming more liberal and Democratic on support for legalization is smaller for African Americans than for whites.  There is no similar interaction between political outlooks and race for gun control; in effect, African Americans are uniformly more pro-gun control than whites at any given point on the political outlook spectrum.

This was basically in line with what I expected. That is, based on what I know of the public opinion research, I expected African Americans to be more pro-gun control and less pro-legalization than whites, and in fact anticipated this effect would be more pronounced for marijuana legalization.

Still, I was surprised by the degree of support for legalization among African Americans generally.  The level of support is higher, I think, than it was in years past. And the differential between African Americans and whites—in relation to political outlooks--smaller than I expected.

I’ll have to ask Tracey Meares, who has studied this, whether I’m right in my recollection/expectation and what she thinks about these results (maybe I can even get her to offer her views on the blog!).

What does this add to the discussion of gun control and drug prohibition?


Information, too, that is only an imprecise indicator of what are in fact very very important differences in perspective, founded on differences in experience and cultural meaning, among citizens who need to deliberate on guns and drugs. 

You should try to learn as much as they can about the sources of these differences, and about the judgments of fact and value that underlie them.  If you don't do this, your positions on gun control and drug prohibition will not be well-considered, no matter what they are.




The black and white -- but mainly gray -- of gun control and drug prohibition, part 1

This is part one of a two-part series on race, gun control, and drug criminalization.

The motivation for the series grows out of discussion relating to my post on legalizing drugs (I’m simplifying; really, something like this) as a means of reducing the gun-homicide rate in the U.S.

A question was posed in the discussion about what African-American public opinion is on drugs and guns.  I’ve culled some data on that from past CCP studies.  I’ll present these data in the next post. In this one, I want to lay out some more background. . . .

One of the reasons I made the drug-legalization proposal was my sense of discomfort with the post-Newtown framing of the gun-homicide crisis in our country. There are (literally) orders of magnitude more children (adults too) being shot & killed in inner-city neighborhoods in cities like Chicago than are being shot & killed in mass shootings by mentally ill people. It’s unclear how effective gun control—of any sort—would be in reducing gun homicides of any type.  But it’s clear that a very large fraction of U.S. gun homicides (leaving out suicide, which accounts for more than half of U.S. gun deaths) are associated with the lethal violence incident to the black market status of recreational drugs like marijuana and cocaine.

Why the hell aren’t we talking about this? Why haven’t we been for years already, given the steady, relentless toll in death (not to mention ruined lives, wasted dollars, etc.) associated with our completely dysfunctional, idiotic drug-law regime?

Well, actually, some people do talk about our drugs laws—and in particular whether their should be legalization—quite a lot. Many of them live in the very areas where drug-related, including drug-law-related, violence is the highest.  And they aren’t—or certainly aren’t all—persuded that drug legalization is the way to go.

My brilliant colleague Tracey Meares first educated me on these issues back when we were both assistant professors (we had to clean the chalk boards after Richard Epstein’s & Cass Sunstein’s lectures; sometimes this kid from the neighborhood, Barack Obama, would help too) at the University of Chicago Law School.

She told me, and showed me with both research and with vivid examples from the neighborhoods right around us, that there was a profound ambivalence about drug criminalization in inner-city, largely African American communities in the South Side of Chicago. 

There was no ambivalence about the existing drug-law enforcement posture of both the state and federal governments: everyone recognized that basically disposing of the lives of people involved in the drug trade by just tossing them in jail for huge amounts of time was idiotic, as well as indescribably cruel both to them and their communities.

But it wasn’t at all the case that the citizens who were most adversely affected by this dumb policy all thought that drugs should be legal. On the contrary, most of them were deeply opposed to legalization on moral grounds. Moral not primarily in the illiberal sense of wanting to tell other adult people how to live their lives, but moral in the sense of wanting the same fair opportunity that all adults are entitled to have to mould the characters of their children, instilling in them the outlooks and dispositions and habits most conducive to their happiness and flourishing.

All parents compete in that shaping process, of course, with myriad other influences.  These include commercial, market-driven ones that try to conjure demand for useless or even pernicious products by inventing fanciful stories about the glamorous or virtuous ways of life they conduce to (think, as you might have been recently, about hideous  “man cards”; but think, too, of Virginia Slim, etc.).

Parents in the communities in the inner-city in the South Side of Chicago (at least in the early to mid-1990s, when I lived there) resented these forces, like all parents do. They hated the companies that plastered their communities with billboards and other forms of advertising targeted (they correctly perceived) at creating in their children a taste—emotional and physical—for products like “malt liquor” and cigarettes.

They worried that if recreational drugs were made legal, manufacturers of those products would join the ranks of commercial firms poisoning the moral environment in which they were raising their children.

The way they—or many of them; reasonable, reasoning citizens inevitably have differences of views on complex matters—tried to reconcile their support for drug criminalization with their opposition to the existing regime of drug-law-enforcement was by trying to reform the former. Organizing through the processes associated with the City’s innovative community policy program, they advocated things like “gang loitering” laws as alternatives to draconian criminal sentences.

I wasn’t sure that I agreed with all the positions members of these communities took on all these issues, particularly on the criminalziation of drugs (the gang-loitering law, in context, did seem quite sensible to  me).  

But one thing I did become convinced of was that they were better situated, in every relevant sense, to figure out how these various issues should be resolved in their lives than I was.  They hadn’t necessarily read the latest issue of the Journal of Criminal Law and Criminology but by seeking out information from people they trust, and employing their informed sense of who knows what about what, they had arrived –by the same means we all do, in all areas of our lives—at perfectly justifiable grounds for making complicated decisions.

Moreover, they were the ones whose lives those decisions, no matter who made them, were going to most meaningfully affect. They were the ones bearing all the relevant costs—in the form of the impact that whatever regime of laws and enforcement strategy would be adopted would have on their sons and daughters, their brothers and sisters, and nephews and nieces, and close friends and neighbors.

There’s no way to honor the dignity that reasoning people are due under these circumstances other than to respect their right to govern themselves.

... What were we talking about again?  Oh, drug laws and guns.

Another thing that people in these communities believed, very strongly, was that there should be significant regulation of guns.  Two weeks ago, the U.S. Court of Appeals for the Seventh Circuit told them that’s not an area in which they are entitled to govern themselves: relying on the U.S. Supreme Court’s recent Second Amendment decisions (including one, also involving Chicago, that reached the by-no-means obvious determination that the Second Amendment limits the power of state governments, and not merely that of the federal one) struck down an Illinois law that prohibits carrying a concealed handgun in public without a permit issued for “cause.”

I’m very doubtful that “tough gun laws”—which like prohibition of alcohol and drugs create black markets and the forms of violence that inevitably attend them—will contribute much to solving the problem of violent homicide in our country.  But I don’t have any doubt that people in places like Chicago have a right to figure that out for themselves.

Okay. . . Next time: some data.


Fung, A. Empowered participation : reinventing urban democracy. (Princeton University Press, Princeton, N.J.; 2004).

Kahan, D.M. & Meares, T.L. The Coming Crisis of Criminal Procedure. Geo. L.J. 86, 1153 (1998).

Meares, T.L. It's a Question of Connections. Val. U. L. Rev. 579 31, 579 (1997).

Meares, T.L. Social Organization and Drug Law Enforcement. Am. Crim. L. Rev. 35, 191 (1998).

Meares, T.L. & Kahan, D.M. Urgent times : policing and rights in inner-city communities. (Beacon Press, Boston; 1999).

Skogan, W.G. Police and community in Chicago : a tale of three cities. (Oxford University Press, Oxford ; New York; 2006).

Skogan, W.G. & Hartnett, S.M. Community Policing Chicago Style. (1997).


Gateway post ...

Hey, now that you are thinking about how consequential our mindless drug laws are for our unacceptable gun homicide rate, you should try the really good stuff ...


Catch up on commentary day

Lots of interesting comments on the "Less drug laws, less gun homicide" & "Mass opinion coherence & do psychologists tell more tall tales than political scientists?" posts, including some on latter from Stats Legend Andrew Gelman (who also responds w/ post at his own site).

Much more interesting than anything I had planned to say today -- plus I'd like to devote my own "blog time" today to re-reading & responding to them.


Do mass political opinions cohere? And do psychologists "generalize without evidence" more often than political scientists?

Stats Legend Andrew Gelman (whose blog everyone who enjoys being surprised and who values high-quality analytical thinking should read daily) has an interesting post on Steven Pinker.

Pinker asks “[w]hy, if you know a person’s position on gay marriage, can you predict that he or she will want to increase the military budget and decrease the tax rate,” a question he answers by observing that “[p]olitical philosophers have long known that the ideologies are rooted in different conceptions of human nature — a conflict of visions so fundamental as to align opinions on dozens of issues that would seem to have nothing in common.”

Gelman responds by (1) doing some quick GSS correlations, on the basis of which he concludes that “attitudes on such diverse issues are not so highly correlated”; and then (2) attributing Pinker’s error to Pinker’s being a psychologist rather than a political scientist and thus prone to “present[ing] ideas that are thought-provoking but . . . too general to quite work,” in contrast to political scientists who “take such ideas and try to adapt them more closely to particular circumstances.”

Some thoughts:

1. Pinker is clearly right to note that mass political opinions on seemingly diverse issues cohere, and Andrew, I think, is way too quick to challenge this.

I could cite to billions of interesting papers, but I’ll just show you what I mean instead. A recent CCP data collection involving a nationally representative on-line sample of 1750 subjects included a module that asked the subjects to indicate on a six-point scale “how strongly . . . you support or oppose” a collection of policies:  

  1. policy_gun  Stricter gun control laws in the United States.
  2. policy_healthcare  Universal health care.
  3. policy_taxcut  Raising income taxes for persons in the highest-income tax bracket.
  4. policy_affirmative action  Affirmative action for minorities.
  5. policy_warming  Stricter carbon emission standards to reduce global warming.


 Positions clustered on these “diverse” items big time. The average inter-item correlation was 0.66. The Cronbach’s alpha—a scale reliability measure based on item covariance and the number of items—was 0.91.

This is a degree of coherence that would  make any social scientist – psychologist or political scientist – beam. The highest possible alpha is 1.0, and anything above 0.70 is usually regarded as signifying a high degree of reliability.  Low reliability, measured in this way, is it’s own punishment, since it constrains the power of any sort of explanatory or predictive model involving the scale. With a score of 0.91 you can be confident that the power of your model won’t be dissipated by the noise associated with the imprecision of the observable "indicators" you are using to measure the latent variable. 

The latent variable being picked up by these policy items is obviously something akin to right-left political preferences, so let’s call the resulting measure “Liberal_policy.” (Additional items cohered better with each other than with these, forming a second "libertarian policy prefernce" scale; but let's keep things simple.)

Being able to form a scale like this with a general population sample is pretty good evidence in itself (and better than just picking two items out of GSS and seeing if they correlate) that people’s opinions on such matters cohere.

But just to make the case even stronger, let’s consider how much of the variance in liberal policy preferences can be explained by ideology. 

In the same data set, there was a five-point measure for self-described “liberal-conservative ideology” and a  seven-point one for identification with the two major political parties. Those two items were also highly correlated (r = 0.70), so I combined them into a scale (α = 0.82) coded to represent a right-wing ideological disposition, which I labeled “Conserv_repub.”

Regressing Liberal_policy on Conserv_repub, I discovered that the percentage of variance explained (R2) was 0.60. That’s high, as any competent psychologist or political scientist would tell you, and as I’m sure Andrew would agree!

Now Andrew noted that the degree of coherence in political preferences tends to be conditional on other characteristics, such as wealth, education, and political interest. Typically, political scientists use a “political knowledge” measure to assess how coherence in ideological positions vary.

I had a measure of that (a 9-item civics-test sort of thing) in the data set too. So I added it and a cross-product interaction term to my regression model. It bumped up the R2 – variance explained – by 4%, an increment that was statistically significant.  

Seems small, but how practically important is that? A commenter on Andrew’s blog noted that I tend to criticize fixating on R2 as an effect-size measure; my point, which is one that good social scientists—political scientists and psychologists! Andrew too!--have been making for decades is that R2 is not a good measure of the practical significance of an effect size, a matter that has to be determined by use of judgment with relation to the phenomenon at issue.

Well, to help us figure that out, I ran a Monte Carlo simulation to generate the predicted probability that a typical “Liberal Democrat” (-1 SD on Conserv_Repub) and a typical “Conservative Republican” (+1 SD) would support “stricter gun control laws” (seems topical; this is pre-Newtown, so it would be interesting to collect some data now to follow up), conditional on being “low” (-1 SD) or “high” (+1 SD) in political knowledge.

Seems (a) like variance in political knowledge (whatever its contribution to R2) can matter a lot – the probability that a high–political-knowledge Republican will oppose gun control is a lot lower than that for a low–political-knowledge one—but (b) there is still plenty of disagreement even among low–political-knowledge subjects. 

I’d say, then, that Andrew is being a bit too harsh on Pinker’s premise about political preference coherence.

2. Pinker is clearly wrong—not just in his answer but in his style of reasoning—to connect this sort of coherence to “different conceptions of human nature" among people of opposing ideologies

Pinker, however, is indeed doing something very objectionable: he is engaged in rank story-telling

He notes that political philosophers identify ideologies with different conceptions of “human nature,” a “conflict of visions so fundamental as to align opinions on dozens of issues.” Well, maybe political philosophers do do that. But the idea that “different conceptions of ‘human nature’ ” explain coherence and variance in mass political opinion is an empirical claim, and as far as I know there’s not any support for it. 

I think it’s almost certainly false. Measures of ideology of the sort that I have used here have not – as far as I know; please do tell me if I’m wrong: the pleasure of learning something new will more than compensate me for the embarrassment of being shown to be ignorant -- been validated as predictors of “different conceptions of human nature.” Indeed, I think the idea that ordinary members of the public have “conceptions of human nature” is extravagant—the sort of thing only someone who has never ventured outside a university campus would likely believe.

There are myriad theories about the puzzling question of how ordinary people, who really aren’t philosophers, aren’t that interested in politics, and who are very consumed with other things can manage to form coherent ideological preferences. And they’ve been tested empirically.

It’s irritating for anyone who is familiar with all that work to see Pinker advance the sort of claim he does—which he presents not even as a conjecture but as a simple, unqualified, fact-of-the matter report.

3. Pinker’s mistake is one psychologists would resent as much as political scientists.

The sort of thing Pinker is doing here generalizes.  Popular commentators love to reach into the grab bag of decision science mechanisms and construct just-so stories that purport to “explain” complicated phenomena (e.g., political controversy over climate change). 

Good social scientists hate this.  Indeed, Pinker generally doesn’t like it, in fact; he complains about this practice in his excellent book, The Better Angels of Our Nature: Why Violence Has Declined, which admirably tries to connect trends in violence over history to mechanisms that themselves have support in evidence. I find it sort of deflating to see that he seems to adopt a different approach in the writing he does for the New York Times.

But the point is, resentment of story-telling is something that psychologists and political scientists would both experience. It’s not a consequence of Pinker being a psychologist!

4. Ironically, Andrew is making the sort of mistake he says Pinker made.

This last point follows from all the others. Andrew sees Pinker doing something irritating, and then treats a conjecture (I think a pretty uninteresting, implausible one; but all conjectures are created equal – test away!) as a general law that explains this particular instance, etc.

But now I will offer a conjecture, based on an observation-grounded theory.

The observation-grounded theory is that Andrew Gelman has a virtuous Bayesian disposition. That is, he is the sort of person who very happily updates and revises his views, which he always regards as just provisional estimates anyway.

The conjecture: that Andrew, on reflection, will agree that he offered a poor diagnosis (“psychologists generalize without evidence, unlike political scientists, who look for concrete evidence in particulars!”) of Pinker’s objectionable style of argumentation here (which, again, strikes me as uncharacteristic of Pinker himself!).

And now, let’s collect some evidence.

(One more prediction, or hope: Andrew will like my graphic!)

p.s. Ideological coherence in policy prefernces isn't nearly as interesting -- nearly as surprising, puzzling --as ideological or cultural coherence in factual beliefs (e.g., "earth is/is not heating up" & "children of gay & lesbians do worse/no worse in life than ones raised by heterosexual parents." That's what CCP research is all about. Perhaps I'll do another post on that.


Abelson, R.P. A Variance Explanation Paradox: When a Little is a Lot. Psychological Bulletin 97, 129-133 (1985).

Delli Carpini, M.X. & Keeter, S. What Americans Know About Politics and Why It Matters. (Yale University Press, New Haven; 1996).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

John, O.P. & Benet-Martínez, V. in Handbook of research methods in social and personality psychology. (eds. H.T. Reis & C.M. Judd) 339-369 (Cambridge University Press, New York; 2000).

King, G. How Not to Lie with Statistics. Am. J. Pol. Sci. 30, 666-687 (1986). 

King, G., Tomz, M. & Wittenberg., J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am. J. Pol. Sci 44, 347-361 (2000).

Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).


Actually, empirical evidence suggests a sure fire way to dramatically lower gun homicides: repeal drug laws

Sticking my umbrella up to try to deflect the tsunami of specious, beside-the-point, hiding-the-cultural-imperialism-ball, insult-to-reason uses of empirical evidence on violence & gun control (emanating from those on both sides of the debate), I offered a couple of feeble blog posts (here and here) urging more thoughtful, grounded arguments that are mindful both of the limits of our knowledge and our duty, as citizens of a liberal democracy, to justify our policy positions on grounds free of the impulse to use law as an expressive weapon for denigrating cultural styles that differ from our own.

I'm not satisfied by my performance.

The problem isn't that what I wrote will make no difference, have no effect, etc. If that were the decisive issue, why would I ever say anything? I'm not trying to change the world; I'm just trying to engage other curious and concerned citizens who also want to think about issues reflectively and form positions that they think are factually supportable and morally defensible. I'm trying to help them so they'll help me back, since I'm by no means certain I'm right either.

No, the problem wasn't the futility of what I said but the incompleteness. "Don't look to statistics" -- could be read as denigrating the utility of empirical inquiry in assessing public policy and as expressing a sort of nihilistic "who cares, nothing we can do!" attitude, both of which would deeply misrepresent how I feel about evidence-based policymaking generally & about using evidence to think about guns.

So to amend I will emend.  

I now want to point out that in fact, while the empirical evidence on the relationship between gun control and homicide is (at this time at least) utterly inconclusive, there certainly are policies out there that we have very solid evidence to believe would reduce gun-related homicides very substantially.

The one at the top of the list, in my view, is to legalize recreational drugs such as marijuana and cocaine.

The theory behind this policy prescription is that illegal markets breed competition-driven violence among suppliers by offering the prospect of monopoly profits and by denying them lawful means for enforcing commercial obligations.

The evidence is ample. In addition to empirical studies of drug-law enforcement and crime rates, it includes the marked increase in homicide rates that attended alcohol prohibition and the subsequent, dramatic deline of it after repeal of the 18th Amendment.

from Claude Fischer, Berkeley Blog Actually, it's pretty interesting to look at homicide rates over a broader historical time frame than typically is brought into view by those who opportunistically crop the picture in one way or another to support their position for or against gun control.  What you see is that there is a pretty steady historical trend toward decline in the US punctuated by expected noisy interludes but also by what appear to be some genuine, and genuinely dramatic, jumps & declines.

One of the jumps appears to have occurred with the onset of prohibition and one of the declines with repeal of prohibition.  Social scientists doing their best to understand the evidence generally have concluded that that those are real shifts, and that they really were caused by prohibition and repeal.  

from Claude Fischer, Berkeley Blog

Criminologists looking at the impact of drug prohibition can use the models developed in connection with alcohol prohibition and other modeling strategies to try to assess the impact of drug prohibition on crime. Obviously the evidence needs to be interpreted, supports reasonable competing interpretations, and can never do more than justify provisional conclusions, ones  that are necessarily subject to revision in light of new evidence, new analyses, and so forth.

But I'd say the weight of the evidence pretty convincingly shows that drug-related homicides generated as a consequence of drug prohibition are tremendously high and account for much of the difference in the homicide rates in the U.S. and those in comparable liberal market societies (the non-liberal, non-market societies all are burdened with homicide rates orders of magnitude higher; guns don't explain that--  the pacifying influence of doux commerce does).  By all means decide for yourself, though; I've cited some reading material at the end of this post & urge others to call our attention to more in the comments section.

There are obviously other spikes & dips in the "secular" (as econometricians would say; here that's a very nice adjective to use) downward trend in homicide in the U.S. E.g., the upsurge of homicides in the 1960s and the decline in the 1990s. Most of the gun-control combatants mine this period for support for their claims -- they should, since the data are rich with support for specious inference.  Scholarly discussion here recognizes that the evidence on the contribution of guns to these jumps is utterly, hopelessly inconclusive.

There is a very interesting empirical study, though, by economist Jeffrey Miron, who concludes that the available evidence is consistent with the hypothesis that the difference in homicide rates in the US and in other liberal market societies is attributable to our drug prohibition policies. Gun availability in the US, according to this hypothesis, doesn't directly account for the difference in homicide rates between the US and these countries; rather, gun availability mediates the impact between drug prohibition and homicide rates in the US, because the criminogenic properties of drug prohibition create both a demand for murder of one's competitors and a demand for guns to use for that purpose.  

One of the very nice things about Miron's analysis, too, is that he is appropriately provisional about his conclusions:  

The empirical results presented above provide a possible explanation for the large differences in violence rates across countries, and they suggest that previous analyses might have spuriously attributed these differences to gun control or availability. According to the analysis here, differences in drug prohibition enforcement explain differences in violence, which in turn explain differences in gun ownership that correlate positively with violence but do not cause that violence. Further, the results provide a hint that restrictive gun control regimes can themselves increase violence. As noted above, these results should be considered suggestive rather than conclusive. Future research on these issues will need to exploit time‐series rather than cross‐sectional data.

 That's what a real scholar sounds like, you see. In my view, it's what an open-minded citizen sounds like too.

Now one thing to note: Obviously, decriminalizing marijuana and cocaine couldn't be expected to prevent mass shootings like the one in Newtown, or Aurora, or Phoenix, or Columbine, etc. (Maybe there'd be fewer guns around, actually, if we didn't have the demand for them associated with their contribution to the illegal drug trade, but there are already so many around -- the gun & people populations are neck-and-neck -- that I think disturbed people would still have no trouble getting their hands on them.)

But here's another thing to note: these very sad incidents "represent only a sliver of America's overall gun violence." Those who are appropriately interested in reducing gun homicides generally and who are (also appropriately) making this tragedy the occasion to discuss how we as a society can and must do more to make our citizens safe, and who are, in the course of making their arguments invoking (appropraitely!) the overall gun homicide rate should be focusing on what we can be done most directly and feasibly to save the most lives.

Repealing drug laws would do more --  much, much, much more -- than banning assault rifles (a measure I would agree is quite appropriate); barring carrying of concealed handguns in public  (I'd vote for that in my state, if after hearing from people who felt differently from me, I could give an account of my position that fairly meets their points and doesn't trade on tacit hostility toward or mere incomprehension of  whatever contribution owning a gun makes to their experience of a meaningful free life); closing the "gun show" loophole; extending waiting periods etc.  Or at least there is evidence for believing that, and we are entitled to make policy on the best understanding we can form of how the world works so long as we are open to new evidence and aren't otherwise interfering with liberties that we ought, in a liberal society, to respect.

...Now, what other policies might help? And in particular, if we are concerned about deaths of children?   Well, there's swimming pools .... But I've said enough for now.


Barnett, R. & Trip, B. Drug Prohibition and the Weakness of Public Policy’(1994). Yale LJ 103, 2593. 

Husak, D. Legalize This! The Case for Decriminalizing Drugs. (Verso, New York; 2002).

Jensen, G.F. Prohibition, Alcohol, and Murder Untangling Countervailing Mechanisms. Homicide Studies 4, 18-36 (2000).

Kleiman, M. Marijuana : costs of abuse, costs of control. (Greenwood Press, New York; 1989).

Kleiman, M., Caulkins, J.P. & Hawken, A. Drugs and drug policy : what everyone needs to know. (Oxford University Press, Oxford ; New York; 2011).

MacCoun, R.J. & Reuter, P. Drug war heresies : learning from other vices, times, and places. (Cambridge University Press, Cambridge, U.K. ; New York; 2001).

Miron, J.A. & Zwiebel, J. The Economic Case Against Drug Prohibition. The Journal of Economic Perspectives 9, 175-192 (1995).

Miron, J.A. Violence, Guns, and Drugs: A Cross‐Country Analysis*. Journal of Law and Economics 44, 615-633 (2001).

 Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).



Statistics: the monopoly money of the gun-debate marketplace of ideas

This is what makes people cynical about empirical arguments in policy debate. Recognize when the data are inconclusive, or else no one will be able to recognize what counts as sound evidence.

If you are contributing to this devaluation of the currency of reason, just stop. In particular, stop insisting that everyone who disagrees with you on facts is either an "idiot" or a "liar."

Look: There are lots of good, thoughtful arguments to be made here, ones based on value and ones based on the best factual surmises we can make based on experience and sense.  

These are arguments that citizens in a liberal society can advance openly, and should, to confirm, for themselves and for others, that their positions aren't motivated by the illiberal ambition to denigrate those whose cultural identities differ from their own. Reliance on one-sided, cherry-picked empirical arguments doesn't furnish that assurance; on the contrary, it's the "smoking gun" of cognitive illiberalism.




"More statistics, less persuasion": the gun control debate continues, and continues to miss the point...

I was planning to write something about Moore v. Madigan, No. 12-1269 (7th Cir. Dec. 11, 2012),  the U.S. Court of Appeals decision earlier in the week that overturned an Illinois law that more or less bans ordinary citizens from carrying concealed weapons in public.

Judge Richard Posner, our century's Oliver Wendell Holmes Jr., wrote the opinion.  Not surprisingly, it's a good read.

Posner irreverently mocks any attempt to wring meaning from the "original intent" of the Second Amendment (Posner is now engaged in a very unseemly but entertaining pissing match with Justice Scalia, in which both have basically said "liar, liar pants on fire!" about Posner's New Republic review of Scalia's grammar-school quality text on  interpretation).

He also ruthlessly --nihilistically, even; Posner, like Holmes, exudes a nihilistic philosophy & style-- arrays against one another conflicting empirical studies on the impact of concealed-weapons laws on violent crime (uncharacteristically, Posner misses a supporting citation: to an "expert consensus" report from the National Academy of Sciences, which declares such evidence utterly inconclusive).

Posner concludes:

A gun is a potential danger to more people if carried in public than just kept in the home. But the other side of this coin is that knowing that many law-abiding citizens are walking the streets armed may make criminals timid. Given that in Chicago, at least, most murders occur outside the home, Chicago Police Dep’t, Crime at a Glance: District 1 13 (Jan.–June 2010), the net effect on crime rates in general and murder rates in particular of allowing the carriage of guns in public is uncertain both as a matter of theory and empirically. “Based on findings from national law assessments, crossnational comparisons, and index studies, evidence is insufficient to determine whether the degree or intensity of firearms regulation is associated with decreased (or increased) violence.” Robert A. Hahn et al., “Firearms Laws and the Reduction of Violence: A Systematic Review,” 28 Am. J. Preventive Med. 40, 59 (2005); cf. John J. Donohue, “The Impact of Concealed-Carry Laws,” in Evaluating Gun Policy Effects on Crime and Violence 287, 314–21 (2003). “Whether the net effect of relaxing concealed-carry laws is to increase or reduce the burden of crime, there is good reason to believe that the net is not large…Based on available empirical data, therefore, we expect relatively little public safety impact if courts invalidate laws that prohibit gun carrying outside the home, assuming that some sort of permit system for public carry is allowed to stand.” Philip J. Cook, Jens Ludwig & Adam M. Samaha, “Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective,” 56 UCLA L. Rev. 1041, 1082 (2009)....

In sum, the empirical literature on the effects of allowing the carriage of guns in public fails to establish a pragmatic defense of the Illinois law. Bishop, supra, at 922–23; Mark V. Tushnet, Out of Range: Why the Constitution Can’t End the Battle over Guns 110–11 (2007). Anyway the Supreme Court made clear in Heller that it wasn’t going to make the right to bear arms depend on casualty counts. 554 U.S. at 636. If the mere possibility that allowing guns to be carried in public would increase the crime or death rates sufficed to justify a ban, Heller would have been decided the other way, for that possibility was as great in the District of Columbia as it is in Illinois.

So ... I was planning to write something along the lines, "the statistical debate has been won by the gun-rights team, which has (a) effectively fought the statistical battle to a standstill & (b) succceeded in getting courts to impose a liberty-preserving standard of proof on those who want to restrict guns. Point (b) -- who should bear the burden of proof -- is an interesting question in a liberal society, yet people will still go on and on about statistics blah blah."  Or some such.

But now this... And I don't know what to say. Except that statistics really are beside the point.

They are beside the point because they are genuinely inconclusive.

They are beside the point because they genuinely don't engage what psychologically motivates people's positions here.

And they are beside the point because they ignore the real moral issues, which are ones of social meaning:

  • What does it say about what people value when they want to own a cache of military-style armaments such as a "Glock," a "SIG Sauer handgun," and a "Bushmaster .223-Caliber Assault rifle"? (These were weapons owned, apparently, by the shooter's mother, his first victim, who was incorrectly reported to be a kindergarten teacher at the school where the shooting spree took place; guess she was also an avid "self-defense" enthusiast?)   
  • What does it say about us when we permit other people to make money -- lots of it -- satisfying the appetite of ordinary people to own the tools of the trade of those whose profession is killing?

  • Yet what would it say about us if in saying this with our law we couldn't also acknowledge that owning various types of guns are also, for many, enmeshed with a host of very different cultural meanings--ones relating to personal virtues like self-reliance, honor, and responsibility; and to social roles and practices that intimately connect them to people, past and present, whom they do value, and very appropriately so?
  • What would it say about our commitment to liberal principles if those of us who don't belong to communities in which guns have these meanings (and in fact are puzzled by them) refused to acknowledge (or just couldn't see; the two are connected) that those who do belong to them understandably see many kinds of gun control as expressing hostility and contempt for their values? Understandably, because in fact, many (not all!) of those who advocate gun control are motivated by (or are simply profiting from) exactly that sort of ugly, illiberal sentiment?

This is a conflict of cultural meaning that must be negotiated by law. And it should be negotiated in a way that is consistent with liberal political principles, which impose on citizens of diverse understandings the duty to show they are committed to accommodating one another, and to resisting making use of laws (whether handgun bans or "stand your ground" provisions) as expressive symbols of dominance over one another.

This is complicated.  Much more complicated than the convoluted (and utterly inconclusive) multivariate regression analyses with which those involved in the "statistics" debate beat each other over the head.

And more complicated than simply mocking those who think statistics (or "history") can solve this issue.

We don't need nihilism in our public discourse. We need genuine liberal statesmanship.

We don't need a Holmes. We need a Lincoln.


A multivariate regression analysis of CRT performance & ideology, plus a preliminary diatribe against mindlessly overspecified regression models

My most recent paper, Ideology, Motivated Reasoning, and Cognitive Reflection, reports data on the relationship between ideology and the disposition to use high-level, "System 1" information processing, as opposed to intuitive, low-effort, heuristic-drive "System 2," as measured by the Cognitive Reflection Test (CRT).  

I report that there really isn't any. That's sort of surprising in light of all the attention being paid to the neo-Authoritarian Personality literature, which asserts that conservativism is characterized by closed-mindedness, aversion to complexity, and the like.

I know reviewers will want to know, "but if one controls for ..." so I've prepared a multivariate regression that includes ideology along with various other individual characteristics (gender, race, education, income, and religiosity) that have been shown to correlate with CRT scores.  Still nothing!

See for yourself by clicking on the thumbnail to the left.

But in fact, I view this analysis as pretty close to worthless -- if one is really trying to figure out if there is an association between ideology and cognitive reflection.  The idea of  "controlling for" these sorts of characteristics in order to measure the "independent" impact of "ideology" is nonsense: it treats "ideology" as some sort of disembodied "essence" inside of people, when in fact it is one facet of an integrated package of attributes that cohere with one another and are all indicators of a latent type of shared identity or style.

I've been planning for a while to go ballistic on the idea of "overspecified" regressions -- and what sort of mistake in thinking (or failing to think, really) they reflect. This isn't really the right vehicle for getting the point across, so I anticipate coming back to this topic at some point.

But as a preview, here is a short text that I prepared to go with this genuinely fantastic multivariate regression analysis of ideology & CRT.  I'm sure the note will be revised -- or dropped altogether -- before I submit the paper for publication:

Finally, a multivariate regression model that included all of these predictors was tested. That analysis can be seen as assessing the effect of ideology on CRT scores “controlling for” gender, race, education, income, and religiosity. It is doubtful, however, that such a “model” bears any meaningful relationship to reality. In the world we live in, people come in packages of demographic and political orientations, which correlate in ways suggestive of various latent forms of identity. Thus, “partialing” out the covariance of gender, race, religion, education, and income in order to estimate the “independent” effect of ideology creates a model of something (either individual characteristics disconnected from people, or people who can be randomly endowed with combinations of characteristics) not actually observed on planet earth (Berry & Feldman 1985, p. 48; Cohen, Cohen, West &  Aiken 2003, p. 419; Gelman & Hill 2006, p. 187). Absent some appropriate aggregation of all these variables into a valid latent-variable measure, a zero-order correlation furnishes a more valid estimate of the influence of subjects' political outlooks on their CRT scores than does the coefficient for that predictor in a model that treats ideology and all of these other characteristics as independent, right-hand side variables in a multivariate regression (Lieberson 1985, pp. 14-43). But for the benefit of those who prefer to regard multivariate regression as a magical black box for capturing the “causal” effect of phantom essences, as opposed to a statistical tool for measuring the relationship of valid measures of real-world phenomena, this blunderbuss analysis shows the the coefficient for Conserv_Repub remained trivially different from zero and nonsignificant (b = 0.06,  p = 0.33). When party self-identification was substituted for Conserv_Repub, that variable continued to predict an increase in CRT score as subjects’ identification with the Republican party intensified, but the effect was reduced and was only marginally significant (b = 0.10, p = 0.06).


Berry, W.D. & Feldman, S. Multiple regression in practice. (Sage Publications, Beverly Hills; 1985).

Cohen, J., Cohen, P., West, S.G. & Aiken, L.S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Edn. 3rd. (L. Erlbaum Associates, Mahwah, N.J.; 2003).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

Lieberson, S. Making it count : the improvement of social research and theory. (University of California Press, Berkeley; 1985).


Query on climate change, culture, ideology & identity-protective cognition

A thoughtful person asks:

I’ve come across your work while trying to make sense of climate change denial. I find your analysis very interesting (along with Flynn et al. 1994, Finucane et al. 2007, and McCright and Dunlap 2011) because it offers a compelling explanation for what seems like a curious social dynamic.

However, there’s something I don’t quite follow, and with your kind forbearance I hope I may ask you a question. Your 2007 Journal of Empirical Legal Studies paper sketches out a synthesis of cultural risk perception and identity-protective cognition. From that I was expecting to see how group identity (conservative, Republican) and world vision (hierarchical, individualistic) somehow mutually reinforced each other in the climate change arena. In fact, though, that seems not to be the case. Instead it is the hierarchical and individualistic world vision itself that cognition seeks to protect. Indeed, your regression 4 in Table 2 (p. 483) if anything seems inconsistent with my expectation, since both "Conservative" and "Democrat" are highly significant, whereas if these substantially overlapped with the hierarchical white male dummy (so to speak), the coefficients would have been insignificant. Is your view that Conservative and Democrat are independent (from the white male effect) determinants of views on climate change? The narrative in McCright and Dunlap directly linking climate change skepticism to conservatism appeals to me, but I'm not sure if it's consistent with your own perceptions and/or findings.

Thanks very much, and thanks for your very interesting paper.

This is my response. Anyone want to add anything? 

I believe that people have unobservable latent predispositions that they acquire as a result of one or another social influence. The thing to do is find observable indicators that one has good reason to believe correlate with those dispositions, combine them into reliable scales, and use those measures to test hypotheses about who sees what & why, & about what sorts of communication strategies are geared to promoting open-minded engagement with information by people of diverse predispositions.  "Republican, Democrat, liberal, conservative, hierarch, egalitarian, individualist," etc are all candidate indicators. Which ones to combine to form scales depends on which latent-variable measurement strategy most instructively enables explanation, prediction & prescription.  

For more info, click on links below; & let me know if you have additional questions or if you have reactions, comments etc.


Yow--lots of great comments on "science literacy vs. climate-change science literacy"

I was going to post something, but it's not as interesting or important as the points that people made in response to yesterday's post on science literacy & climate-change science literacy.  So check those out! And add more.



Science literacy vs. "climate science literacy"

This is in the department "recurring misunderstanding that I should say something about in a single place so that I can simply refer people to it."

Last May, CCP researchers published a study in Nature Climate Change presenting evidence suggesting that political controversy over climate change in the US cannot be attributed to any sort of deficit in the public's comprehension of science.

As science literacy and numeracy  (a technical reasoning disposition associated with more discerning perception of risk) increase, members of the general public do not converge in their perceptions of the risks posed by climate change. Instead, they become even more culturally polarized.

This finding fit the hypothesis that individuals can be expected to engage information in a manner that fits their interest in forming and maintaining beliefs that reflect their membership in, and loyalty to, important affinity groups.

Competing positions on climate change, unfortunately, are now conspicuously associated with opposing cultural groups. Being out of line with one's group on this issue exposes an individual to a social cost, whereas forming a mistaken view on the science of climate change has zero impact on the risk that individual, or anyone or anything she cares about, faces, insofar as one individual's personal behavior (as consumer, voter, public discussant, etc.) has no material effect on the climate.

One doesn't have to be a rocket scientist to figure out what side of the issue one's cultural group is on in a debate like the one over climate change. But if one is, well, not a rocket scientist, but someone who has an above-average command of basic science and an above-average ability to make sense of fairly complicated technical and quantitative information, then one necessarily has skills --an ability to search out  supportive evidence, fight off counterarguments, etc.-- that one can use to be even more successful at forming and persisting in group-convergent beliefs.

The survey data reported in the Nature Climate Change study supported this conjecture. The experimental findings in the most recent CCP study -- on ideology, motivated reasoning, and cognitive reflection--supply even more support for it.

Now, the response to the Nature Climate Change study that I have in mind says, "Wait -- you didn't measure climate change literacy! Regardless of their worldviews, if people knew more about climate change science they surely would converge on the best understanding of the risks that climate change poses!"

That response is in fact a non sequitur.

Yes, of course, people who are "climate science literate," by definition, understand and accept the best scientific evidence on climate change. 

The whole point of the study, though, was to test hypotheses about why members of the general public haven't converged on that evidence -- or why, in other words, they aren't uniformly climate-science literate.  

We measured their general science literacy to assess the (widespread) claim that a general deficit in science comprehension explains this particular aspect of confusion about science.  What we found -- that members of the general public who display the greatest general science comprehension are the most culturally polarized on climate change risks -- is flatly inconsistent with that claim.

Imagine we had measured "climate change literacy" instead and used it to predict "climate change risk perception." We would have found that the former predicts the latter quite well -- because in fact, they are, analytically, the same thing.  

But then we'd still be left with the key question -- what explains deficits in "climate science literacy"?  By measuring general science literacy--something that is analytically distinct from climate change risk perception--we were able to help show that one common conjecture about that -- that people are not "climate change science literate" because they can't comprehend basic science -- is inconsistent with empirical evidence.

If one genuinely wants to explain public conflict over climate change, one has to offer and test explanations that don't just amount to redescribing the phenomenon.

And if the goal is to promote public recognition of the best available evidence on climate change -- and other societal risks -- then the sort of science illiteracy we need to remedy relates to our collective ability to protect our science communication environment from the sorts of toxic cultural meanings that make it individually rational for ordinary citizens -- including the most science literate ones -- to pay more attention to what positions on risk say about who they are than to whether those positions are true.


Kahan, D. Why we are poles apart on climate change. Nature 488, 255 (2012).

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

Kahan, D.M., Wittlin, M., Peters, E., Slovic, P., Ouellette L.L., Braman, D., Mandel, G. The Tragedy of the Risk-Perception Commons: Culture Conflict, Rationality Conflict, and Climate Change. CCP Working Paper No. 89 (June 24, 2011).

Ideology, Motivated Reasoning, and Cognitive Reflection, CCP Working Paper 107 (Nov. 29, 2012)

Peters, E., Västfjäll, D., Slovic, P., Mertz, C.K., Mazzocco, K. & Dickert, S. Numeracy and Decision Making. Psychol Sci 17, 407-413 (2006).




Cultural cognition is not a bias -- and the corruption of it is no laughing matter!

Well, I feel sort of bad for coming across as gleeful in reporting that further analysis of data confirmed Indepdendents, just like politial partisans, display ideologically motivated reasoning.  A commentator (Metamorph, aka "Metamorph") called me out on that.

My punishment is to write 500 times ...

1. Cultural cognition is not a bias (parts one and two).

2. It's the science communication environment, stupid -- not stupid people!

3. Cultural cognition is not a bummer (parts one and two).