follow CCP

Recent blog entries
Thursday
Dec272012

Wisdom from Silver’s Signal & Noise, part 1: “Predicting” magnitude 7-8-9 terrorist attacks

So I finished Nate Silver’s Signal & the Noise a couple days ago.  I loved it! 

I
n fact, it managed, very unexpectedly, to sneak in as a late entrant and catch what looked like the sure winner for “my personal favorite book of the year,” Sharon Bertsch McGrayne’s The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy. The two ended up in a dead heat, I’d say  (super honorable mention to George Dyson’s Turing’s Cathedral & James Gleick’s The Information).

McGrayne is the better story teller. Silver's not bad in that regard but the reason he managed to score so high is that his book contains some genuine elements of wisdom. Or at least I thought so!

Over the course of three posts (I’m guessing), I’m going to feature three of the nuggets of insight that I spotted in the book.

The first has to do with the important parallels between earthquakes and terrorist attacks.

As likely everyone—whether or not they’ve read the book—knows, Signal is about the simultaneous indispensability and fallibility of data-driven forecasting. The book essentially uses certain cases as prototypes for what forecasting can and can’t do, when it does and doesn’t work, and why.

Baseball, e.g., works well because there are so much data and the outcomes of interest so straightforward.

Economic forecasting is absurdly bad, in large part because of the bad statistical hygiene of economists, who are theoretically dogmatic, enamored of post-hoc story-telling (they use a statistical tool for this: “overfitting”), and uninterested in iterative calibration of their models.

Another prototype is forecasting of earthquake occurrence.  Pretty much can’t be done: there aren’t enough events to form and test models; the underlying dynamics are likely chaotic—if not technically so, then practically, given the high number of interacting mechanisms; and measurement of those mechanisms is confined to exceedingly crude proxies.

But the kind of forecasting that can’t be done has to do with predicting the timing of earthquakes, particularly ones of significant magnitude.

What can be predicted, however, is the likely scale and frequency of earthquakes in particular locales.

The reason that can be done (and actually, one of the reasons that precise forecasting of timing probably can’t be) is that earthquake magnitude and frequency reflect a “power law” distribution.

In a power-law distribution, there will be a predictable incidence of extreme values—ones many more standard deviations from the mean than you’d ever expect to see in a normal, Gaussian or “bell-shaped” distribution. The classic illustration is height, which is normally distributed, vs. wealth, which reflects a power-law distribution: you’ll never find even one person 20 SDs taller than average (10.5', or so), whereas people who are 20 SDs wealthier than average (net worth of, oh, $2 billion,) or even 100 SDs ($10 billion, more or less), while not commonplace, definitely are out there (for this reason, it is not very meaningful to talk about "standard deviations" in wealth).

Well, if you have enough data to estimate the key parameters of a power law, you can form a reasonably precise estimate of the frequency of extreme values relative to more typical, more common-place ones.

One can do this for earthquakes, it turns out. The Gutenberg-Richter law characterizes the power-law distribution for earthquakes and makes it possible, based on the frequency of relatively “mild” earthquakes to conclude that San Francisco can be expected to have a “major” one (> 6.75 on Richter scale) once ever 30 years, whereas New York will have a major one on average, once every 12,000 years (Signal, p. 150).

This is useful information. It doesn’t tell you when you are likely to have a major earth quake. But it tells you that if you live in San Francisco you are an idiot not to plan for one, whereas if you live in New York, you’d likely be out of your mind to insist that every building be able to able to handle the same impact.

Okay, so Silver goes through all of this. But later in the book he uses this information to say something pretty insightful (to me!) about terrorism.

Terrorist incidents, Silver observes, reflect a power law distribution, too (pp. 430-31). From the number of relatively “small” incidents in a given locale, one can estimate, with a reasonable degree of precision, the expected frequency of progressively “larger” ones.

Could 9/11 have been “predicted”?  As to when & where—probably not (Silver hedges on this; I think he is guilty of a bit of hindsight bias). But as to whether? For sure.

Applying a power-law model with data compiled by Aaron Clauset, Silver concludes that, as of Sept. 10, 2001,  “a September 11-scale attack” could have been expected to occur on average “about once every eighty years in a NATO country” (p. 432), which (by my math) comes out to over a 10% chance in a given decade.

Updating the power-law estimate to incorporate this the new information associated with the occurrence of 9/11 itself, Silver reports that we now have reason to expect a 9/11-scale attack in a NATO country once every 40 years. That’s close to a 25% chance every decade (p. 432).

He also now estimates that there is a “10 percent chance of an attack that would kill at least 10,000 people in a NATO country,” a “3 percent change of an attack that would kill 100,000, and a 0.6 percent chance of one that would kill one million or more” in the next decade (p. 437).

The estimates, like any ones generated by Bayesian techniques, are provisional, and here are appropriately qualified in light of the relative spareness of the data and also various judgment calls about how to define the relevant class of events.

But the point is we know more with this information than we would without it.  Just as the Gutenberg-Richter law supplies information that can help determine the appropriate level of preparedness for earthquakes, this Clauset-Silver terrorism power law supplies us with information we can use to engage in more rational planning for terrorism.

There will (predictably) be many many many times more smaller terrorist incidents than “big” ones—just as there will be many many times more minor than major earthquakes.

But for both quakes as for terrorist attacks, the destructive energy of progressively “larger” ones increases exponentially relative to their diminishing frequency.

We might expect, say, 10 magnitude 6 quakes for every magnitude 7, but the magnitude 7 quake (according to Silver, pp. 437-38) releases 1,000x more energy than each of the 6’s and in the absence of appropriate preparation cause much more damage than all the 6’s combined.

Similarly, in NATO countries, there were scores of terrorist acts smaller than 9/11—a “magnitude 7” attack--in the three decades that led up to it, but the 9/11 death toll was higher than that for all of those combined (p. 438). 

If you have limited resources, it makes sense to invest them in the way that minimizes the expected harm by as great an amount as possible.

San Francisco should be devoting much more resources to bracing for the magnitude 7 quake than it does trying to prepare for the damage associated with the much more frequent but much less damaging smaller ones.

We devote a huge amount of resources to trying to deter or preempt small-scale attacks. Fine. But a blown up commuter-rail train or subway station will likely kill only a few dozen people, and even the downing of a commercial airliner only a few hundred. 

A “magnitude 8” or even “9”-- the release of a chemical or biological agent or the detonation of a nuclear device in an urban area--  has the potential to kill 10,000s or even 100,000s and, while much less likely seems well within the range of power-law range of possibility.

Certainly we should—and I’m sure are-- doing a lot to try to avert the latter sort of attack.  But trying to detect the precursors of such a thing—so that we can intervene and avert it—is very difficult; it is a lot like trying to predict when a magnitude 7 or 8 or 9 earthquake is going to occur.

Nevertheless, we should be preparing for a magnitude 7 or 8 or 9 terrorist attack, in the same way that we do prepare for magnitude 7 and above earthquakes in the places where they can be expected to occur. Cities like San Francisco mandate that structures be built to withstand major quakes and as a result only scores of people die in ones that kill 10,000s in countries like Iran and Armenia.

A civil preparedness capacity to respond to a magnitude 8 or 9 terrorist attack could likewise make a difference in orders of magnitude in the number of people killed by it.

We don’t have such a regime.  Relatively little of the money that the Department of Homeland Security has doled out since its inception have been used for large-scale crisis-response planning.  Moreover, major cities like Los Angeles and New York make what many would regard as only half-hearted efforts to prepare their emergency-response capacities, in part because things like mock evacuations and similar drills tend to be viewed as disruptive and also anxiety-provoking.

Instead, as Silver notes, the post-911 world is pervaded by various forms of “security theatre”— cumbersome screening procedures and goofy alert systems that gratify a public demand for action but are unlikely to contribute anything to public safety.

After the (completely inept) “underwear bomber” was foiled in Christmas 2009, President Obama responded to the media frenzy by upbraiding our intelligence services for “failing to connect the dots,” and “order[ed] an immediate effort to strengthen the criteria used to add individuals to our terrorist watchlists, especially the  ‘no-fly’ list”—a device that experts view as too bloated to be effective and as imposing needless burdens on travelers and security personnel alike.

A guy as smart as Obama doesn't actually need Nate Silver to tell him that you don’t try to improve your ability to pick up a signal by adding noise.

Rather, what he and the rest of us need is a science of science communication that would make as as smart about managing the political-psychology dynamics of risk perception as Silver is about data-driven forecasting.  If we lack the former, the substantial contribution the latter can make to securing the public welfare is wasted.

Tuesday
Dec252012

The black and white -- but mainly gray -- of gun control and drug prohibition, part 2

So ... this is actually part two of a two-part series on race, gun control, and drug criminalization.

Last time I went into the motivation behind the series. The more proximate cause was a question posed in the discussion of my post on legalizing drugs. 

The more remote—but more fundamental—was the unsettling sense I had upon reflecting on my feelings on the Newtown shooting.  The shooting upset me (as it did many, of course). But it upset me, too, to realize that I’m not that upset more or less continuously, because in fact young kids are being shot more or less continuously—not in elementary schools in communities like Newtown, but on street corners & playgrounds in cities like Chicago, Los Angeles and Detroit.

They are essentially part of the War on Drug’s “collateral damage.” And I guess in the same way we don’t worry overmuch about “collateral damage” in the form of deaths to civilians in our other wars, we don’t really get distracted by it here at home. . . .

However much progress one thinks can be made by laws restricting firearms, orders of magnitude more can be made by ending drug Prohibition, which like alcohol Prohibition combines opportunities for monopoly profits with the necessity for violent, extralegal enforcement of commercial obligations, creating a hyper-homicide cocktail potent enough to bend the historical curve of pacifism that is the signature of liberal markety societies.

There are some unconscious emotional dynamics at work here that ought to be exposed and critically examined. They include the partisan cultural meanings of recreational drugs that account for their being treated differently from alcohol.  They include, too, the predictably parochial nature of our empathic sensibilities, which lead us to attend selectively to forms of suffering and loss that are in fact universal in their nature and concentrated disproportionately on certain members of our society.

But as always, matters are complicated.

One of the complexities is the profound moral ambivalence that members of inner-city, largely minority communities themselves have toward drug criminalization.  That was the subject of the my last post, which described the powerful case that members of those communities often make for their entitlement to use the expressive power of the law as a countervailing force to oppose social influences—including commercial advertising of lawful products--that they believe unfairly diminish their power to instill in their children the dispositions, tastes, and habits that they (like all parents everywhere) perceive to be essential to their children’s’ prospects for living happy, flourishing lives.

Precisely because the impact of our policies on guns and drugs so profoundly and distinctively affect these communities, the power of their members to make law in a manner that relfects their judgments about the morally complicated, empirically uncertain issues involves necessarily goes straight to the core of their entitlement to enjoy meaningful self-government.

I don’t know what this element of the dilemma means for how political disputes over guns and over drugs should be resolved. If I said I did, I’d be contradicting myself (likely I already have in some place and in some way connected to this topic; only someone who doesn’t actually get the complexity involved will manage to get all the pieces to fit together in an analytically neat and tidy little pattern).  My point is just that this is an element of the dilemma that has to be taken into account; and for that reason, I want to take it into account by acknowledging that it complicates the argument I have been making about the mistaken overemphasis on regulating guns relative to deregulating drugs as a response to our Nation’s unacceptable gun-homicide rate. . . .

So, finally, some data (I’m sorry for going on; I should just accept that it is impossible ever to say enough and not keep trying).

It’s not a perfect measure of the ambivalence that the communities I’m describing feel, but what I’ve done, combining data from several CCP studies conducted over the last year, is break down attitudes on gun control and marijuana legalization by race. 

The sample consists of about 2500 individuals drawn from a nationally representative on-line sample of individuals recruited to participate in CCP studies of one sort or another.  I wanted to cobble together several data sets in order to get enough African-American sample to enable reasonably precise estimates; African-Americans make up 11% of the sample overall.

Start with this:

It shows that African Americans and whites in the sample held comparable attitudes overall toward marijuana legalization—about 60% support, actually—but were divided sharply on gun control, with African Americans decidedly more supportive. 

I’ve put Hispanics and Asians in, too, but realize that they (especially Asians) comprised an even smaller part of the sample, diminishing the precision of any estimate of the attitudes of these groups in the general population (the error bars are standard errors; double the interval, essentially, if you want 0.95 levels of confidence).  But we are on relatively solid ground for African Americans and whites.

Now to do a meaningful comparison of African American and whites, it is also useful to take account of ideology.  Support for gun control and for legalizing marijuana both are associated with being liberal (as that term is used in ordinary political discourse). African Americans are, on average, much more liberal than whites.  Can we see the racial difference, then, as reflecting the greater ideological diversity in the white population?

To answer that question, I think it is helpful to start by getting a sense of just how much more liberal African Americans are than whites.  Consider:

What I’ve done, essentially, is plot the distribution (with a kernel density estimate) of whites and African Americans respectively over a continuous measure of right-left political orientation formed by aggregating the subjects’ self-reported identification with the Democratic and Republican parties and their ideology on a “liberal-conservative” scale.

You can “see” how much more “liberal Democratic” African-Americans are. Whereas the mean score on the scale for whites puts them just shy of the middle of the “Liberal Democratic” scale, the mean for African-Americans is closer to +1 Standard Deviation toward “Liberal Democratic. Minus 1 SD in the distribution for African-Americans equates to being around the middle of the scale; minus 1 SD for whites is a little more than one SD toward "Conservative Republican." 

Basically, “normalizing” the scale for race, one would say that a “middle of the road” African American is as far right within his or her group as a “moderately conservative Republican” white is for his or hers. 

Now consider this:

 

These are “fitted values” based on regression analyses (click on the thumbnail if you'd like to see them) that treat race, “liberal Democratic” political orientation, and the interaction of those two as predictors for support for gun control and support for marijuana legalization, respectively.  They tell you, in effect, how a progressively more “liberal Democratic” orientation influences positions on these issues for both groups.  I’ve started plotting the African-American line at the middle of the “liberal Democratic” scale because the proportion of African Americans more “conservative Republican” than that is so small—and it is visually misleading, I think, to include them if one is trying to assess how important ideology is in explaining the difference in white & African-American views on drugs & on guns.

What you can see, in effect, is that African Americans are more supportive of gun control, and less supportive of marijuana legalization, than whites of comparable political outlook. 

The effect is most pronounced for marijuana legalization. There, political outlooks and race interact, signifying that the impact of becoming more liberal and Democratic on support for legalization is smaller for African Americans than for whites.  There is no similar interaction between political outlooks and race for gun control; in effect, African Americans are uniformly more pro-gun control than whites at any given point on the political outlook spectrum.

This was basically in line with what I expected. That is, based on what I know of the public opinion research, I expected African Americans to be more pro-gun control and less pro-legalization than whites, and in fact anticipated this effect would be more pronounced for marijuana legalization.

Still, I was surprised by the degree of support for legalization among African Americans generally.  The level of support is higher, I think, than it was in years past. And the differential between African Americans and whites—in relation to political outlooks--smaller than I expected.

I’ll have to ask Tracey Meares, who has studied this, whether I’m right in my recollection/expectation and what she thinks about these results (maybe I can even get her to offer her views on the blog!).

What does this add to the discussion of gun control and drug prohibition?

Information. 

Information, too, that is only an imprecise indicator of what are in fact very very important differences in perspective, founded on differences in experience and cultural meaning, among citizens who need to deliberate on guns and drugs. 

You should try to learn as much as they can about the sources of these differences, and about the judgments of fact and value that underlie them.  If you don't do this, your positions on gun control and drug prohibition will not be well-considered, no matter what they are.

 

 

Monday
Dec242012

The black and white -- but mainly gray -- of gun control and drug prohibition, part 1

This is part one of a two-part series on race, gun control, and drug criminalization.

The motivation for the series grows out of discussion relating to my post on legalizing drugs (I’m simplifying; really, something like this) as a means of reducing the gun-homicide rate in the U.S.

A question was posed in the discussion about what African-American public opinion is on drugs and guns.  I’ve culled some data on that from past CCP studies.  I’ll present these data in the next post. In this one, I want to lay out some more background. . . .

One of the reasons I made the drug-legalization proposal was my sense of discomfort with the post-Newtown framing of the gun-homicide crisis in our country. There are (literally) orders of magnitude more children (adults too) being shot & killed in inner-city neighborhoods in cities like Chicago than are being shot & killed in mass shootings by mentally ill people. It’s unclear how effective gun control—of any sort—would be in reducing gun homicides of any type.  But it’s clear that a very large fraction of U.S. gun homicides (leaving out suicide, which accounts for more than half of U.S. gun deaths) are associated with the lethal violence incident to the black market status of recreational drugs like marijuana and cocaine.

Why the hell aren’t we talking about this? Why haven’t we been for years already, given the steady, relentless toll in death (not to mention ruined lives, wasted dollars, etc.) associated with our completely dysfunctional, idiotic drug-law regime?

Well, actually, some people do talk about our drugs laws—and in particular whether their should be legalization—quite a lot. Many of them live in the very areas where drug-related, including drug-law-related, violence is the highest.  And they aren’t—or certainly aren’t all—persuded that drug legalization is the way to go.

My brilliant colleague Tracey Meares first educated me on these issues back when we were both assistant professors (we had to clean the chalk boards after Richard Epstein’s & Cass Sunstein’s lectures; sometimes this kid from the neighborhood, Barack Obama, would help too) at the University of Chicago Law School.

She told me, and showed me with both research and with vivid examples from the neighborhoods right around us, that there was a profound ambivalence about drug criminalization in inner-city, largely African American communities in the South Side of Chicago. 

There was no ambivalence about the existing drug-law enforcement posture of both the state and federal governments: everyone recognized that basically disposing of the lives of people involved in the drug trade by just tossing them in jail for huge amounts of time was idiotic, as well as indescribably cruel both to them and their communities.

But it wasn’t at all the case that the citizens who were most adversely affected by this dumb policy all thought that drugs should be legal. On the contrary, most of them were deeply opposed to legalization on moral grounds. Moral not primarily in the illiberal sense of wanting to tell other adult people how to live their lives, but moral in the sense of wanting the same fair opportunity that all adults are entitled to have to mould the characters of their children, instilling in them the outlooks and dispositions and habits most conducive to their happiness and flourishing.

All parents compete in that shaping process, of course, with myriad other influences.  These include commercial, market-driven ones that try to conjure demand for useless or even pernicious products by inventing fanciful stories about the glamorous or virtuous ways of life they conduce to (think, as you might have been recently, about hideous  “man cards”; but think, too, of Virginia Slim, etc.).

Parents in the communities in the inner-city in the South Side of Chicago (at least in the early to mid-1990s, when I lived there) resented these forces, like all parents do. They hated the companies that plastered their communities with billboards and other forms of advertising targeted (they correctly perceived) at creating in their children a taste—emotional and physical—for products like “malt liquor” and cigarettes.

They worried that if recreational drugs were made legal, manufacturers of those products would join the ranks of commercial firms poisoning the moral environment in which they were raising their children.

The way they—or many of them; reasonable, reasoning citizens inevitably have differences of views on complex matters—tried to reconcile their support for drug criminalization with their opposition to the existing regime of drug-law-enforcement was by trying to reform the former. Organizing through the processes associated with the City’s innovative community policy program, they advocated things like “gang loitering” laws as alternatives to draconian criminal sentences.

I wasn’t sure that I agreed with all the positions members of these communities took on all these issues, particularly on the criminalziation of drugs (the gang-loitering law, in context, did seem quite sensible to  me).  

But one thing I did become convinced of was that they were better situated, in every relevant sense, to figure out how these various issues should be resolved in their lives than I was.  They hadn’t necessarily read the latest issue of the Journal of Criminal Law and Criminology but by seeking out information from people they trust, and employing their informed sense of who knows what about what, they had arrived –by the same means we all do, in all areas of our lives—at perfectly justifiable grounds for making complicated decisions.

Moreover, they were the ones whose lives those decisions, no matter who made them, were going to most meaningfully affect. They were the ones bearing all the relevant costs—in the form of the impact that whatever regime of laws and enforcement strategy would be adopted would have on their sons and daughters, their brothers and sisters, and nephews and nieces, and close friends and neighbors.

There’s no way to honor the dignity that reasoning people are due under these circumstances other than to respect their right to govern themselves.

... What were we talking about again?  Oh, drug laws and guns.

Another thing that people in these communities believed, very strongly, was that there should be significant regulation of guns.  Two weeks ago, the U.S. Court of Appeals for the Seventh Circuit told them that’s not an area in which they are entitled to govern themselves: relying on the U.S. Supreme Court’s recent Second Amendment decisions (including one, also involving Chicago, that reached the by-no-means obvious determination that the Second Amendment limits the power of state governments, and not merely that of the federal one) struck down an Illinois law that prohibits carrying a concealed handgun in public without a permit issued for “cause.”

I’m very doubtful that “tough gun laws”—which like prohibition of alcohol and drugs create black markets and the forms of violence that inevitably attend them—will contribute much to solving the problem of violent homicide in our country.  But I don’t have any doubt that people in places like Chicago have a right to figure that out for themselves.

Okay. . . Next time: some data.

References 

Fung, A. Empowered participation : reinventing urban democracy. (Princeton University Press, Princeton, N.J.; 2004).

Kahan, D.M. & Meares, T.L. The Coming Crisis of Criminal Procedure. Geo. L.J. 86, 1153 (1998).

Meares, T.L. It's a Question of Connections. Val. U. L. Rev. 579 31, 579 (1997).

Meares, T.L. Social Organization and Drug Law Enforcement. Am. Crim. L. Rev. 35, 191 (1998).

Meares, T.L. & Kahan, D.M. Urgent times : policing and rights in inner-city communities. (Beacon Press, Boston; 1999).

Skogan, W.G. Police and community in Chicago : a tale of three cities. (Oxford University Press, Oxford ; New York; 2006).

Skogan, W.G. & Hartnett, S.M. Community Policing Chicago Style. (1997).

Sunday
Dec232012

Gateway post ...

Hey, now that you are thinking about how consequential our mindless drug laws are for our unacceptable gun homicide rate, you should try the really good stuff ...


Friday
Dec212012

Catch up on commentary day

Lots of interesting comments on the "Less drug laws, less gun homicide" & "Mass opinion coherence & do psychologists tell more tall tales than political scientists?" posts, including some on latter from Stats Legend Andrew Gelman (who also responds w/ post at his own site).

Much more interesting than anything I had planned to say today -- plus I'd like to devote my own "blog time" today to re-reading & responding to them.

Thursday
Dec202012

Do mass political opinions cohere? And do psychologists "generalize without evidence" more often than political scientists?

Stats Legend Andrew Gelman (whose blog everyone who enjoys being surprised and who values high-quality analytical thinking should read daily) has an interesting post on Steven Pinker.

Pinker asks “[w]hy, if you know a person’s position on gay marriage, can you predict that he or she will want to increase the military budget and decrease the tax rate,” a question he answers by observing that “[p]olitical philosophers have long known that the ideologies are rooted in different conceptions of human nature — a conflict of visions so fundamental as to align opinions on dozens of issues that would seem to have nothing in common.”

Gelman responds by (1) doing some quick GSS correlations, on the basis of which he concludes that “attitudes on such diverse issues are not so highly correlated”; and then (2) attributing Pinker’s error to Pinker’s being a psychologist rather than a political scientist and thus prone to “present[ing] ideas that are thought-provoking but . . . too general to quite work,” in contrast to political scientists who “take such ideas and try to adapt them more closely to particular circumstances.”

Some thoughts:

1. Pinker is clearly right to note that mass political opinions on seemingly diverse issues cohere, and Andrew, I think, is way too quick to challenge this.

I could cite to billions of interesting papers, but I’ll just show you what I mean instead. A recent CCP data collection involving a nationally representative on-line sample of 1750 subjects included a module that asked the subjects to indicate on a six-point scale “how strongly . . . you support or oppose” a collection of policies:  

  1. policy_gun  Stricter gun control laws in the United States.
  2. policy_healthcare  Universal health care.
  3. policy_taxcut  Raising income taxes for persons in the highest-income tax bracket.
  4. policy_affirmative action  Affirmative action for minorities.
  5. policy_warming  Stricter carbon emission standards to reduce global warming.

 

 Positions clustered on these “diverse” items big time. The average inter-item correlation was 0.66. The Cronbach’s alpha—a scale reliability measure based on item covariance and the number of items—was 0.91.

This is a degree of coherence that would  make any social scientist – psychologist or political scientist – beam. The highest possible alpha is 1.0, and anything above 0.70 is usually regarded as signifying a high degree of reliability.  Low reliability, measured in this way, is it’s own punishment, since it constrains the power of any sort of explanatory or predictive model involving the scale. With a score of 0.91 you can be confident that the power of your model won’t be dissipated by the noise associated with the imprecision of the observable "indicators" you are using to measure the latent variable. 

The latent variable being picked up by these policy items is obviously something akin to right-left political preferences, so let’s call the resulting measure “Liberal_policy.” (Additional items cohered better with each other than with these, forming a second "libertarian policy prefernce" scale; but let's keep things simple.)

Being able to form a scale like this with a general population sample is pretty good evidence in itself (and better than just picking two items out of GSS and seeing if they correlate) that people’s opinions on such matters cohere.

But just to make the case even stronger, let’s consider how much of the variance in liberal policy preferences can be explained by ideology. 

In the same data set, there was a five-point measure for self-described “liberal-conservative ideology” and a  seven-point one for identification with the two major political parties. Those two items were also highly correlated (r = 0.70), so I combined them into a scale (α = 0.82) coded to represent a right-wing ideological disposition, which I labeled “Conserv_repub.”

Regressing Liberal_policy on Conserv_repub, I discovered that the percentage of variance explained (R2) was 0.60. That’s high, as any competent psychologist or political scientist would tell you, and as I’m sure Andrew would agree!

Now Andrew noted that the degree of coherence in political preferences tends to be conditional on other characteristics, such as wealth, education, and political interest. Typically, political scientists use a “political knowledge” measure to assess how coherence in ideological positions vary.

I had a measure of that (a 9-item civics-test sort of thing) in the data set too. So I added it and a cross-product interaction term to my regression model. It bumped up the R2 – variance explained – by 4%, an increment that was statistically significant.  

Seems small, but how practically important is that? A commenter on Andrew’s blog noted that I tend to criticize fixating on R2 as an effect-size measure; my point, which is one that good social scientists—political scientists and psychologists! Andrew too!--have been making for decades is that R2 is not a good measure of the practical significance of an effect size, a matter that has to be determined by use of judgment with relation to the phenomenon at issue.

Well, to help us figure that out, I ran a Monte Carlo simulation to generate the predicted probability that a typical “Liberal Democrat” (-1 SD on Conserv_Repub) and a typical “Conservative Republican” (+1 SD) would support “stricter gun control laws” (seems topical; this is pre-Newtown, so it would be interesting to collect some data now to follow up), conditional on being “low” (-1 SD) or “high” (+1 SD) in political knowledge.

Seems (a) like variance in political knowledge (whatever its contribution to R2) can matter a lot – the probability that a high–political-knowledge Republican will oppose gun control is a lot lower than that for a low–political-knowledge one—but (b) there is still plenty of disagreement even among low–political-knowledge subjects. 

I’d say, then, that Andrew is being a bit too harsh on Pinker’s premise about political preference coherence.

2. Pinker is clearly wrong—not just in his answer but in his style of reasoning—to connect this sort of coherence to “different conceptions of human nature" among people of opposing ideologies

Pinker, however, is indeed doing something very objectionable: he is engaged in rank story-telling

He notes that political philosophers identify ideologies with different conceptions of “human nature,” a “conflict of visions so fundamental as to align opinions on dozens of issues.” Well, maybe political philosophers do do that. But the idea that “different conceptions of ‘human nature’ ” explain coherence and variance in mass political opinion is an empirical claim, and as far as I know there’s not any support for it. 

I think it’s almost certainly false. Measures of ideology of the sort that I have used here have not – as far as I know; please do tell me if I’m wrong: the pleasure of learning something new will more than compensate me for the embarrassment of being shown to be ignorant -- been validated as predictors of “different conceptions of human nature.” Indeed, I think the idea that ordinary members of the public have “conceptions of human nature” is extravagant—the sort of thing only someone who has never ventured outside a university campus would likely believe.

There are myriad theories about the puzzling question of how ordinary people, who really aren’t philosophers, aren’t that interested in politics, and who are very consumed with other things can manage to form coherent ideological preferences. And they’ve been tested empirically.

It’s irritating for anyone who is familiar with all that work to see Pinker advance the sort of claim he does—which he presents not even as a conjecture but as a simple, unqualified, fact-of-the matter report.

3. Pinker’s mistake is one psychologists would resent as much as political scientists.

The sort of thing Pinker is doing here generalizes.  Popular commentators love to reach into the grab bag of decision science mechanisms and construct just-so stories that purport to “explain” complicated phenomena (e.g., political controversy over climate change). 

Good social scientists hate this.  Indeed, Pinker generally doesn’t like it, in fact; he complains about this practice in his excellent book, The Better Angels of Our Nature: Why Violence Has Declined, which admirably tries to connect trends in violence over history to mechanisms that themselves have support in evidence. I find it sort of deflating to see that he seems to adopt a different approach in the writing he does for the New York Times.

But the point is, resentment of story-telling is something that psychologists and political scientists would both experience. It’s not a consequence of Pinker being a psychologist!

4. Ironically, Andrew is making the sort of mistake he says Pinker made.

This last point follows from all the others. Andrew sees Pinker doing something irritating, and then treats a conjecture (I think a pretty uninteresting, implausible one; but all conjectures are created equal – test away!) as a general law that explains this particular instance, etc.

But now I will offer a conjecture, based on an observation-grounded theory.

The observation-grounded theory is that Andrew Gelman has a virtuous Bayesian disposition. That is, he is the sort of person who very happily updates and revises his views, which he always regards as just provisional estimates anyway.

The conjecture: that Andrew, on reflection, will agree that he offered a poor diagnosis (“psychologists generalize without evidence, unlike political scientists, who look for concrete evidence in particulars!”) of Pinker’s objectionable style of argumentation here (which, again, strikes me as uncharacteristic of Pinker himself!).

And now, let’s collect some evidence.

(One more prediction, or hope: Andrew will like my graphic!)

p.s. Ideological coherence in policy prefernces isn't nearly as interesting -- nearly as surprising, puzzling --as ideological or cultural coherence in factual beliefs (e.g., "earth is/is not heating up" & "children of gay & lesbians do worse/no worse in life than ones raised by heterosexual parents." That's what CCP research is all about. Perhaps I'll do another post on that.

References

Abelson, R.P. A Variance Explanation Paradox: When a Little is a Lot. Psychological Bulletin 97, 129-133 (1985).

Delli Carpini, M.X. & Keeter, S. What Americans Know About Politics and Why It Matters. (Yale University Press, New Haven; 1996).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

John, O.P. & Benet-Martínez, V. in Handbook of research methods in social and personality psychology. (eds. H.T. Reis & C.M. Judd) 339-369 (Cambridge University Press, New York; 2000).

King, G. How Not to Lie with Statistics. Am. J. Pol. Sci. 30, 666-687 (1986). 

King, G., Tomz, M. & Wittenberg., J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am. J. Pol. Sci 44, 347-361 (2000).

Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).

Tuesday
Dec182012

Actually, empirical evidence suggests a sure fire way to dramatically lower gun homicides: repeal drug laws

Sticking my umbrella up to try to deflect the tsunami of specious, beside-the-point, hiding-the-cultural-imperialism-ball, insult-to-reason uses of empirical evidence on violence & gun control (emanating from those on both sides of the debate), I offered a couple of feeble blog posts (here and here) urging more thoughtful, grounded arguments that are mindful both of the limits of our knowledge and our duty, as citizens of a liberal democracy, to justify our policy positions on grounds free of the impulse to use law as an expressive weapon for denigrating cultural styles that differ from our own.

I'm not satisfied by my performance.

The problem isn't that what I wrote will make no difference, have no effect, etc. If that were the decisive issue, why would I ever say anything? I'm not trying to change the world; I'm just trying to engage other curious and concerned citizens who also want to think about issues reflectively and form positions that they think are factually supportable and morally defensible. I'm trying to help them so they'll help me back, since I'm by no means certain I'm right either.

No, the problem wasn't the futility of what I said but the incompleteness. "Don't look to statistics" -- could be read as denigrating the utility of empirical inquiry in assessing public policy and as expressing a sort of nihilistic "who cares, nothing we can do!" attitude, both of which would deeply misrepresent how I feel about evidence-based policymaking generally & about using evidence to think about guns.

So to amend I will emend.  

I now want to point out that in fact, while the empirical evidence on the relationship between gun control and homicide is (at this time at least) utterly inconclusive, there certainly are policies out there that we have very solid evidence to believe would reduce gun-related homicides very substantially.

The one at the top of the list, in my view, is to legalize recreational drugs such as marijuana and cocaine.

The theory behind this policy prescription is that illegal markets breed competition-driven violence among suppliers by offering the prospect of monopoly profits and by denying them lawful means for enforcing commercial obligations.

The evidence is ample. In addition to empirical studies of drug-law enforcement and crime rates, it includes the marked increase in homicide rates that attended alcohol prohibition and the subsequent, dramatic deline of it after repeal of the 18th Amendment.

from Claude Fischer, Berkeley Blog Actually, it's pretty interesting to look at homicide rates over a broader historical time frame than typically is brought into view by those who opportunistically crop the picture in one way or another to support their position for or against gun control.  What you see is that there is a pretty steady historical trend toward decline in the US punctuated by expected noisy interludes but also by what appear to be some genuine, and genuinely dramatic, jumps & declines.

One of the jumps appears to have occurred with the onset of prohibition and one of the declines with repeal of prohibition.  Social scientists doing their best to understand the evidence generally have concluded that that those are real shifts, and that they really were caused by prohibition and repeal.  

from Claude Fischer, Berkeley Blog

Criminologists looking at the impact of drug prohibition can use the models developed in connection with alcohol prohibition and other modeling strategies to try to assess the impact of drug prohibition on crime. Obviously the evidence needs to be interpreted, supports reasonable competing interpretations, and can never do more than justify provisional conclusions, ones  that are necessarily subject to revision in light of new evidence, new analyses, and so forth.

But I'd say the weight of the evidence pretty convincingly shows that drug-related homicides generated as a consequence of drug prohibition are tremendously high and account for much of the difference in the homicide rates in the U.S. and those in comparable liberal market societies (the non-liberal, non-market societies all are burdened with homicide rates orders of magnitude higher; guns don't explain that--  the pacifying influence of doux commerce does).  By all means decide for yourself, though; I've cited some reading material at the end of this post & urge others to call our attention to more in the comments section.

There are obviously other spikes & dips in the "secular" (as econometricians would say; here that's a very nice adjective to use) downward trend in homicide in the U.S. E.g., the upsurge of homicides in the 1960s and the decline in the 1990s. Most of the gun-control combatants mine this period for support for their claims -- they should, since the data are rich with support for specious inference.  Scholarly discussion here recognizes that the evidence on the contribution of guns to these jumps is utterly, hopelessly inconclusive.

There is a very interesting empirical study, though, by economist Jeffrey Miron, who concludes that the available evidence is consistent with the hypothesis that the difference in homicide rates in the US and in other liberal market societies is attributable to our drug prohibition policies. Gun availability in the US, according to this hypothesis, doesn't directly account for the difference in homicide rates between the US and these countries; rather, gun availability mediates the impact between drug prohibition and homicide rates in the US, because the criminogenic properties of drug prohibition create both a demand for murder of one's competitors and a demand for guns to use for that purpose.  

One of the very nice things about Miron's analysis, too, is that he is appropriately provisional about his conclusions:  

The empirical results presented above provide a possible explanation for the large differences in violence rates across countries, and they suggest that previous analyses might have spuriously attributed these differences to gun control or availability. According to the analysis here, differences in drug prohibition enforcement explain differences in violence, which in turn explain differences in gun ownership that correlate positively with violence but do not cause that violence. Further, the results provide a hint that restrictive gun control regimes can themselves increase violence. As noted above, these results should be considered suggestive rather than conclusive. Future research on these issues will need to exploit time‐series rather than cross‐sectional data.

 That's what a real scholar sounds like, you see. In my view, it's what an open-minded citizen sounds like too.

Now one thing to note: Obviously, decriminalizing marijuana and cocaine couldn't be expected to prevent mass shootings like the one in Newtown, or Aurora, or Phoenix, or Columbine, etc. (Maybe there'd be fewer guns around, actually, if we didn't have the demand for them associated with their contribution to the illegal drug trade, but there are already so many around -- the gun & people populations are neck-and-neck -- that I think disturbed people would still have no trouble getting their hands on them.)

But here's another thing to note: these very sad incidents "represent only a sliver of America's overall gun violence." Those who are appropriately interested in reducing gun homicides generally and who are (also appropriately) making this tragedy the occasion to discuss how we as a society can and must do more to make our citizens safe, and who are, in the course of making their arguments invoking (appropraitely!) the overall gun homicide rate should be focusing on what we can be done most directly and feasibly to save the most lives.

Repealing drug laws would do more --  much, much, much more -- than banning assault rifles (a measure I would agree is quite appropriate); barring carrying of concealed handguns in public  (I'd vote for that in my state, if after hearing from people who felt differently from me, I could give an account of my position that fairly meets their points and doesn't trade on tacit hostility toward or mere incomprehension of  whatever contribution owning a gun makes to their experience of a meaningful free life); closing the "gun show" loophole; extending waiting periods etc.  Or at least there is evidence for believing that, and we are entitled to make policy on the best understanding we can form of how the world works so long as we are open to new evidence and aren't otherwise interfering with liberties that we ought, in a liberal society, to respect.

...Now, what other policies might help? And in particular, if we are concerned about deaths of children?   Well, there's swimming pools .... But I've said enough for now.

References

Barnett, R. & Trip, B. Drug Prohibition and the Weakness of Public Policy’(1994). Yale LJ 103, 2593. 


Husak, D. Legalize This! The Case for Decriminalizing Drugs. (Verso, New York; 2002).

Jensen, G.F. Prohibition, Alcohol, and Murder Untangling Countervailing Mechanisms. Homicide Studies 4, 18-36 (2000).

Kleiman, M. Marijuana : costs of abuse, costs of control. (Greenwood Press, New York; 1989).

Kleiman, M., Caulkins, J.P. & Hawken, A. Drugs and drug policy : what everyone needs to know. (Oxford University Press, Oxford ; New York; 2011).

MacCoun, R.J. & Reuter, P. Drug war heresies : learning from other vices, times, and places. (Cambridge University Press, Cambridge, U.K. ; New York; 2001).

Miron, J.A. & Zwiebel, J. The Economic Case Against Drug Prohibition. The Journal of Economic Perspectives 9, 175-192 (1995).

Miron, J.A. Violence, Guns, and Drugs: A Cross‐Country Analysis*. Journal of Law and Economics 44, 615-633 (2001).

 Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).

 

Monday
Dec172012

Statistics: the monopoly money of the gun-debate marketplace of ideas

This is what makes people cynical about empirical arguments in policy debate. Recognize when the data are inconclusive, or else no one will be able to recognize what counts as sound evidence.

If you are contributing to this devaluation of the currency of reason, just stop. In particular, stop insisting that everyone who disagrees with you on facts is either an "idiot" or a "liar."

Look: There are lots of good, thoughtful arguments to be made here, ones based on value and ones based on the best factual surmises we can make based on experience and sense.  

These are arguments that citizens in a liberal society can advance openly, and should, to confirm, for themselves and for others, that their positions aren't motivated by the illiberal ambition to denigrate those whose cultural identities differ from their own. Reliance on one-sided, cherry-picked empirical arguments doesn't furnish that assurance; on the contrary, it's the "smoking gun" of cognitive illiberalism.

 

 

Saturday
Dec152012

"More statistics, less persuasion": the gun control debate continues, and continues to miss the point...

I was planning to write something about Moore v. Madigan, No. 12-1269 (7th Cir. Dec. 11, 2012),  the U.S. Court of Appeals decision earlier in the week that overturned an Illinois law that more or less bans ordinary citizens from carrying concealed weapons in public.

Judge Richard Posner, our century's Oliver Wendell Holmes Jr., wrote the opinion.  Not surprisingly, it's a good read.

Posner irreverently mocks any attempt to wring meaning from the "original intent" of the Second Amendment (Posner is now engaged in a very unseemly but entertaining pissing match with Justice Scalia, in which both have basically said "liar, liar pants on fire!" about Posner's New Republic review of Scalia's grammar-school quality text on  interpretation).

He also ruthlessly --nihilistically, even; Posner, like Holmes, exudes a nihilistic philosophy & style-- arrays against one another conflicting empirical studies on the impact of concealed-weapons laws on violent crime (uncharacteristically, Posner misses a supporting citation: to an "expert consensus" report from the National Academy of Sciences, which declares such evidence utterly inconclusive).

Posner concludes:

A gun is a potential danger to more people if carried in public than just kept in the home. But the other side of this coin is that knowing that many law-abiding citizens are walking the streets armed may make criminals timid. Given that in Chicago, at least, most murders occur outside the home, Chicago Police Dep’t, Crime at a Glance: District 1 13 (Jan.–June 2010), the net effect on crime rates in general and murder rates in particular of allowing the carriage of guns in public is uncertain both as a matter of theory and empirically. “Based on findings from national law assessments, crossnational comparisons, and index studies, evidence is insufficient to determine whether the degree or intensity of firearms regulation is associated with decreased (or increased) violence.” Robert A. Hahn et al., “Firearms Laws and the Reduction of Violence: A Systematic Review,” 28 Am. J. Preventive Med. 40, 59 (2005); cf. John J. Donohue, “The Impact of Concealed-Carry Laws,” in Evaluating Gun Policy Effects on Crime and Violence 287, 314–21 (2003). “Whether the net effect of relaxing concealed-carry laws is to increase or reduce the burden of crime, there is good reason to believe that the net is not large…Based on available empirical data, therefore, we expect relatively little public safety impact if courts invalidate laws that prohibit gun carrying outside the home, assuming that some sort of permit system for public carry is allowed to stand.” Philip J. Cook, Jens Ludwig & Adam M. Samaha, “Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective,” 56 UCLA L. Rev. 1041, 1082 (2009)....

In sum, the empirical literature on the effects of allowing the carriage of guns in public fails to establish a pragmatic defense of the Illinois law. Bishop, supra, at 922–23; Mark V. Tushnet, Out of Range: Why the Constitution Can’t End the Battle over Guns 110–11 (2007). Anyway the Supreme Court made clear in Heller that it wasn’t going to make the right to bear arms depend on casualty counts. 554 U.S. at 636. If the mere possibility that allowing guns to be carried in public would increase the crime or death rates sufficed to justify a ban, Heller would have been decided the other way, for that possibility was as great in the District of Columbia as it is in Illinois.

So ... I was planning to write something along the lines, "the statistical debate has been won by the gun-rights team, which has (a) effectively fought the statistical battle to a standstill & (b) succceeded in getting courts to impose a liberty-preserving standard of proof on those who want to restrict guns. Point (b) -- who should bear the burden of proof -- is an interesting question in a liberal society, yet people will still go on and on about statistics blah blah."  Or some such.

But now this... And I don't know what to say. Except that statistics really are beside the point.

They are beside the point because they are genuinely inconclusive.

They are beside the point because they genuinely don't engage what psychologically motivates people's positions here.

And they are beside the point because they ignore the real moral issues, which are ones of social meaning:

  • What does it say about what people value when they want to own a cache of military-style armaments such as a "Glock," a "SIG Sauer handgun," and a "Bushmaster .223-Caliber Assault rifle"? (These were weapons owned, apparently, by the shooter's mother, his first victim, who was incorrectly reported to be a kindergarten teacher at the school where the shooting spree took place; guess she was also an avid "self-defense" enthusiast?)   
  • What does it say about us when we permit other people to make money -- lots of it -- satisfying the appetite of ordinary people to own the tools of the trade of those whose profession is killing?

  • Yet what would it say about us if in saying this with our law we couldn't also acknowledge that owning various types of guns are also, for many, enmeshed with a host of very different cultural meanings--ones relating to personal virtues like self-reliance, honor, and responsibility; and to social roles and practices that intimately connect them to people, past and present, whom they do value, and very appropriately so?
  • What would it say about our commitment to liberal principles if those of us who don't belong to communities in which guns have these meanings (and in fact are puzzled by them) refused to acknowledge (or just couldn't see; the two are connected) that those who do belong to them understandably see many kinds of gun control as expressing hostility and contempt for their values? Understandably, because in fact, many (not all!) of those who advocate gun control are motivated by (or are simply profiting from) exactly that sort of ugly, illiberal sentiment?

This is a conflict of cultural meaning that must be negotiated by law. And it should be negotiated in a way that is consistent with liberal political principles, which impose on citizens of diverse understandings the duty to show they are committed to accommodating one another, and to resisting making use of laws (whether handgun bans or "stand your ground" provisions) as expressive symbols of dominance over one another.

This is complicated.  Much more complicated than the convoluted (and utterly inconclusive) multivariate regression analyses with which those involved in the "statistics" debate beat each other over the head.

And more complicated than simply mocking those who think statistics (or "history") can solve this issue.

We don't need nihilism in our public discourse. We need genuine liberal statesmanship.

We don't need a Holmes. We need a Lincoln.

Friday
Dec142012

A multivariate regression analysis of CRT performance & ideology, plus a preliminary diatribe against mindlessly overspecified regression models

My most recent paper, Ideology, Motivated Reasoning, and Cognitive Reflection, reports data on the relationship between ideology and the disposition to use high-level, "System 1" information processing, as opposed to intuitive, low-effort, heuristic-drive "System 2," as measured by the Cognitive Reflection Test (CRT).  

I report that there really isn't any. That's sort of surprising in light of all the attention being paid to the neo-Authoritarian Personality literature, which asserts that conservativism is characterized by closed-mindedness, aversion to complexity, and the like.

I know reviewers will want to know, "but if one controls for ..." so I've prepared a multivariate regression that includes ideology along with various other individual characteristics (gender, race, education, income, and religiosity) that have been shown to correlate with CRT scores.  Still nothing!

See for yourself by clicking on the thumbnail to the left.

But in fact, I view this analysis as pretty close to worthless -- if one is really trying to figure out if there is an association between ideology and cognitive reflection.  The idea of  "controlling for" these sorts of characteristics in order to measure the "independent" impact of "ideology" is nonsense: it treats "ideology" as some sort of disembodied "essence" inside of people, when in fact it is one facet of an integrated package of attributes that cohere with one another and are all indicators of a latent type of shared identity or style.

I've been planning for a while to go ballistic on the idea of "overspecified" regressions -- and what sort of mistake in thinking (or failing to think, really) they reflect. This isn't really the right vehicle for getting the point across, so I anticipate coming back to this topic at some point.

But as a preview, here is a short text that I prepared to go with this genuinely fantastic multivariate regression analysis of ideology & CRT.  I'm sure the note will be revised -- or dropped altogether -- before I submit the paper for publication:

Finally, a multivariate regression model that included all of these predictors was tested. That analysis can be seen as assessing the effect of ideology on CRT scores “controlling for” gender, race, education, income, and religiosity. It is doubtful, however, that such a “model” bears any meaningful relationship to reality. In the world we live in, people come in packages of demographic and political orientations, which correlate in ways suggestive of various latent forms of identity. Thus, “partialing” out the covariance of gender, race, religion, education, and income in order to estimate the “independent” effect of ideology creates a model of something (either individual characteristics disconnected from people, or people who can be randomly endowed with combinations of characteristics) not actually observed on planet earth (Berry & Feldman 1985, p. 48; Cohen, Cohen, West &  Aiken 2003, p. 419; Gelman & Hill 2006, p. 187). Absent some appropriate aggregation of all these variables into a valid latent-variable measure, a zero-order correlation furnishes a more valid estimate of the influence of subjects' political outlooks on their CRT scores than does the coefficient for that predictor in a model that treats ideology and all of these other characteristics as independent, right-hand side variables in a multivariate regression (Lieberson 1985, pp. 14-43). But for the benefit of those who prefer to regard multivariate regression as a magical black box for capturing the “causal” effect of phantom essences, as opposed to a statistical tool for measuring the relationship of valid measures of real-world phenomena, this blunderbuss analysis shows the the coefficient for Conserv_Repub remained trivially different from zero and nonsignificant (b = 0.06,  p = 0.33). When party self-identification was substituted for Conserv_Repub, that variable continued to predict an increase in CRT score as subjects’ identification with the Republican party intensified, but the effect was reduced and was only marginally significant (b = 0.10, p = 0.06).

References

Berry, W.D. & Feldman, S. Multiple regression in practice. (Sage Publications, Beverly Hills; 1985).

Cohen, J., Cohen, P., West, S.G. & Aiken, L.S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Edn. 3rd. (L. Erlbaum Associates, Mahwah, N.J.; 2003).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

Lieberson, S. Making it count : the improvement of social research and theory. (University of California Press, Berkeley; 1985).

Wednesday
Dec122012

Query on climate change, culture, ideology & identity-protective cognition

A thoughtful person asks:

I’ve come across your work while trying to make sense of climate change denial. I find your analysis very interesting (along with Flynn et al. 1994, Finucane et al. 2007, and McCright and Dunlap 2011) because it offers a compelling explanation for what seems like a curious social dynamic.

However, there’s something I don’t quite follow, and with your kind forbearance I hope I may ask you a question. Your 2007 Journal of Empirical Legal Studies paper sketches out a synthesis of cultural risk perception and identity-protective cognition. From that I was expecting to see how group identity (conservative, Republican) and world vision (hierarchical, individualistic) somehow mutually reinforced each other in the climate change arena. In fact, though, that seems not to be the case. Instead it is the hierarchical and individualistic world vision itself that cognition seeks to protect. Indeed, your regression 4 in Table 2 (p. 483) if anything seems inconsistent with my expectation, since both "Conservative" and "Democrat" are highly significant, whereas if these substantially overlapped with the hierarchical white male dummy (so to speak), the coefficients would have been insignificant. Is your view that Conservative and Democrat are independent (from the white male effect) determinants of views on climate change? The narrative in McCright and Dunlap directly linking climate change skepticism to conservatism appeals to me, but I'm not sure if it's consistent with your own perceptions and/or findings.

Thanks very much, and thanks for your very interesting paper.

This is my response. Anyone want to add anything? 

I believe that people have unobservable latent predispositions that they acquire as a result of one or another social influence. The thing to do is find observable indicators that one has good reason to believe correlate with those dispositions, combine them into reliable scales, and use those measures to test hypotheses about who sees what & why, & about what sorts of communication strategies are geared to promoting open-minded engagement with information by people of diverse predispositions.  "Republican, Democrat, liberal, conservative, hierarch, egalitarian, individualist," etc are all candidate indicators. Which ones to combine to form scales depends on which latent-variable measurement strategy most instructively enables explanation, prediction & prescription.  

For more info, click on links below; & let me know if you have additional questions or if you have reactions, comments etc.

Tuesday
Dec112012

Yow--lots of great comments on "science literacy vs. climate-change science literacy"

I was going to post something, but it's not as interesting or important as the points that people made in response to yesterday's post on science literacy & climate-change science literacy.  So check those out! And add more.

 

Monday
Dec102012

Science literacy vs. "climate science literacy"

This is in the department "recurring misunderstanding that I should say something about in a single place so that I can simply refer people to it."

Last May, CCP researchers published a study in Nature Climate Change presenting evidence suggesting that political controversy over climate change in the US cannot be attributed to any sort of deficit in the public's comprehension of science.

As science literacy and numeracy  (a technical reasoning disposition associated with more discerning perception of risk) increase, members of the general public do not converge in their perceptions of the risks posed by climate change. Instead, they become even more culturally polarized.

This finding fit the hypothesis that individuals can be expected to engage information in a manner that fits their interest in forming and maintaining beliefs that reflect their membership in, and loyalty to, important affinity groups.

Competing positions on climate change, unfortunately, are now conspicuously associated with opposing cultural groups. Being out of line with one's group on this issue exposes an individual to a social cost, whereas forming a mistaken view on the science of climate change has zero impact on the risk that individual, or anyone or anything she cares about, faces, insofar as one individual's personal behavior (as consumer, voter, public discussant, etc.) has no material effect on the climate.

One doesn't have to be a rocket scientist to figure out what side of the issue one's cultural group is on in a debate like the one over climate change. But if one is, well, not a rocket scientist, but someone who has an above-average command of basic science and an above-average ability to make sense of fairly complicated technical and quantitative information, then one necessarily has skills --an ability to search out  supportive evidence, fight off counterarguments, etc.-- that one can use to be even more successful at forming and persisting in group-convergent beliefs.

The survey data reported in the Nature Climate Change study supported this conjecture. The experimental findings in the most recent CCP study -- on ideology, motivated reasoning, and cognitive reflection--supply even more support for it.

Now, the response to the Nature Climate Change study that I have in mind says, "Wait -- you didn't measure climate change literacy! Regardless of their worldviews, if people knew more about climate change science they surely would converge on the best understanding of the risks that climate change poses!"

That response is in fact a non sequitur.

Yes, of course, people who are "climate science literate," by definition, understand and accept the best scientific evidence on climate change. 

The whole point of the study, though, was to test hypotheses about why members of the general public haven't converged on that evidence -- or why, in other words, they aren't uniformly climate-science literate.  

We measured their general science literacy to assess the (widespread) claim that a general deficit in science comprehension explains this particular aspect of confusion about science.  What we found -- that members of the general public who display the greatest general science comprehension are the most culturally polarized on climate change risks -- is flatly inconsistent with that claim.

Imagine we had measured "climate change literacy" instead and used it to predict "climate change risk perception." We would have found that the former predicts the latter quite well -- because in fact, they are, analytically, the same thing.  

But then we'd still be left with the key question -- what explains deficits in "climate science literacy"?  By measuring general science literacy--something that is analytically distinct from climate change risk perception--we were able to help show that one common conjecture about that -- that people are not "climate change science literate" because they can't comprehend basic science -- is inconsistent with empirical evidence.

If one genuinely wants to explain public conflict over climate change, one has to offer and test explanations that don't just amount to redescribing the phenomenon.

And if the goal is to promote public recognition of the best available evidence on climate change -- and other societal risks -- then the sort of science illiteracy we need to remedy relates to our collective ability to protect our science communication environment from the sorts of toxic cultural meanings that make it individually rational for ordinary citizens -- including the most science literate ones -- to pay more attention to what positions on risk say about who they are than to whether those positions are true.

References 

Kahan, D. Why we are poles apart on climate change. Nature 488, 255 (2012).

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

Kahan, D.M., Wittlin, M., Peters, E., Slovic, P., Ouellette L.L., Braman, D., Mandel, G. The Tragedy of the Risk-Perception Commons: Culture Conflict, Rationality Conflict, and Climate Change. CCP Working Paper No. 89 (June 24, 2011).

Ideology, Motivated Reasoning, and Cognitive Reflection, CCP Working Paper 107 (Nov. 29, 2012)

Peters, E., Västfjäll, D., Slovic, P., Mertz, C.K., Mazzocco, K. & Dickert, S. Numeracy and Decision Making. Psychol Sci 17, 407-413 (2006).

 

 

Saturday
Dec082012

Cultural cognition is not a bias -- and the corruption of it is no laughing matter!

Well, I feel sort of bad for coming across as gleeful in reporting that further analysis of data confirmed Indepdendents, just like politial partisans, display ideologically motivated reasoning.  A commentator (Metamorph, aka "Metamorph") called me out on that.

My punishment is to write 500 times ...

1. Cultural cognition is not a bias (parts one and two).

2. It's the science communication environment, stupid -- not stupid people!

3. Cultural cognition is not a bummer (parts one and two).

 

Wednesday
Dec052012

WSMD? JA!, episode 3: It turns out that Independents are as just partisan in cognition as Democrats & Republicans after all!

This is the third episode in the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

Okay, so I was all freaked out by the discovery that Independents are more reflective, in terms of the Cognitive Reflection Test scores, than partisans and wondering if this signified that somehow that Independents are these magical creatures who don't become even more vulnerable to ideologically motivated reasoning as their disposition to engage in analytical, System 2 reasoning becomes more pronounced (one of the findings of the latest CCP study).

Enticed by my promise to share the Nobel Prize in whatever 4 or 5 disciplines would surely award it to us for unravelling  this cosmic mystery, Isabel Penraeth (aka "Gemeinshaft Girl) and NiV (aka "NiV") told me to just calm down and use some System 2 thinking. Did it ever occur to you, NiV asked with barely concealed exasperation, that the problem might be that Independents are members of a "cultural in-group" that evades the dopey 1-dimensional left-right measure used in the study? Yes, you fool, added Geminschaft Girl, have you even botherd to see whether Independents behave at all differently from Partisans (let's use that term for those who identify as either Republicans or Democrats) when their worldviews are measured with the "CCR group-grid scale?"

Doh! Of course, this is the right way to figure out if there's really any difference in how Independents and Partisans process information. 

The basic hypothesis of the study was that ideologically motivated reasoning is a consequence of a kind of "identity-protective cognition" that reflects the stake people have in forming perceptions of risk and other policy-relevant facts consistent with the ones that predominate in important affinity groups.

This is actually the core idea behind cultural cognition generally. Usually, too -- as in always before now, really-- our studies have used "cultural worldview" scales, derived from the "group-grid framework of Mary Douglas, to measure the motivating group commitments that we hypothesized drive identity-protective cognition on climate change, gun control, nuclear power, the HPV vaccine, Rock 'n Roll vs. Country, & like issues.

We do that, I've explained, because we think the cultural worldview measures are better than left-right measures. They are more discerning of variations in the outlooks of ordinary, nonpartisan folk, and thus do a better job of locating the source and magnitude of cultural divisions on risk issues.

The reason I used right-left in the most recent study was that I wanted to maximize engagement with the researchers whose interesting ideas motivated me to conduct it. These included the Neo–Authoritarian Personality scholars, whose work is expertly synthesized in Chris Mooney's Republican Brain. They all use right-left measures, which, like I said, I don't think are as good as cultural-worldview ones but are (as I've explained before) plausibly viewed as alternative indicators of the same latent motivating predispositions.

So for crying out loud, why not just see how Independents compare with Partisans when instead of right-left ideology cultural worldviews are used as the predictor in the motivated-reasoning experiment described in the study?! Of course, I have the data on the subjects cultural worldviews; like the participants in all of our studies, they were part of a large, nationally diverse subject pool recruited to take part in cultural cognition studies generally.

As I'm sure you all remember vividly, the experiment tested whether subjects would show motivated reasoning in assessing evidence of the "validity" of Shane "No limit video poker world champion" Frederick's gold-standard "System 1 vs. System 2" Cognitive Reflection Test.  Subjects were assigned to one of three conditions: (1) a control group, whose members were told simply that psychologists view CRT as valid test of open-mindedness and reflection; (2) a "skeptic-is-biased" condition, whose members were told in addition that "climate skeptics" tend to get lower CRT scores (i.e., are more closed-minded and unreflective); and (3) a "nonskeptic-is-biased" condition, whose members were told that "climate believers" get lower scores (i.e., are more closed-minded and unreflective). 

As hypothesized, subjects polarized along ideological lines in patterns that reflected their disposition to fit their assessment of scientific informtion--here on a test that measures open-mindedness and reflection--to their ideological commitments. So relative to their counterparts in the control, more liberal, Democratic subjects were more likely to deem the CRT valid, and more conservative Republican ones to deem it invalid, in the "skeptic-is-biased" condition; these positions were flipped in the "nonskeptic-is-biased condition." Moreveover, this effect was magnified by subjects' scores on the CRT test--i.e., the more disposed they are to use analytical rather than heuristic-driven reasoning, the more prone subjects are to ideologically motivated reasoning.  

Necessarily, though, Independents, didn't show such an effect (how could they, logically speaking? they aren't left or right to a meaningful degree) and they happened to score a bit higher than Partisans (Dems or Repubs) on CRT. Hmmmm....

But Independents, just like Democrats and Republicans, have cultural outlooks. So I reanalyzed the study data using the cultural cognitoin "Hierarchy-egaltiarian" and "Individualist-communitarian" worldview scales.

Because climate change is an issue that tends to divide Hierarch Individualists (HIs) and Egalitarian Communitarians (ECs), my principal hypotheses were (1) that HIs and ECs would display motivated reasoning effects equivalent to the ones of conservative Republicans and liberal Democrats, respectively, and (2) that this effect would increase as subjects CRT reflectiveness scores increased. The competiting additional hypotheses: (3a) that Independents wouldn't behave any differently in this respect than Partisans; and (3b) that Independents would be shown to be magic, supherhuman (possibly outerspace alien) beings who are immune to motivated cognition.

I had my money (a $10,000 bet made w/ Willard, a super rich guy who doesn't pay any income taxes) on 3a. Independents, like Democrats and Republicans, have cultural worldviews; why wouldn't they be motivated to protect their cultural identities just like everyone else?

Results? Hypotheses (1) and (2) were confirmed. When I just looked at subjects defined in terms of their worldviews, I observed the expected pattern of polarization. Indeed, HIs and ECs reacted in an even more forcefully polarizing manner to the experimental manipulation than did conservative Republicans and liberal Democrats, an effect that should come as no surprise because the culture measures are indeed better--i.e., more discerning--measures of the group dispositions that motivated biased processing of information on risk and other policy-relevant facts.

Next, I compared the size of this culturally motivated reasoning effect for Partisans and Independents, respectively.  The regression model that added the appropriate variables for being an Independent did add explanatory power relative to the model that pooled Indepedents and Partisans. But the effect was associated almost entirely with the tendency of Independents to polarize more forcefully in the "skeptic-is-biased" condition. The same basic pattern--HIs and ECs polarizing in the expected ways, and magnification of that effect by higher CRT scores--obtained among both Partisans and Independents.

 You can see that there are some small differences, ones that reflect the relationship I described between being an Independent and being assigned to the "skeptic-is-biased" condition.  But I myself don't view these differences as particularly meaningful; when you start to slice & dice, you'll always see something, so if it wasn't something you were looking for on the basis of a sensible hypothesis, more than likely you are looking at noise.

So I say this is corroboration of hypothesis (3a): Independents are just as partisan in their assessment of information that threatens their cultural identities as political Partisans. I'm done being freaked out!

But hey, if you disagree, tell me! Come up with an interesting hypothesis about how Independents are "different" & I'll test it with our data, if I can, in another episode of WSMD? JA!

 WSMD? JA! episode 1

WSMD? JA! episode 2

Ideology, Motivated Reasoning, and Cognitive Reflection, CCP Working Paper 107 (Nov. 29, 2012)

Tuesday
Dec042012

What does the science of science communication say about communicating & expanding interest in noncontroversial but just really really cool science?

I got this really thoughtful email from Jason Davis, who is doing graduate work in journalism at the University of Arizona, and who operates Astrsosaurus, an interesting-looking science journalism site:

I just finished watching your recent science communication talk for the Spitfire Strategies' speaker series on YouTube. . . .

I'm curious to hear your thoughts on communicating "non-controversial" science. There seems to be no shortage of plausible, but not necessarily correct, ideas on addressing polarized subjects like global warming, stem cell research, nuclear power, etc. But what about something more innocuous, like spaceflight? Do you think the "you tell me what works and what doesn't" approach you emphasize in your lecture is equally valid in these areas? Or since we're treading on less-controversial turf, are we back to a deficit approach? Or perhaps it's naive to assume any science issue can be communicated without controversy?

As an example, I'm personally fascinated by the vast unknowns in our own solar system, but I know not everyone shares my enthusiasm. To make some gross assumptions here, if we had more public support for NASA, perhaps its budget would be increased, and perhaps we would have more spacecraft uncovering the mysteries of Jupiter and Saturn's moons. So, are we to assume that more effective science communication could close that gap? If so, what should that communication look like? Or should I just concede that not everyone is interested in these things, and convincing someone to care about the moon Enceladus is akin to someone else trying convincing me to care about, say, Hollywood gossip?...

Jason Davis

 www.astrosaur.us

Here is my response, which I think is OK, but definitely could be improved upon -- by others who have thoughts, insights, and experience.  For one thing, looking back on this, I can see that I sort of avoided the "how to generate interest" question & instead focused on "how to satisfy the appetite of culturally diverse people who are curious to know what is known" -- which I view as the critical mission that that science journalism plays in the Liberal Republic of Science. So feel free to supplement my response in comments section (I'd like to know answers to Jason's questions, too!).

My response:

1. I think this is a very very different sort of science communication issue. The biggest fallacy that the science of science communication addresses is that there is no need for a science of science communication -- sound science communicates itself. But the second biggest is that the science of science communication is one thing -- that the same insights into how communicate probabilistic information to an individual trying to make an informed decision about a medical procedure are the same ones that will "solve" the climate change dispute.  Science communication is 5 things +/- 2.

2.  One of the things it is is systematic, empirical inquiry into how to make what's known to science known to curious people. Many many people in the Liberal Republic of Science are thrilled and awed by what the use of our collective intelligence has revealed to our species about the workings of the universe; it is part of what makes this political regime so good that it invests resources to produce information to satisfy that interest in knowing what's known, and that many of our smartest & most creative people are excited to play the translation role that science journalism involves.

3.  The state of knowledge on this part of science communication is in good shape -- it is in much better shape than the part of our scientific knowledge that relates to protecting the quality of the science communication environment from toxic meanings that disable citizens' ordinarily reliable sense of who knows what.  Indeed, this part of science knowledge -- the part that involves making what's known by science known to curious people -- is woven into the sophisticated and successful craft of science journalists & related communicators.

4.  But I still think there are ways in which the use of scientific inquiry-- indeed, the incorporation of scientific tools, insights, methods into the craft of science journalism -- could make this sort of science communication better. One problem, which I address here, is that I think the resource of accessible and entertaining communications of what is known to science is not as readily available to all cultural groups in our society.  I am curious to know if you think I'm right in this hunch; surely you are in a position to tell me -- you are thoughtful & are dedicating your life to this field.

 

Monday
Dec032012

Three theories of why we see ideological polarization on facts

Explaining the phenomenon of political conflict over risk and related policy-consequential facts is, of course, the whole point -- or the starting point; the "whole point" includes counteracting it, too-- of the Cultural Cognition Project.

But what's being explained admits of diverse characterizations, and in fact tends to get framed and reframed across studies in a way that enables the basic conjecture to be probed and re-probed over & over from a variety of complementary angles (and supplementary ones too)

Yes, simple obsessiveness is part of what's going on. But so is validity. 

No finding is ever uniquely determined by a single theory. One makes a study as singularly supportive as possible of the "basic account" from which the study hypothesis derived. But corroboration of the hypothesis can't by itself completely rule out the possibility that something else might have generated the effect observed in a particular study.

The way to deal with this issue is not to argue until one's blue in the face w/ someone who says, "well, yes, but maybe it was ..."; rather it is to do another study, and another, and another, in which the same basic account is approached and re-approached from different related angles, enabled by slightly different reframings of the basic conjecture. Yes, in each instance, "something else" -- that is, something other than the conjecture you meant to be testing -- might "really"  explain the result. But the likelihood that "something else" was  "really" going on -- and necessarily something different every time; if there's one particular alternative theory that fits every single one of your results just as well as your theory, then you are doing a pretty bad job in study design! -- becomes vanishingly more remote as more and more studies that all reflect your conjecture reframed in another way way keep piling up.

The framing of the latest CCP study, Ideology, Motived Reasoning, and Cognitive Reflection, is a reframing in that spirit.  

The study presents experimental evidence that supports the hypotheses that ideologically motivated reasoning is symmetric or uniform across different systems of political values and that it increases in intensity as individuals' disposition to engage in conscious and systematic information processing-- as opposed to intuitive, heuristic-driven information processing-- increases.  

Those findings lend support to the "basic account" of cultural cognition: that political polarization over risk reflects the entanglement of policy-relevant facts in antagonistic social meanings; fixing the science communication problem, then, depends on disentangling meaning and fact.

But the story is told here as involving a competition between three "theories" of how dual-process reasoning, motivated cognition, and ideology relate to each other.  That story is meant to be interesting in itself, even if one hasn't tuned into all the previous episodes.

Here is the description of those theories in the paper; see if you can guess which one is really "cultural cognition"!

a. Public irrationality thesis (PIT). PIT treats dual-process reasoning as foundational and uses motivated cognition to explain individual differences in risk perception. The predominance of heuristic or System 1 reasoning styles among members of the general public, on this view, accounts for the failure of democratic institutions to converge reliably on the best available evidence as reflected in scientific consensus on issues like climate change (Weber 2006). Dynamics of motivated cognition, however, help to explain the ideological character of the resulting public controversy over such evidence. Many of the emotional resonances that drive system 1 risk perceptions, it is posited, originate in (or are reinforced by) the sorts of affinity groups that share cultural or ideological commitments. Where the group-based resonances that attach to putative risk sources (guns, say, or nuclear power plants) vary, then, we can expect to see systematic differences in risk perceptions across members of ideologically or culturally uniform groups (Lilienfeld, Ammirati, Landfield 2009; Sunstein 2007).

b.  Republican Brain hypothesis (RBH). RBH—so designated here in recognition of the synthesis constructed in Mooney (2012); see also Jost & Amado (2011)—treats the neo–authoritarian personality findings as foundational and links low-quality information processing and motivated cognition to them. Like PIT, RBH assumes motivated cognition is a heuristic-driven form of reasoning. The mental dispositions that the neo–authoritarian personality research identifies with conservative ideology—dogmatism, need for closure, aversion to complexity, and the like—indicate a disposition to rely predominantly on System 1. Accordingly, the impact of ideologically motivated cognition, even if not confined to conservatives, is disproportionately associated with that ideology by virtue of the negative correlation between conservativism and the traits of open-mindedness, and critical reflection—System 2, in Kahneman terms—that would otherwise check and counteract it (e.g., Mooney 2012;  Jost, Blaser, Kruglanski & Sulloway 2003; Kruglanski 2004; Thórisdóttir & Jost 2011; Feygina, Jost & Goldsmith 2010; Jost, Nosek & Gosling 2008).

It is primarily this strong prediction of asymmetry in motivated reasoning that distinguishes RBH from PIT. PIT does predict that motivated reasoning will be correlated with the disposition to use System 1 as opposed to System 2 forms of information processing. But nothing intrinsic to PIT furnishes a reason to believe that these dispositions will vary systematically across persons of diverse ideology.

c.  Expressive rationality thesis (ERT). ERT lays primary emphasis on identity-protective motivated reasoning, which it identifies as a form of information processing that rationally advances individual ends (Kahan, Peters, Wittlin, Slovic, Ouellette, Braman & Mandel 2012). The link it asserts between identity-protective cognition, so conceived, and dual-process reasoning creates strong points of divergence between ERT and both PIT and RBH.

One conception of “rationality” applies this designation to mental operations when and to the extent that they promote a person’s ends defined with reference to some appropriate normative standard. When individuals display identity-protective cognition, their processing of information will more reliably guide them to perceptions of fact congruent with their membership in ideologically or culturally defined affinity groups than to ones that reflect the best available scientific evidence. According to ERT, this form of information processing, when applied to the sorts of facts at issue in polarized policy disputes, will predictably make ordinary individuals better off. Any mistake an individual makes about the science on, say, the reality or causes of climate change will not affect the level of risk for her or for any other person or thing he cares about: whatever she does—as consumer, as voter, as participant in public discourse—will be too inconsequential to have an impact. But insofar as opposing positions on climate change have come to express membership in and loyalty to opposing self-defining groups, a person’s formation of a belief out of keeping with the one that predominates in hers could mark her as untrustworthy or stupid, and thus compromise her relationships with others. It is therefore “rational” for individuals in that situation to assess information in a manner that aligns their beliefs with those that predominate in their group whether or not those beliefs are correct—an outcome that could nevertheless be very bad for society at large (Kahan 2012b).

It is important to recognize that nothing in this account of the individual rationality of identity-protective cognition implies that this process is conscious. Indeed, the idea that people will consciously manage what they believe about facts in order to promote some interest or goal extrinsic to the truth of their beliefs reflects a conceptually incoherent (and psychologically implausible) picture of what it means to “believe” something (Elster 1983). Rather the claim is simply that people should be expected to converge more readily on styles of information processing, including unconscious ones, that promote rather than frustrate their individual ends. At least in regard to the types of risks and policy-relevant facts typically at issue in democratic political debate, ordinary people’s personal ends will be better served when unconscious modes of cognition reliably focus their attention in a manner that enables them to form and maintain beliefs congruent with their identity-defining commitments. They are thus likely to display that form of reasoning at the individual level, whether or not it serves the collective interest for them to do so (Kahan et al. 2012).

Individuals disposed to resort to low-level, System 1 cognitive processing should not have too much difficulty fitting in. Conformity to peer influences, receptivity to “elite” cues, and sensitivity to intuitions calibrated by the same will ordinarily guide them reliably to stances that cohere with and express their group commitments.

But if individuals are adept as using high-level, System 2 modes of information processing, then they ought to be even better at fitting their beliefs to their group identities. Their capacity to make sense of more complex forms of evidence (including quantitative data) will supply them with a special resource that they can use to fight off counterarguments or to identify what stance to take on technical issues more remote from ones that that figure in the most familiar and accessible public discussions.

ERT thus inverts the relationship that PIT posits between motivated cognition and dual-process reasoning. Whereas PIT views ideological polarization as evidence of a deficit in System 2 reasoning capacities, ERT predicts that the reliable employment of higher-level information processing will magnify the polarizing effects of identity-protective cognition (Kahan et al. 2012).

Again, the argument is not that such individuals will be consciously managing the content of their beliefs. Rather it is that individuals who are disposed and equipped to make ready use of high-level, conscious information processing can be expected to do so in the service of their unconscious motivation to form and maintain beliefs that foster their connection to identity-defining groups.

ERT’s understanding of the source of ideologically motivated reasoning also puts it into conflict with RBH. To begin, identity-protective cognition—the species of motivated reasoning that ERT understands to be at work in such conflicts—is not a distinctively political phenomenon. It is likely to be triggered by other important affinities, too—such as the institutional affiliations of college students or the team loyalties of sports fans. Unless there is something distinctive about “liberal” political groups that makes them less capable of underwriting community attachment than all other manner of group, it would seem odd for motivated reasoning to display the asymmetry that RBH predicts when identity-protective cognition operates in the domain of politics.

In addition, because RBH, like PIT, assumes motivated reasoning is a feature of low-level, System 1 information processing, ERT calls into question the theoretical basis for RBH’s expectation of asymmetry. Like PIT, ERT in fact suggests no reason to believe that low-level, System 1 reasoning dispositions will be correlated with ideological or other values. But because ERT asserts that high-level, System 2 reasoning dispositions magnify identity-protective cognition, the correlations featured in the neo–authoritarian-personality research would, if anything, imply that liberals—by virtue of their disposition to use systematic reasoning—are all the more likely to succeed in resisting evidence that challenges the factual premises of their preferred policy positions. Again, however, because ERT is neutral on how System 1 and System 2 dispositions are in fact distributed across the population, it certainly does not entail such a prediction.

 

Sunday
Dec022012

What does "Lincoln" mean?

Saw Lincoln last night.

On 2d try. The 1st time theater was sold out.  

So this time I got my tickets in advance. But because I made the "mistake" of showing up 10 mins after "screen time" -- still a good 15 mins before end of the annoying previews -- I had to sit in the first row of an overpacked theater.   Overpacked w/ ordinary people in an ordinary central CT "suburban" (what passes for that in CT) multiplex.

The full house burst into applause at end, and the theater didn't empty until all the credits had run out...

The movie was beautiful & moving.

Learning (the hard way?) that so many other people -- ones with nothing particular in common with me except for belonging to the same vast and vastly diverse society -- also found the movie so beautiful & moving was even more so.

 

Saturday
Dec012012

Okay, now *this* is the Liberal Republic of Science!

Gets the point across so much more succinctly-- and inspiringly!

 

Friday
Nov302012

A surprising (to me) discovery: reflective Independents...

The analyses I did for my latest paper—on ideology, cognitive reflection, and motivated reasoning—really surprised me in one respect.

They didn’t surprise me altogether. Indeed, they corroborated the hypothesis (one that also was explored in the CCP study of science comprehension & climate change polarization) that people who are more disposed to use System 2 reasoning (conscious, analytical, reflective) are more likely to to selectively credit or discredit evidence in patterns that fit their ideological predispositions.  This is contrary to how most people thing heuristic-driven, System 1 reasoning contributes to public confusion and controversy on issues like climate change.

But what did surprise me was the finding that self-identified Independents are more reflective—more disposed to use System 2 rather than System 1 reasoning.  I assumed people who were in middle were just less reflective.  The difference isn’t huge (and actually, no one, of any particular political orientation or non-orientation demonstrates a high degree of reflection on Shane Frederick's gold-standard CRT test), but it’s there.

It also follows from the analyses that are in the paper that Independents display less motivated reasoning than partisans. Of course, that’s sort of a logical thing; if they don’t have a predisposition, they can’t be fitting their interpretation of evidence to it. But I think there’s more to it than that.

Why am I surprised? My experience in doing studies has caused me to form the impression that people who are “in the middle” on measures of cultural or ideological predispositions are sort of like statistical noise—random, unreliable--& not that important for figuring out what is going on, at least if the signal you get from people w/ a more choate sense of identity is a clear one.

Well, it looks anyway, like the Independents are not simply inert or confused. They are reflective people, engaging information of political significance in a non-ideological way.  That’s something to try to figure out, not dismiss.  What are they thinking? Who the hell are they?

At this point, I’m not suffering any great intellectual crisis.  I suspect if I thoughtfully engage the data a bit more, I’ll discover something that, without necessarily making this finding unimportant, reveals that it it poses no particular problem for the basic hypothesis behind the study (which is that individuals rationally engage information on societal risks in a manner that reflects their interest in forming and maintaining group connections).

But I’m curious. Also a bit excited and anxious; maybe I’m missing something really important.

What I’m going to do for now is think for a bit. Also read and re-read some other things (including John Bullock's great study on need for cognition and partisanship).  And try to form some interesting hypotheses about what the “Reflective Independent” datum might mean. Then I’ll see if there is a way to test those hypotheses, at least provisionally, with this data set.

Like I said, I don’t think I’ll find anything here that makes me think I have to adjust my thinking in a major way. But I want to approach this minor nuggest of surprise in a way that wouldn’t obscure the possibility that just beneath it is a deep deposit of information that would liberate me from the intellectually destitute state of unrecognized ignorance.

So to start this inquiry:

Do you have ideas about this little datum? What to make of it; how to explore its signficance?  

Fine to tell me, too, if you think this was “obvious” for reason x, y, & z; but do realize that you could have been assigned to one condition in a “many worlds” experimental design that includes another condition in which my doppelgänger has just blogged, “See! Independents are less reflective!,” and in which yours is typing up the response, “Of course, that was obvious!