follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk


What is "cultural cognition"? I'll show you... (Slides)

These slides are from a talk I gave last August at SENCER summer institute.  I think I didn't put it up then b/c we hadn't yet put out the working paper on "motivated numeracy."

The talk is sort of upsidedown relative to one that I sometimes give to lawyers (including law scholars & judges).  In that talk, I start w/ the "science communication problem" & then say: "now behold: the law has a similar difficulty -- the 'neutrality communication problem!'"

Here, in contrast, I said, "look at the law -- it doesn't get that neutral legal decisionmaking doesn't communicate its own neutrality to public. Now behold: science has same problem--valid science doesn't communicate its own validity!"

I guess the idea is that it's easier to recognize how commitment to a way of seeing things is interfering with one's goals if one is first shown the same phenomenon vexing someone else...

Of course, the SENCER folks aren't vexed by what they can't see; they are vexed by what they can see -- the failure of science education and related professions that generate science-informed "products" to use evidence-based methods to assess and improve how they deliver the same. The whole point of SENCER is to get people to see that & do something about it (do what? experiment w/ various possibilities & report the results, of course!).

So maybe I was preaching to the choir.  But it still seemed to make sense to enter unexpectedly through the side door/roof/chimney.  And maybe what I enabled them to see-- even if it was no surprise -- is that the law could use some SENCERizing too. 


Mitigation & adaptation: Two remedies for a polluted science communication environment

One of the “models” or metaphors I use to try to structure my thinking about (and testing of conjectures on) public conflict over decision-relevant science attributes that problem to a “polluted science communication environment.”  This picture helps not only to sharpen one’s understanding of what the "science communication problem" consists in and what its causes are but also the identity and logic of remedies for it.

1. The science communication environment. People need to recognize as known by science many more things than they could understand or corroborate for themselves. They generally do this by immersing themselves in affinity groups—ones whose members share their basic outlooks on life, and whom they thus get along with and understand, and whose members can be relied upon to concentrate and transmit valid scientific insights (e.g., “bring your baby to the pediatrician—and not the faith healer!—if he or she becomes listless and develops a fever!”).  These diverse networks of certification, then, can be thought of as the “science communication environment” in which culturally diverse citizens, exercising ordinary science intelligence, rationally apprehend what is known to science in a pluralistic society.

2.  A polluted science communication environment. This system for (rationally!) figuring out “who knows what about what” breaks down, though, when risks or like policy-facts become entangled in contentious cultural meanings that transform them, in effect, into badges of membership in and loyalty to opposing groups (“your pediatrician advised you to give your daughter the HPV vaccine? Honey, you need to get a new doctor!”). At that point, the psychic stake that individuals have in maintaining their standing in their group will unconsciously motivate them to adopt modes of engaging information that more reliably connect them to their groups' position than to the best available scientific evidence.  These antagonistic cultural meanings are a form of pollution or contamination of ordinary citizens’ science communication environment that disables (quite literally!) the rational faculties by which individuals reliably apprehend collective knowledge.

3.  Two remedial strategies. We can think of two strategies for responding to a polluted science communication environment.  One is to try to decontaminate it by disentangling toxic meanings from cultural identities, and by adopting processes that prevent such entanglements from occurring in the first place. 

Call this the mitigation strategy.  We can think of “value affirmation,” “cultural source credibility,” “narrative framing” and like mechanisms as instances of it.  There are others too, including systemic or institutional responses aimed at forecasting and avoiding the entanglement of decision-relevant science in antagonistic meanings.

A second strategy is adaptation.  These are devices that counteract the consequences of a contaminated science communication environment not by dispelling it but rather by strengthening the cognitive processes that are disabled by it—or that activate alternative, complimentary cognitive processes that help to compensate for such disablement. 

Again, there are a variety of examples. E.g., satire uses humor to lure individuals into engaged reflection with evidence that might otherwise trigger identity-defensive resistance.  Self-affirmation is similarly thought to furnish a buffer against the anxiety associated with critically re-examining beliefs that have come to symbolize allegiance to one or another opposing cultural style. 

Or consider curiosity. Curiosity is the motivation to experience the pleasure of discovering something new and surprising. In this state (I conjecture), the defensive processes that block open-minded engagement with valid evidence that challenges existing identity-congruent beliefs are silenced.

We could thus see efforts to cultivate curiosity as a character disposition or to concentrate engagement with decision-relevant science in locations (e.g., museums or science-entertainment media) that predictably excite curiosity as a way to neutralize the detrimental impact of the entanglement of risks and other policy-relevant facts with antagonistic cultural meanings.

I’m sure there are more devices and techniques that operate this way—that is operate to rehabilitate disabled faculties or activate alternatives within a polluted science communication environment.  One of the aims of the science of science communication, as a "new political science," should be to identify and learn how to deploy them.

4. Pragmatic "scicomm environmental protection."  Just as mitigation and adaptation are not mutually exclusive strategies for responding to threats to the natural environment, so I would argue that mitigation and adaptation of the sort I’ve just described are not mutually exclusive responses to a polluted science communication environment.  We should be empirically investigating both as part of the program to identify the most reliable means of repelling the threat that a polluted science communication environment poses to the Liberal Republic of Science.


Culture, rationality, and science communication (video)

Here is the video version of this lecture.  Slides here.


Communicating the normality/banality of climate science (lecture slides)

Gave talk yesterday at National Oceanic and Atmospheric Administration. Slides here.

The talk was part of a science-communication session held in connection NOAA's 38th Climate Diagnostics and Prediction Workshop.

The other speaker at the event was Rick Borchelt, Director for Communications and Public Affairs at the Department of Energy's Office of Science, who is an outstanding (a)  natural scientist, (b) scientist of science communication, and (c) science communicator rolled into one. Not a very common thing. I got the benefit of his expertise as he translated some of my own answers into questions into terms that made a lot more sense to everyone, including me.

Our session was organized by David Herring, Director of Communication and Education in NOAA's Climate Program Office, who also possesses the rare and invaluable skill of being able to construct bridging frameworks that enable the insights a particular community of empirical researchers discerns through their professionalized faculty of perception to be viewed clearly and vividly by those from outside that community.  Magic!

My talk was aimed at helping the climate scientists in the audience appreciate that the "information" that ordinary citizens are missing, by and large, has little to do with the content of climate science.

There is persistent confusion in the public not because people "don't get" climate science. They quite understandably don't really "get" myriad bodies of decision-relevant science --from medicine to economics, from telecommunications to aeronautics -- that they intelligently make use of in their lives, personal, professional, and civic.

Moreover, the ordinary citizens best situated to "get" any kind of science -- the ones who have the highest degree of science knowledge and most acutely developed critical reasoning skills -- are the ones most culturally divided on climate change risks.

The most important kind of "science comprehension" -- the foundation of rational thought -- is the capacity to reliably recognize what has been made known by valid science, and to reliably separate it from the myriad claims being made by those who are posing or who are peddling forms of insight not genuinely ground in science's way of knowing.

People exercise that capacity by exploiting the ample stock of cues and signs that their diverse cultural communities supply them and that effectively certify which claims, supported by which evidence, warrant being credited.

The public confusion over climate change, I suggested, consists in the disordered, chaotic, conflictual state of those cues and signs across the diverse communities that members of our pluralistic society inhabit.

The information they are missing, then, consists in vivid, concrete, intelligible examples of people they identify with using valid climate science to inform their practical decisionmaking -- not just as government policymakers but as business people and property owners, and as local citizens engaged in working with one another to assure the continuing viability of the ways of life that they all value and on which their common well-being depends.

This is one of the animating themes of field-based science communication research that the Cultural Cognition Project is undertaking in Florida in advising the Southeast Regional Climate Compact, a coalition of four counties (Broward, Miami-Dade, Palm Beach, and Monroe) that are engaged in updating their comprehensive landuse plans to protect one or another element of the local infrastructure from the persistent weather and climate challenges that it faces, and has faced, actually, for hundreds of years.

One critical element of the Compact's science communication strategy, I believe, consists in furnishing citizens with a simple, unvarnished but unobstructed view of the myriad ways in which all sorts of actors in their community are, in a "business as usual' manner, making use of and demanding that public officials make use of valid climate science to promote the continuing vitality of their way of life.

It's normal, banal. Maybe it's even boring to many of these citizens, who have their own practical affairs to attend to and who busily apply their reason to acquiring and making sense of the information that that involves.

But as citizens they rightfully, sensibly look for the sorts of information that would reliably assure them that the agents they are relying on in government to attend to vital public goods are carrying out their tasks in a manner that reflects an informed understanding of the scientific data on which it depends.  

So give them that--and then leave it to them, applying their own reliable ability to make sense of what they see, to decide for themselves if they are satisfied and to say what more information they'd like if not.

And then give as clear and usable an account of the content of of what science knows about climate to the myriad practical decisionmakers--in government and out--whose decisions must be guided by it.

Doing that is easier said than done too; and it doing it effectively is something that requires evidence-based practice.

But the point is, communicating the substance of valid science for those who will make direct use of it in their practical decisionmaking is an entirely different thing from supplying ordinary citizens with the information that they legitimately are entitled to have to assure them that those engaged in such decisionmaking are relying on the best available scientific evidence.

It's the latter sort of information that there is a deficit of in our public discourse, and that deficit can be remedied only with evidence of that sort -- not evidence relating to the details of the mechanisms that valid climate science is concerning itself with.

This is a theme that I've emphasized before (I'm always saying the same thing; why? Someone must be studying this...).

I'll say more about it, too, I'm sure, in future posts, including ones that relate more of the details of the field-based research we are doing in Florida.

But the most important thing is just how many resourceful, energetic, intelligent, dedicated people are doing the same thing--investigating the same problem by the same means of forming conjectures, gathering evidence to test them, and then sharing what they learn with others so that they can extend and refine the knowledge such activity produces.

David Herring and Rick Borchelt and their colleagues are among those people.


Are "moderates" less affected by politically motivated reasoning? Either "yes by definition," or "maybe, depending on what you mean exactly"

click me ... click me ...A thoughtful person wrote to me about our Motivated Numeracy experiment, posing a variation of a question that I'm frequently asked. That question, essentially, relates to the impact of identity-protective cognition -- the species of motivated reasoning that cultural cognition & politically motivated reasoning are concerned with -- in individuals of a "moderate" political ideology or "Independent" partisan identification.

Here is her question:

I finally got around to looking at your interesting research working paper (that I learned about from

 One thing that bothers me about the design is the creation and labeling of the two political orientation groups. The description in the paper says: "Responses to these two items formed a reliable aggregate Likert scale (α = 0.83), which was labeled "Conserv_Repub" and transformed into a z-score to facilitate interpretation (Smith 2000)." 

In the study this relatively continuous scale was split in the middle into two groups. While I agree that people at each end of the political spectrum would generally subscribe to opposing positions on the utility of gun bans, I do not think that applies to people in the middle third or half of the political spectrum.  I think it is inappropriate to ascribe MOTIVATED numeracy on the gun ban problem to people in the middle of the political spectrum. 

How would your results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution?

My response:

As you note, the scale for political outlooks is a continuous one -- or at least is treated as such when we test for the hypothesized effects. We split the sample only for purposes of exploratory or preliminary analysis -- when we are trying to show a "picture," essentially of the raw data, as in Fig 6.  In the regression model (Table 1), we estimate the impact of "Conservrepub," including its interaction w/ Numeracy in the various experimental conditions, as a continuous variable; Fig. 7 reflects predicted probabilities derived from the model -- not the responses for different subsamples ("high numeracy" & "low numeracy" "conservative republicans" &. "liberal democrats" etc.) determined w/ reference to the means on Conservrepub or Numeracy.

Necessarily, then, were we to model the performance of subjects in the "middle" of Conservrepub, we'd see no (or, if not literally at the "middle" but at intervals relatively close to either side of the mean, "less") motivated reasoning. But that is what we are constrained to see if we choose to measure the hypothesized motivating disposition with a continuous measure, the effect of which is assumed to be uniform or linear across its range of values.  

If one wanted to test the hypothesis that "moderates" or "Independents" are less subject to motivated reasoning, one would have to have a way to model the data that made this claim something other than a tautology.

One way to do it would be to measure the motivating disposition independently of how people identify themselves on the party-id and liberal-conservative ideology measures.  Then we could construct a model that estimates the motivated reasoning effect w/ respect to variance in that disposition & see if *that* effect interacts with being an "Independent" (on the party id scale) or a "moderate" (on the ideology scale).  

I did that with the data from an experiment that had a similar design -- one that tested whether identity-protective cognition, the kind of motivated reasoning we are concerned with, varies with respect to "cognitive reflection' as opposed to Numeracy.  I substituted "cultural worldviews" for political outlooks as the mesure of  the motivating disposition -- and then did as I said by looking at whether the influence the motivating disposition interacted with being a political "Independent." See this blog post (title is hyperlinked) for details:

WSMD? JA!, episode 3: It turns out that Independents are as just partisan in cognition as Democrats & Republicans after all!

I could do the same for the data in Motivated Numeracy.  Likely I will at some point!

You ask what our "results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution." 

We could, in fact, split the continuous predictor Conservrepub into thirds or quarters and measure the impact of "motivated reasoning" separately in each --  by, say, comparing the means for the different groups at different levels of numeracy within them or by fitting a separate regression model to each subgroup. But I'd not trust the results of any such analysis.

For one thing, because the subsamples would be relatively small, such a testing strategy would be underpowered, so we'd not be able to draw any inferences from "null" findings w/ respect to the middling groups, if that is what you hypothesize.  Also, splitting continuous predictors like conservrepub & numeracy risks generating spurious differences among subgrups as a result of the random or lumpy distribution of (or really the imprecision of our measurement of) a "true" linear effect. See Maxwell, S.E. & Delaney, H.D. Bivariate Median Splits and Spurious Statistical Significance. Psychological Bulletin 113, 181-190 (1993).
Accordingly, sample splitting of this sort  s not a valid strategy, in my view, for testing hypotheses relating to variation in motivated reasoning across the left-right spectrum (whether the hypothesis is that the effect grows more extreme toward both ends, as you surmise, or is that it grows more extreme as one goes toward one end but not the other-- the so-called "asymmetry thesis"...).

But I'm sure there are other valid strategies, too, for testing the hypothesis that motivated reasoning increases as a function of the intensity of political partisanship, a proposition that is, as indicated, *assumed* rather than tested in the model we use in the paper.  Be happy to hear of any you come up with.  Might make for a fun episode of WSMD? JA! 

I am also curious, though, why this would be a surprising or interesting thing? Measurement issues aside, why wouldn't it be just a matter of logic to say that the higher the level of partisan motivation, the higher the impact of politically or culturally motivated reasoning?  Or what is the motivation for asserting such a claim?

Is it the sense that the effect we are demonstrating experimentally is confined to "hard core" partisans?  For that, one needs to have some practical way of assessing the experimental impact-- one that rests on assumptions outside the experiment (e.g.,  aboutwho a "hardcore" partisan is w/ respect to the values reflected in Conservrepub & the relative impact that people of varying levels of partisanship make to the overall shape of public opinion & overall character of deliberations, etc.).

In that regard, one more thing you might find interesting is:

Politically nonpartisan folks are culturally polarized on climate change



Congratulations, tea party members: You are just as vulnerable to politically biased misinterpretation of science as everyone else! Is fixing this threat to our Republic part of your program?

A recurring irony in the empirical study of politically biased misunderstandings of science is how often people misconstrue empirical evidence of this very phenomenon as a result of politically biased reasoning.

It’s funny.

It’s painful.

And it’s depressing—indeed, the 50th time you see it, it is mainly just depressing

So I wasn’t “surprised”—much less “stunned”—when I observed descriptions of the data I presented on the correlation between science comprehension and identification with the tea party being warped by this same dynamic.

The 14 billion regular readers of this blog (exactly 2,503,232 of whom identity with the tea party) know that I believe that there is no convincing empirical evidence that the science communication problem—the failure of compelling, widely accessible scientific evidence to dispel culturally fractious disputes over societal risks and other policy-relevant facts—can be attributed to any supposed correlation between a “conservative” political outlook & a deficit in science literacy, critical reasoning skills, or commitment to science’s signature methods for discovery of truth.

On the contrary, I believe that the popularity of this claim reflects the vulnerability of those who harbor a “nonconservative” (“liberal,” “egalitarian,” or
whatever one chooses to style it) outlook to accept invalid or ill-supported empirical assertions that affirm their cultural outlooks. 

That vulnerability, I believe, is perfectly “symmetrical” with respect to the right-left political spectrum (and the two-dimensional space defined by the cultural continua of “hierarchy-egalitarianism” and “individualism-communitarianism”).

I believe that, in part, because of a study I conducted in which I found evidence that there was an ideologically uniform tendency—one equal in strength, among both “conservatives” and “liberals”—to credit or dismiss empirical evidence supporting the validity of an “open-mindedness” test depending on whether study subjects were told that the test showed that those who share their ideology were more or less open-minded than those subscribing to the opposing one.

Not only do I think the “asymmetry thesis” (AT)—the view that this pernicious deficiency in reasoning is disproportionately associated with conservativism—is wrong.

I think the contempt typically evinced (typically but not invariably; it's possible to investigate such hypotheses without ridiculing people) toward "conservatives" by AT proponents strengthens the dynamics that account for this reason-effacing, deliberation-distorting form of motivated cognition.

I want reasoning people to understand this.  I want them to understand it so that they won’t be lulled into behaving in a way that undermines the prospects for enlightened democracy.  I want them to understand it so that they can, instead, apply their reason to the project of ridding the science communication environment of the toxic partisan entanglement of facts with cultural meanings that is the source of this pathology.

The “tea party science comprehension” post was written in that spirit.  It presented evidence that a particular science comprehension measure I am working on (in an effort to help social scientists, educators, and others improve  existing measures, all of which are very crude) has no meaningful correlation with political outlooks.

Actually, the measure did correlate negatively—“r = - 0.05, p < 0.05”—with a scale assessing one’s disposition to identify one’s ideology as “conservative” and one’s party affiliation as “Republican.”

I noted that, and pointed out that this association was far too trivial to be afforded any practical significance whatsoever, much less to be regarded as the source of the fierce conflicts in our society over climate change and other issues turning on decision-relevant science.

But anticipating that politically motivated reasoning would likely induce some readers who identify as “liberal” and “Democratic” to seize on this pitifully small correlation as evidence that of course politically biased reasoning explains why those who identify as "conservative" & "Republican"  disagree with them, I advised any such readers to consider the correlation between science comprehension and identifying with the tea-party: r = 0.05, p = 0.05.

Anyone who might be tempted to beat his or her chest in a triumphal tribal howl over the practically meaningless correlation between right-left political outlooks & science comprehension could thus expect to find him- or herself fatally impaled the very next instant on the sharp spear tip of simple, unassailable logic.

I figured this warning would be clear enough even for "liberals” (it's sad that our contemporary political discourse has so compacted the meaning of this word) at the higher end of the “science comprehension” scale (ones lower in science comprehension would be even less likely to draw politically biased inferences from the data), and thus deter them from engaging in such an embarrassing display of partisan unreason.

I also owned that I myself had expected that likely I’d find a modest negative correlation between tea-party membership and science comprehension.

I did that for a couple reasons.  The first was that I really did expect that's what I'd see. I surmised, for one thing, that there was likely a correlation between religiosity and tea-party membership (there is: r = 0.16, p < 0.01), and I know religion correlates negatively with “cognitive reflection” and “science literacy” measures—in ways that empirical evidence shows make no meaningful contribution to disputes over climate change etc.

Second, I thought it would be instructive and constructive for me to show how goddam virulent the politically motivated reasoning bias is. Knowing about it is certainly no defense.  The only protection is regular infusions of valid empirical evidence administered under conditions that reveal the terrifying prospect that one will in fact display symptoms of true idiocy if one succombs to it.

But despite all this, many many many tea-party partisans succumbed to politically biased reasoning in their assessment of the evidence in my post.

Characterizing a blog post on exploratory probing of a new science comprehension measure as a “study” (indeed, a “Yale study”; I guess I was “misled” again by the “liberal media” about whether the tea party treats Ivy League universities as credible sources of information) , scores of commentators (in blogs, political opinion columns, in comments on my blog, etc) gleefully crowed that the data showed tea party members were "more science literate,” "better at understanding science" etc. than non-members.

My observation that the size of the effect was “trivial,” and my statement that the “statistical” significance level was practically meaningless and as likely to disappear as reappear in any future survey (where one observes a “p-value” very close to 0.05, then one should expect half of the attempted replications to have a p-value above 0.05 and half below that) was conveniently ignored (indeed, writers tried to add force to the reported result by using  meaningless terms like “solid” etc. to the describe it).

Also ignored, of course, was that liberals scored higher than conservatives on the same measure and in the same dataset. 

Did these zealots feel the sting of 50,000 logic arrows burrowing into their chests moments after they got done beating on them?  Doubt it.

So, what to say? I dunno, but here are four observations.

1.  Tea party members are like everyone else, as far as I can tell, when it comes to science comprehension. 

Is this something to be proud of?  I don’t think so. It means that if we were to select a tea-party member at random, there would be a 50% chance he or she would say that “antibiotics kill viruses as well as bacteria” and less than a 40% chance that he or she would be able to correctly interpret data from a simple experiment involving a new skin-rash treatment.

2.  Because tea-party members are “just like everyone else,” they too have among their number some individuals who combine a high degree of scientific knowledge with an impressively developed capacity for engaging in critical reasoning. 

But because they are like everyone else, these high "science comprehending" tea-party members will be more likely to display politically biased misinterpretations of empirical data than people who display a lower "science comprehension" apptitude. The greater their capacity to engage in analytical thinking, the more systematically they will use that capacity to ferret out evidence congenial to their predispositions and block out and rationalize away everything else.

Moreover, because others who share their values very sensibly rely on them when trying to keep up with what’s known to science, these high science-comprehending tea-party members -- just like high science-comprehending "Democrats" and "Republicans'" and "libertarians" and "socialists" et al.-- will play a principal role in transmitting the reason-effacing pathogens that pervade our polluted science communication environment.

3. Also like everyone else, tea-party members can be expected, as a result of living in a contaminated science communication environment, to behave in a manner that evinces not only an embarrassing deficiency in self-awareness but also an exceedingly ugly form of contempt for others , thereby amplifying the dynamics that are depriving them along with all the other culturally diverse citizens in the Liberal Republic of Science of the full benefit that this magnificent political regime uniquely confers on reasoning, free individuals.

4. Finally, because they are like everyone else, some of the individuals who have used their reason and freedom to join with others in a project they call the “tea-party” movement realize that they have exactly the same stake in repulsing this repulsive pathology as those individuals who’ve used their reason and their freedom to form associations like the “Democratic Party,” the “Republic Party,” the “Libertarian Party,” the “Socialist Party” etc.

They know the only remedy for this insult to our common capacity to reason is to use our common capacity to reason to fashion a new political science, one cognizant of the distinctive challenge that pluralistic democracies face in enabling their citizens to recognize the significance of the unprecedented volume of scientific knowledge that their free institutions have made it possible for them to acquire.

They are resolved to try to make all of this clear to those who share their values—and to reach out to those who don’t to make common cause with them in protecting the science communication environment that enlightened self-government depends on.

The best available evidence doesn’t tell anyone what policy is best. That depends on judgments of value, which will vary—inevitably and appropriately—among free and reasoning people.

Mine differ profoundly from those held by individuals who identify as tea party members.  We will have plenty to disagree about in the democratic process even when we agree about the facts. 

But without a reliable apprehension of the best available evidence, neither I nor they nor anyone else will be able to confidently identify which policies can be expected to advance our respective values.   

In the polluted science communication environment we inhabit,  none of us can be as confident as we have a right to be that we truly know what has come to be collectively known through science.


Cognitive Illiberalism Lecture at Penn State Dickinson School of Law (slides)

Gave lecture yesterday at Penn State Dickinson School of Law.

Focus was problem of "cognitive liberalism" -- in both law & risk regulation, and what those who study in each of these fields can learn from the other about the significance of cultural cognition for the project of perfecting liberal principles of self-governance. Slides here.

The lecture parented the core themes and roughly tracked the structure of the paper Cognitive Bias and the Constitution of the Liberal Republic of Science. Except that I substituted the "Motivated Numeracy" and enlightened self-government study for the nanotechnology risk perceptions one.  The focus on "gun control" in the former study definitely better fits the themes of the  paper.

The audience was fantastic. The law school faculty at Dickinson is flush with productive, insightful scholars -- including, e.g., David Kaye, a preeminent scholar of forensic science; Jamie Colburn, an expert in environmental law; Lara Fowler, whose exertise in mediation and alternative dispute resolution is rich with insight for improving productive and informed public engagement with decision-relevant science, an aspect of her work that accounts for her central role in the  Penn State Institutes on Energy and the Environment; and Adam Muchmore, one of whose specialties is food & drug regulation & who shared some informed reactions to my proposal that there be a "science communication impact" component of procedures in that agency & others.  These scholars and others in the audience presented me with a host of interesting and challenging comments and observations.

Must be great to be part of the Penn State intellectual community -- as student or faculty member!


Lecture on Science of Science Communication at Penn State (lecture slides)

Gave talk today at Penn State. Slides here.

Lecture was sponsored by Penn State Institutes on Energy and the Environment, which is the central component of a larger set of programs in the University that that reflect Penn State's commitment to contributing to its share to the good of integrating the practice of science and science-informed policymaking with the science of science communication.

Seems like people took a lot of interest in the finding that members of the Tea Party are not meaningfully different from the population as a whole in science comprension.  I'll say more about this topic -- and about the nature of the responses -- tomorrow.

But for now, here is some evidence showing that individuals whose outlooks are characterized by the cultural cognition worldviews all display practically equivalent levels of science comprehension too (there are differences but like those between Liberals and Conservatives & between Tea Party members and nonmembers, they are trivial from a practical standpoint).


Some data on education, religiosity, ideology, and science comprehension

No, this blog post is not a federally funded study. It's neither "federally funded" nor a "study"! Doesn't it bug you that our hard-earned tax dollars pay the salary of a federal bureaucrat too lazy to figure out simple facts like this?

Because the "asymmetry thesis" just won't leave me alone, I decided it would be sort of interesting to see what the relationship was between a "science comprehension" scale I've been developing and political outlooks.

The "science comprehension" measure is a composite of 11 items from the National Science Foundation's "Science Indicators" battery, the standard measure of "science literacy" used in public opinion studies (including comparative ones), plus 10 items from an extended version of the Cognitive Reflection Test, which is normally considered the best measure of the disposition to engage in conscious, effortful information processing ("System 2") as opposed to intuitive, heuristic processing ("System 1").  

The items scale well together (α= 0.81) and can be understood to measure a disposition that combines substantive science knowledge with a disposition to use critical reasoning skills of the sort necessary to make valid inferences from observation. We used a version of a scale like this--one combining the NSF science literacy battery with numeracy--in our study of how science comprehension magnifies cultural polarization over climate change and nuclear power.

Although the scale is designed to (and does) measure a science-comprehension aptitude that doesn't reduce simply to level of education, one would expect it to correlate reasonably strongly with education and it does (r = 0.36, p < .01). The practical significance of the impact education makes to science comprehension so measured can be grasped pretty readily, I think, when the performance of those who have and who haven't graduated from college is graphically displayed in a pair of overlaid histograms:

The respondents, btw, consisted of a large, nationally representative sample of U.S. adults recruited to participate in a study of vaccine risk perceptions that was administered this summer (the data from that are coming soon!).

Both science literacy and CRT have been shown to correlate negatively with religiosity. And there is, in turns out, a modest negative correlation (r = -0.26, p < 0.01) between the composite science comprehension measure and a religiosity scale formed by aggregating church attendance, frequency of prayer, and self-reported "importance of God" in the respondents' lives.

I frankly don't think that that's a very big deal. There are plenty of highly religious folks who have a high science comprehension score, and plenty of secular ones who don't.  When it comes to conflict over decision-relevant science, it is likely to be more instructive to consider how religiosity and science comprehension interact, something I've explored previously.

Now, what about politics?

Proponents of the "asymmetry thesis" tend to emphasize the existence of a negative correlation between conservative political outlooks and various self-report measures of cognitive style--ones that feature items such as  "thinking is not my idea of fun" & "the notion of thinking abstractly is appealing to me." 

These sorts of self-report measures predict vulnerability to one or another reasoning bias less powerfully than CRT and numeracy, and my sense is that they are falling out of favor in cognitive psychology. 

In my paper, Ideology, Motivated Reasoning, and Cognitive Reflection, I found that the Cogntive Reflection Test did not meaningfully correlate with left-right political outlooks.

In this dataset, I found that there is a small correlation (r = -0.05, p = 0.03) between the science comprehension measure and a left-right political outlook measure, Conservrepub, which aggregates liberal-conservative ideology and party self-identification. The sign of the correlation indicates that science comprehension decreases as political outlooks move in the rightward direction--i.e., the more "liberal" and "Democrat," the more science comprehending.

Do you think this helps explain conflicts over climate change or other forms of decision-relevant science? I don't.

But if you do, then maybe you'll find this interesting.  The dataset happened to have an item in it that asked respondents if they considered themselves "part of the Tea Party movement." Nineteen percent said yes.

It turns out that there is about as strong a correlation between scores on the science comprehension scale and identifying with the Tea Party as there is between scores on the science comprehension scale and Conservrepub.  

Except that it has the opposite sign: that is, identifying with the Tea Party correlates positively (r = 0.05, p = 0.05) with scores on the science comprehension measure:

Again, the relationship is trivially small, and can't possibly be contributing in any way to the ferocious conflicts over decision-relevant science that we are experiencing.

I've got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I'd be shown a modest negative correlation between identifying with the Tea Party and science comprehension.

But then again, I don't know a single person who identifies with the Tea Party.  All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).  

I'm a little embarrassed, but mainly I'm just glad that I no longer hold this particular mistaken view.

Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for. I just no longer assume that the people who happen to hold those values are less likely than people who share my political outlooks to have acquired the sorts of knowledge and dispositions that a decent science comprehension scale measures.

I'll now be much less surprised, too, if it turns out that someone I meet at, say, the Museum of Science in Boston, or the Chabot Space and Science Museum in Oakland, or the Museum of Science and Industry in Chicago is part of the 20% (geez-- I must know some of them) who would answer "yes" when asked if he or she identifies with the Tea Party.  If the person is there, then it will almost certainly be the case that that he or she & I will agree on how cool the stuff is at the museum, even if we don't agree about many other matters of consequence.

Next time I collect data, too, I won't be surprised at all if the correlations between science comprehension and political ideology or identification with the Tea Party movement disappear or flip their signs.  These effects are trivially small, & if I sample 2000+ people it's pretty likely any discrepancy I see will be "statistically significant"--which has precious little to do with "practically significant."


A fragment: The concept of the science commmunication environment

Here is a piece of something. . . .

I. An introductory concept: the “science communication environment”

In order to live well (really, just to live), all individuals (all of them—even scientists!) must accept as known by science vastly more information than they could ever hope to attain or corroborate on their own.  Do antibiotics cure strep throat (“did mine”)? Does vitamin C (“did mine”)? Does smoking cause cancer (“. . . happened to my uncle”)? Do childhood vaccinations cause autism (“. . . my niece”)? Does climate change put us at risk (“Yes! Hurricane Sandy destroyed my house!”)? How about legalizing gay marriage (“Yes! Hurricane Sandy destroyed my house!”)?

The expertise individuals need to make effective use of decision-relevant science consists less in understanding particular bodies of specialized knowledge than in recognizing what has been validly established by other people—countless numbers of them—using methods that no one person can hope to master in their entirety or verify have been applied properly in all particular instances. A foundational element of human rationality thus necessarily consists in the capacity to reliably identify who knows what about what, so that we can orient our lives to exploit genuine empirical insight and, just as importantly, steer clear of specious claims being passed off by counterfeiters or by those trading in the valueless currency of one or another bankrupt alternative to science’s way of knowing (Keil 2010).

Individuals naturally tend to make use of this collective-knowledge recognition capacity within particular affinity groups whose members hold the same basic values (Watson, Kumar & Michelsen 1993). People get along better with those who share their cultural outlooks, and can thus avoid the distraction of squabbling.  They can also better “read” those who “think like them”—and thus more accurately figure out who really knows what they are talking about, and who is simply BS’ing. Because all such groups are amply stocked with intelligent people whose knowledge derives from science, and possess well functioning processes for transmitting what their members know about what’s collectively known, culturally diverse individuals tend to converge on the best available evidence despite the admitted insularity of this style of information seeking.

The science communication environment comprises the sum total of the everyday cues and processes that these plural communities of certification supply their members to enable them to reliably orient themselves with regard to valid collective knowledge.  Damage to this science communication environment—any influence that disconnects these cues and processes from the collective knowledge that science creates—poses a threat to individual and collective well-being every bit as significant as damage to the natural environment.

Persistent public conflict over climate change is a consequence of one particular form of damage to the science communication environment: the entanglement of societal risk risks with antagonistic cultural meanings that transform positions on them into badges of membership in and loyalty to opposing cultural groups (Kahan 2012).  When that happens, the stake individuals have in maintaining their standing within their group will often dominate whatever stake they have in forming accurate beliefs. Because nothing an ordinary member of the public does—as consumer, voter, or public advocate—will have a material impact on climate change, any mistake that person makes about the sources or consequences of it will not actually increase the risk that climate change poses to that person or anyone he or she cares about. But given what people now understand positions on climate change to signify about others’ character and reliability, forming a view out of line with those in one’s group can have devastating consequences, emotional as well as material. In these circumstances individuals will face strong pressure to adopt forms of engaging information—whether it relates to what most scientists believe (Kahan, Jenkins-Smith & Braman 2011) or even whether the temperature in their locale has been higher or lower than usual in recent years (Goebbert, Jenkins-Smith, et al. 2012)—that more reliably connects them to their group than to the position that is most supported by scientific evidence.

Indeed, those members of the public who possess the most scientific knowledge and the most developed capacities for making sense of empirical information are the ones in whom this “myside bias” is likely to be the strongest (Kahan, Peters, et al. 2012; Stanovich & West 2007). Under these pathological circumstances, such individuals be expected to use their knowledge and abilities to search out forms of identity-supportive evidence that would likely evade the attention of others in their group, and to rationalize away identity-threatening forms that others would be saddled with accepting.  Confirmed experimentally (Kahan 2013a; Kahan, Peters, Dawson & Slovic 2013), the power of critical reasoning dispositions to magnify culturally biased assessments of evidence explains why those members of the public who are highest in science literacy and quantitative reasoning ability are in fact the most culturally polarized on climate change risks. Because these individuals play a critical role in certifying what is known to science within their cultural groups, their errors propagate and percolate through their communities, creating a state of persistent collective confusion.

The entanglement of risks and like facts with culturally antagonistic meanings is thus a form of pollution in the science communication environment.  It literally disables the faculties of reasoning that ordinary members of the public rely on—ordinarily to good effect—in discerning what is known to science and frustrates the common stake they have in recognizing how decision-relevant science bears on their individual and collective interests. It thus deprives them, and their society, of the value of what is collectively known and the investment they have made in thieir own ability to generate, recognize, and use that knowledge.

Protecting the science communication environment from such antagonistic meanings is thus an essential element of effective science communication--indeed of enlightened self-government (Kahan 2013b). Because the entanglement of positions on risk with cultural identity impels ordinary members of the public to use their knowledge and reason to resist evidence at odds with their groups’ views, nothing one does to make scientific information more accessible or widely distributed can be expected to counteract the forms of group polarization that this toxin generates.


Goebbert, K., Jenkins-Smith, H.C., Klockow, K., Nowlin, M.C. & Silva, C.L. Weather, Climate and Worldviews: The Sources and Consequences of Public Perceptions of Changes in Local Weather Patterns. Weather, Climate, and Society (2012).

Kahan, D. Why We Are Poles Apart on Climate Change. Nature 488, 255 (2012).

Kahan, D.M. Ideology, Motivated Reasoning, and Cognitive Reflection. Judgment and Decision Making 8, 407-424 (2013a).

Kahan, D.M. A Risky Science Communication Environment for Vaccines. Science 342, 53-54 (2013b).

Kahan, D.M., Jenkins-Smith, H. & Braman, D. Cultural Cognition of Scientific Consensus. J. Risk Res. 14, 147-174 (2011).

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The Polarizing Impact of Science Literacy and Numeracy on Perceived Climate Change Risks. Nature Climate Change 2, 732-735 (2012).

Keil, F.C. The Feasibility of Folk Science. Cognitive science 34, 826-862 (2010).

Stanovich, K.E. & West, R.F. Natural Myside Bias Is Independent of Cognitive Ability. Thinking & Reasoning 13, 225-247 (2007).

Watson, W.E., Kumar, K. & Michaelsen, L.K. Cultural Diversity's Impact on Interaction Process and Performance: Comparing Homogeneous and Diverse Task Groups. The Academy of Management Journal 36, 590-602 (1993).



Mooney's revenge?! Is there "asymmetry" in Motivated Numeracy?

Just when I thought I finally had gotten the infernal "asymmetry thesis" (AT) out of my system once and for all, this hobgoblin of the science communication problem has re-emerged with all the subtlty and charm of a bad case of shingles.

AT, of course, refers to the claim that ideologically motivated reasoning (of which cultural cognition is one species or conception), is not "symmetric" across the ideological spectrum (or cultural spectra) but rather concentrated in individuals of a right-leaning or conservative (or in cultural cognition terms "hierarchical") disposition.

It is most conspicuously associated with the work of the accomplished political psychologist John Jost, who fnds support for it in the correlation between conservatism and various self-report measures of "dogmatic" thinking. It is also the animating theme of Chris Mooney's The Republican Brain, which presents an elegant and sophisticated synthesis of the social science evidence that supports it.

I don't buy AT. I've explained why 1,312 times in previous blogs, but basically AT doesn't cohere with the best theory for politically motivated reasoning and is not supported -- indeed, is at odds with -- the best evidence of how this dynamic operates.

The best theory treats politically motivated reasoning as a form of identity-protective cognition.

People have a big stake--emotionally and materially--in their standing in affinity groups consisting of individuals of like-minded goals and outlooks. When positions on risks or other policy relevant-facts become symbolically identified with membership in and loyalty to those groups, individuals can thus be expected to engage all manner of information--from empirical data to the credibility of advocates to brute sense impressions--in a manner that aligns their beliefs with the ones that predominate in their group.

The kinds of affinity groups that have this sort of significance in people's lives, however, are not confined to "political parties."  People will engage information in a manner that reflects a "myside" bias in connection with their status as students of a particular university and myriad other groups important to their identities.

Because these groups aren't either "liberal" or "conservative"--indeed, aren't particularly political at all--it would be odd if this dynamic would manifest itself in an ideologically skewed way in settings in which the relevant groups are ones defined in part by commitment to common political or cultural outlooks.

The proof offered for AT, moreover, is not convincing. Jost's evidence, for example, doesn't consist in motivated-reasoning experiments, any number of which (like the excellent ones of Jarret Crawford and his collaborators)  have reported findings that display ideological symmetry.

Rather, they are based on correlations between political outlooks and self-report measures of "open-mindedness," "dogmatism" & the like. 

These measures --ones that consist, literally, in people's willingness to agree or disagree with statements like "thinking is not my idea of fun" & "the notion of thinking abstractly is appealing to me"--are less predictive of the disposition to critically interrogate one's impressions based on available information than objective or performance-based measures like the Cognitive Reflection Test and Numeracy.  And thse performance-based measures don't meaningfully correlate with political outlooks.

In addition, while there is plenty of evidence that the disposition to engage in reflective, critical reasoning predicts resistance to a wide array of cognitive bias, there is no evidence that these dispositions predict less vulnerability to politically motivated reasoning.

On the contrary, there is mounting evidence that such dispositions magnify politically motivated reasoning. If the source of this dynamic is the stake people have in forming beliefs that are protective of their status in groups, then we might expect people who know more and and are more adept at making sense of complex evidence to use these capacities to promote the goal of forming identity-protective beliefs.

CCP studies showing that cultural polarization on climate change and other contested risk issues is greater among individuals who are higher in science comprehension, and that individuals who score higher on the Cognitive Reflection Test are more likely to construe evidence in an ideologically biased pattern, support this view.

The Motivated Numeracy experiment furnishes additoinal support for this hypothesis. In it, we instructed subjects to perform a reasoning task--covariance detection--that is known to be a highly discerning measure of the ability and disposition of individuals to draw valid causal inferences from data.

We found that when the problem was styled as one involving the results of an experimental test of the efficacy of a new skin-rash treatment, individuals who score highest in Numeracy-- a measure of the ability to engage in critical reasoning on matters involving quantitative information--were much more likely to corretly interpret that data than those who had low or modest Numeracy scores.

But when the problem was styled as one involving the results of gun control ban, those subjects highest in Numeracy did betteronly when the data presented supported the result ("decreases crime" or "increases crime") that prevails among persons with their political outlooks (liberal Democrats and conservative Republicans, respectively). When the data, properly construed, threatened to trap them in a conclusion at odds with their political outlooks, the high Numeracy people either succumbed to a tempting but lotically specious response to the problem or worked extra hard to pry open some ad hoc, confabulatory escape hatch.

As a result, higher Numeracy experiment subjects ended up even more polarized when considering the same data -- data that in fact objectively supported one position more strongly than the other -- than subjects who subjects who were less adept at making sense of empirical information.

But ... did this result show an ideological asymmetry?!

Lots of people have been telling me they see this in the results. Indeed, one place where they are likely to do so is in workshops (vettings of the paper, essentially, with scholars, students and other curious people), where someone will almost say, "Hey, wait! Aren't conservative Republicans displaying a greater 'motivated numeracy' effect than liberal Democrats? Isn't that contrary to what you said you found in x paper? Have you called Chris Mooney and admitted you were wrong?"

At this point, I feel like I'm talking to a roomful of people with my fly open whenver I present the paper!

In fact, I did ask Mooney what he thought -- as soon as we finished our working paper.  I could see how people might view the data as displaying an asymmetry and wondered what he'd say.

His response was "enh."

He saw the asymmetry, he said, but told me he didn't think it was all that interesting in relation to what the study suggested was the extent of the vulnerability of all the subjects, regardless of their political outlooks, to a substantial degradation in reasoning when confronted with data that disappointed their political predispositions--a point he then developed in an interesting Mother Jones commentary.

That's actually something I've said in the past, too--that even if there were an "asymmetry" in politically motivated reasoning, it's clear that the problem is more than big enough for everyone to be a serious practical concern.

Well, the balanced, reflective person that he is, Mooney is apparently able to move on, but I, in my typical OCD-fashion, can't...

Is the asymmetry really there? Do others see it? And how would they propose that we test what they think they see so that they can be confident their eyes are not deceiving them?

The location of the most plausible sighting--and the one where most people point it out--is in Figure 6, which presents a lowess plot of the raw data from the gun-control condition of the experiment:

What this shows, essentially, is that the proportion of the subjects (about 800 of them total) who correctly interpreted the data was a function of both Numeracy and political outlook. As Numeracy increases, the proportion of subjects selecting the correct answer increases dramatically but only when the correct answer is politically congenial ("decreases crime" for liberal Democrats, and "increases crime" for conservative Republicans; subjects' political outlooks here are determined based on the location of their score in relation to the mean on a continuous measure that combined "liberal-conservative" ideology & party identification).

But is there a difference in the pattern for liberal Democrats, on the on hand, and conservative Republicans, on the other?

Those who see the asymmetry tend to point to the solid black circle. There, in middling range of Numeracy, conservative Republicans display a difference in their likelihood of getting the correct answer based on which experiment condition ("crime increases" vs. "crime decreases"), but liberal Democrats don't.  

A ha! Conservative Republicans are displaying more motivated reasoning!

But consider the dashed circle to the right.  Now we can see that conservative Republicans are becoming slightly more likely to interpret the data correctly in their ideologically uncongenial condition ("crime decreases") -- whereas liberal Democrats aren't budging in theirs ("crime increases").  

A ha^2! Liberal Democrats are showing more motivated Numeracy--the disposition to use quantitative reasoning skills in an ideologically selective way!

Or we are just looking at noise.  The effects of an experimental treatment will inevitably be spread out unevenly across subjects exposed to it.  If we split the sample up into parts & scrutinize the effect separately in each, we are likely to be mistake random fluctuations in the effect for real differences in effect among the groups so specified.

For that reason, one fits to the entire dataset a statistical model that assumes the treatment has a particular effect--one that informed the experiment hypothesis.  If the model fits the real data well enough (as reflected in conventional standards like p < 0.05), then one can treat what one sees -- if it looks like what one expected -- as a corroboration of the study prediction.

Click me!!!We fit a multivariate regression model to the data that assumed the impact of politically motivated reasoning (reflected in the difference in likelihood of getting the answer correct conditional on its ideological congeniality) would increase as subjects' Numeracy increases. The model fit the data quite well, and thus, for us, corroborated the pattern we saw in Figure 6, which is one in which politically motivated reasoning and Numeracy interact in the manner hypothesized.

The significance of the model is hard to extract from the face of the regression table that reports it, but here is a graphical representation of what the model predicts we should see among subjects of different political outlooks and varying levels of Numeracy in the various experimental conditions:

The "peaks" of the density distributions are, essentially, the point estimates of the model, and the slopes of the curves (their relative surface area, really) a measure of the precision of those estimates.

The results display Motivated Numeracy: assignment to the "gun control" conditions creates political differences in the likelihood of getting the right answer relative to the assignment to the "skin treatment" conditions; and the size of those differences increases as Numeracy increases.

Now you might think you see asymmetry here too!  As was so for figure depicting the raw data, this Figure suggests that low Numeracy conservative Republicans' performance is more sensitive to the experimental assignment. But unlike the raw-data lowess plot, the plotted regression estimates suggest that the congeniality of the data had a bigger impact on the performance of higher Numeracy conservative Republicans, too!

But this is not a secure basis for inferring asymmetry in the data.  

As I indicated, the model that generated these predicted probabilities included parameters that corresponded to the prediction that political outlooks, Numeracy, and experimental condition would all interact in determining the probability of a correct response.  The form of the model assumed that the interaction of Numeracy and political outlooks would be uniform or symmetric.

The model did generate predictions in which the difference in the impact of politically motivated reasoning was different for conservative Republicans and liberal Democrats at low and high levels of Numeracy.

But that difference is attributable -- necessarily -- to other parameters in the model, including the point along the Numeracy scale at which the probability of the correct answer changes dramatically (the shape of the "sigmoid" function in a logit model), and the tendency of all subjects, controlling for ideology, to get the right answer more often in the "crime increases" condition.

I'm not saying that the data from the experiment don't support AT.  

I'm just saying that to support the inference that it does, one would have to specify a statistical model that reflected the hypothesized asymmetry and see whether it fits the data better than the one that we used, which assumes a uniform or symmetric effect.

I'm willing to fit such a model to the data and report the results.  But first, someone has to tell me what that model is!  That is, they have to say, in conceptual terms, what sort of asymmetry they "see" or "predict" in this experiment, and what sort of statistical model reflects that sort of pattern.

Then I'll apply it, and announce the answer! 

If it turns out there is asymmetry here, the pleasure of discovering the world is different from what I thought will more than offset any embarrassment associated with my previously having announced a strong conviction that AT is not right.

So-- have at it!  

To help you out, I've attached a slide show that sketches out seven distinct possible forms of asymmetry.  So pick one of those or if you think there is another, describe it.  Then tell me what sort of adjustment to the regression model we used in Table 1 would capture an asymmetry of that sort (if you want to say exactly how the model should be specified, great, but also fine to give me a conceptual account of what you think the model would have to do to capture the specified relationship between Numeracy, political outlooks, and the experimental conditions).

Of course, the winner(s) will get a great prize!  Winning, moreover, doesn't consist in confirming or refuting AT; it consists only in figuring out a way to examine this data that will deepen our insight.

In empirical inquiry, it's not whether your hypothesis is right or wrong that matters; it's how you extract a valid inference from observation that makes it possible to learn something.

Click on this -- and you too will go insane!


Knowledge is not scary; being *afraid to know* is

Andrew Revkin directed me and a collection of others to a very well-done talk he gave on the state of social science research on climate-science communication. The subject line of the email was "the scariest climate science is the social science..." Well, that didn't match at all the message of AR's column or his talk. But it did what he likely intended, which was provoke me (likely other recipients will be provoked too) to respond the suggestion that there is something "scary" -- or maybe "hopeless" -- about the sort of research that I and others with whom I'm in scholarly conversation do. That idea is out there, not in Andy's remarks but in the attitudes of many people who are worried about the state of public engagement with climate science, & is dead wrong. Here is what I said:

I see nothing scary in the state of the research on the dynamics of public conflict on climate change.

The scary thing would be not knowing which of the various plausible dynamics that could be generating persistent public conflict over climate science really are doing so, and to what extent. There are more plausible candidates--plausible because rooted in valid insight on the mechanisms of risk perceptions-- then can be true. Only empirical investigation can help to winnow down the possibilities (steer us clear of endless story-telling) and focus attention on the most consequential, most tractable sources of the failure of reasoning people to converge on the best available evidence (as they normally do; the number of matters addressed by decision-relevant science on which see conflict of this sort relative to the number in which we don't is minuscule, albeit fraught w/ significance).

But that is the point of doing such research: to figure out what is really going on, so that genuinely responsive strategies for promoting open-minded and constructive public engagement can be fashioned. I believe that we now know a tremendous amount about the sources of persistent public conflict over decision-relevant science thanks to empirical research on risk perception and communication amassed over the course of over three decades.

It is precisely b/c of that work, and the systematic application of it to problems involving climate science communication, that we are now in a position to form sensible hypotheses about what sorts of processes might neutralize the dynamics in question. Using the same methods that have helped to generate a more focused picture of what the problem really is, we can enlarge our understanding of how to remove the conditions that are disabling ordinary people from using their ordinarily reliable faculties for recognizing what's known to science.

But we will have to use the same methods: disciplined, structured observation and inference. There are more plausible accounts of what might work to fix the problem than can be true too.

So we must do more empirical study, and do it, I think, primarily in the field. Social scientists should collaborate with experienced communicators who can identify using their situation sense what sorts of interventions in the real-world might reproduce in their real-world settings the sorts of positive results that people have observed in lab studies. The latter have more reliable, more informed insights on that than the former; but the former can help the latter, both by sharing with them what is known as a result of empirical inquiry into science communication and by enabling these real-world communicators to collect and evaluate evidence of what really works and what doesn't -- and then to tell others about it, so they can use that knowledge, too, and build on it.

I don't think we should be scared by what we have learned about the disabling effect of a polluted science communication environment on our capacity to engage in collective reason.

That some people might be afraid of this--because it shows, say, that they have made mistakes in the past, or that the world doesn't work as they might wish that it does-- is much more frightening, for they are likely to cling in a determined, fearful, ineffectual way to mistaken understandings.

So far from making us afraid, the vast amount we have learned should make us confident that we can use our collective reason, guided by disciplined methods of empirical observation and inference, to repair the deliberative environment on which enlightened self-government depends and indeed to protect it from such degradation in the future.


Well, things are going slowly in the kitchen, so here's another "vaccine risk perception" appetizer -- on the house

Okay, so my goal was to get a big (N = 2000) study that combines public opinion and experimental analysis of vaccine risk perceptions done by today. 

I wanted to do that mainly so the evidence would be out there at the same time as my Perspective piece today in Science, which uses the HPV vaccine disaster and empirically uniformed risk communication about public attitudes toward childhood vaccines to draw attention to the need for a more systematic policy of "science communication environment protection," both in government and in relevant professional and civic institutions.

But it's easier to be in a magazine than to run one, which requires among other things meeting all kinds of deadlines etc.  

I'm not going to meet mine for getting "the report" out.  I want to fine tune some things (including estimates made with survey weights that I've now fine tuned more precisely).  Maybe it won't matter but I'd rather feel 100% comfortable before calling peopel's attention to something that I hope can help them make decisions of consequence.

But I'm okay giving you a bit more to chew on -- more "conventional wisdom" that has zero evidence and when examined turns out to be untrue (like the idea that there is some connection between positions on climate change & evolution & concern about vaccines).

Know how people say, "belief that vaccines cause autism is for the left what climate denial is for the right ..." blah blah? I guess that's based on a poll-- of Robert Kennedy, Jr.

Here's evidence from a nationally representative sample of 900 ordinary people. It's a cool lowess plot that shows how political outlooks shape differences in perceived vaccine risk perceptions.

The y-axis uses the industrial strength risk perception measure for vaccines, global warming, guns, and marijuana legalization, and the x-axis is a continuous right-left ideology measure formed by aggregating party affiliation and liberal-conservative ideology.   

Gee, becoming progressively more liberal doesn't make people think childhood vaccines are more risky.

Actually, people become more concerned as they become more conservative.  

But the effect is genuinely tiny --  as you can see by holding it up to comparison w/ other politically contested risks as a benchmark.

You can't figure out the practical significance of variation by looking at a correlation coefficient or a complicated structural equation model. You have to know what sort of variance is being explained/modeled 

Here it's the difference between thinking it is genuinely asinine to worry about vaccines and thinking that it's just really really dumb.

And to complement yesterday's data, here is a look at how perceptions of the balance of vaccine risks and benefits (y-axis!) relate to science comprehension (measured with a pretty powerful composite scale that fortified the NSF's science indicator battery with an extended "Cognitive Reflection Test" battery) and also to religiosity (again, a highly reliable composite scale, here comprising church attendance, "importance of God," and frequency of prayer):

 Well?  There are relationships-- the balance tips a tad toward benefit as science comprehension increases and toward risk a tad as religiosity does.

But again, these are small effects, in statistical terms, and irrelevant ones in practical ones.  Those at both ends of both spectra are concentrated toward the "benefit greater than risk" end of the measure.

It's not enough to explain variance; one has to know what the difference is that is being explained.

Actually, though, the religiosity & science comp relationship is more interesting than this picture lets on. It turns out that these two interact. So even though it looks like science comprehension has no effect, it does-- but it depends on how religious one is!  Sound familiar?  Same thing as in climate change, where the impact of science comprehension turns on whether one has a cultural predisposition toward crediting or dismissing environmental risks.

Except not really

This figure plots the interaction in relation to a composite scale that combines a bunch of indicia into a (very reliable!) scale that measures perception of the value of universal vaccination as a public health measure.  That scale is normalized -- the units are standard deviations.  Same thing with the "science comprehension measure."

So basically, we are talking about a shift of about 1/4 of standard deviation in every standard deviation of difference in standard deviation in science comprehension.

Hey-- I could put three "***" next to the coefficient that measures the interaction b/c it is really really significant. But only in a "statistical" sense, not a practical one.

Unlike people who are below average in religiosity, people who are above average in religiosity don't become even more enamored of vaccines as they become more science comprehending.  But everyone in this story loves vaccines-- the mean on the scale reflects things like 75% of people agreeing with the statement that "I am confident in the judgment of the public health officials who are responsible for identifying generally recommended childhood vaccinations."

Yeah but only super confident--why not super duper, like people who are below average in religiosity and above in science comprehension?

So maybe you see where this is going?  

But actually, the report is not "all about nothing."

The something has to do with what happens when you stick in people's faces information that tells them that "anti-vaccine," "climate change skepticism," "denial of evolution" are all of a piece in some massive assault on science in our socciety.... 

So more on that. Tomorrow. I think!


Busy lately but tomorrow -- lots of data on vaccine risk perceptions

I'm not dead (I was abducted and held captive by aliens for 70 yrs, but they kept their promise to return me to present without anyone experiencing me as having been absent, so that has nothing to do with it), just deep underwater.

But tomorrow some interesting things: the results of a large national opinion study of public perceptions of the risk of childhood vaccines (including an experimental component on the impact of typical forms of communication about public attitudes and behavior). 

A preview ... 

The trope ...




... some actual evidence 

Tune in for more details!


"So what?" vs. "You tell me!"

A thoughtful persons writes,

Thanks for this study [on "Motivated Numeracy & Enlightened Self-Government.

So, what?  As a consumer of your work (rather than as a fellow academic and/or peer reviewer), I need to know how to use it. I'm a journalist and world citizen. The insights you provide join others that say that people, no matter how ignorant or how lackadaisical toward subjects of common interest, would rather fight than switch, that American political party affiliation is bound so closely to our self-identification that we will assert it and defend it irrationally. Stuff like that.

Please don't tell me it's not your job to write a "therefore" codicil. I know that, but outside the boundaries of academia there's a natural impulse when confronting potentially useful information to wonder how best to use it. I'm among those guys.

My answer:

Dear X:

Thanks for the note. 

2 answers: 

1. Long, less interesting: I and my collaborators have done studies & written papers that try to address the "what is to be done?" question once one accepts (if one does; the matter certainly remains open, and in need of more investigation) that the source of the "science communication problem" isn't any defect in the public's knowledge or reasoning ability but rather the contamination of the science communication environment with toxic partisan meanings that disable their normally reliable ability to figure out what's known by science.  Some conjecture on possible strategies for decontaminating the science communication enviornment; others test one or another of these; and still others say how to go about identifying possible #scicomm environment protection strategies (by evidence-based means, of course).  A sampling...

2. Shorter, more urgent.  You tell me 

Seriously. You are a professional communicator with a wealth of experienced-informed knowledge about how to communication what to whom. I'm clueless. don't do science communication; I study it. But b/c I study it -- empirically -- I think I can supply you with information of genuine consequence.  A study like this tries to identify which of the many many  plausible accounts of what is going on is truly the source of the problem & which not; it does that by creating a model from which the cacophony of influences that exist in any particular setting are more-or-less stripped away so that we can reliably observe & manipulate cognitive mechanisms of interest. Well, here you go then.  Here's what I see; it's this ("of coruse; obviously!") & not that (something that appeared just as obvious; this is the nub of the problem, of course).  Now that you have more reason to believe that this is what's going on, surely you, as someone with a wealth of experienced-informed knowledge who understands all the things I stripped out of my model, can identify somewhere between 50 & 10,000 things that might engage this genuinely consequential mechanism that the study identified!  Realize, however, that although they are all "obvious" only some will genuinely reproduce in the field things that I (or others doing what I do) can manage to do in the lab.  However, that I can help you with. Pick 1 or 2 or 3 of the things you think will engage the mechanisms I've identified in a constructive way, and I'll measure what happens & give you more information ....  

But you tell me; it's your move.  

Your fellow citizen (of the Liberal Republic of Science),



Cultural cognition and "in group" dynamics: informational vs. social effects

A thoughtful person who had read some CCP studies asked me a really good question about the relationship between “in group” dynamics and cultural cognition.

The behavioral and cognitive influences of being affiliated with one group—and unaffiliated with another, competing one—have been a central focus of social psychology for decades. This research pervasively informs our study of cultural cognition. 

But neither I nor my collaborators have offereed a focused and systematic account that situates the mechanisms we are observing in relation to that more general body of work. We should do that. My response to the query gestures toward such an account.

Here is the question:

I've been thinking about [the studies and our previous correspondence], and perhaps a simple 'in-group/out-group' model might explain a lot. The starting point is that the problem is so complicated that no layman is going to master the details in their spare time.  Most people who work on it full time only understand a part of it!  I'm certainly in the latter category.  So people reach their conclusions based on advice that feels right. . . .

[F]olks heavily weight information by who delivers that information.  At first I thought the selectivity was a symptom - e.g. listen to messages you want to hear.  But listening to people you trust sounds a lot more believable, and a lot less evil.  I do it myself. . . .

Do I have this right?  Or am I off in the weeds?

My response:

The interpretation [you] propose-- that the cultural cognition reflects tendency of ordinary people to weight the members of some important "in group" when forming assessments of what science knows -- seems right to me. But I think I would want to add more specificity to it, both to make it more reliable in explaining or predicting "who believes what about what" and to help assess how we should feel about this dynamic. 

Here are some reactions: 

1. The impact of "in group" dynamics on belief and attitude formation is known to be very substantial. But it is also known to comprise many diverse mechanisms

Some are, essentially, "informational." E.g, people might be exposed disproportionately to the views of those with whom they have the most contact, and so, if they are effectively treating the views of others as "evidence" of what is true, will end up with a sample biased toward crediting the position of others who share their views.

Others are "social."  Individuals might be unconsciously motivated to form views that fit the ones held by others with whom they have important connections in order to avoid the reputational consequences of holding "deviant" opinions. This is identity-protective cognition

Indeed, there can be an interaction between these influences. E.g., individuals might stifle expression of "deviant" views in order to avoid reputational consequences, thereby denying others in the group evidence from which they might infer that both that the dominant view is incorrect and that they will not be judged negatively for holding the alternative position. 

2.  There is also the question of which "in groups" matter.   

In "lab" settings, one can generate "in group" effects in completely contrived & artificial ways (by making participants where different colored "badges," e.g.).   

But outside the lab, things can't be so plastic; we are all members of so many "in groups" (graduates of particular universities, residents of particular cities, fans of particular athletic teams, members of professions, etc.) that the "in group" effect would get washed out by noise if all groups matter in all contexts for all things to the same extent! 

3.  The "cultural cognition" framework, then, tries to be specific on these matters.   

Using a theory associated with anthropologist Mary Douglas & political scientist Aaron Wildavsky, it tries to specify what the relevant in-group affiliation is & what the mechanisms are through which it influences the formation of perceptions of risk and like fact, at least among ordinary members of the public.   

The "cultural worldview" scales are a means of measuring the degree of affinity to groups that are believed to be the ones of consequence.  We use experiments to test hypotheses about the diverse mechanisms that connect membership in those groups to risk perceptions. 

4. I’m confident that the mechanisms we identify with cultural cogntion make both an informational and a social contribution to individual’s apprehension of decision-relevant science. 

In fact, I think the informational contiribution is likely of foundational importance. Like you say, individuals need to accept as known by science more than they can possibly comprehend on their own. Accordingly, they develop an expertise in knowing who knows what about what—one the reliability of which will be higher when they use it inside of affinity groups, whose members they can more efficiently interact and reliably read.  

Usually, too, these groups, all of whom have their fair share of informed and educated and diversely experiened people who make it their businesss to know what’s known, guide their individual members toward the best available evidence relevant to their well being (in groups that didn’t do that reliably wouldn’t be of consequence in people’s lives for long!), and thus promote convergence on decision-relevant science among culturally diverse people. 

But under unusual conditions, positions on risks or other facts addressed by decision-relevant science can become attached to social meanings that make them emblematic of membership in, and loyalty to, one’s group.  When that happens, the social influence component of in-group affiliation will be dominant and will in fact frustrate convergence of diverse groups on the best available evidence—to the detriment of their individual members’ collective well- being. 

That’s what drives conflicts over climate change, nuclear power, gun control, the HPV vaccine, etc.  With respect to those kinds of issues—ones attended by antagonistic meanings—individuals are aggressively, albeit unconsciously fitting their assessments of evidence to views that predominate in their group in a manner that cannot be explained in a satisfactory way w/ a model that sees the effect as "informational" only.   

a. One powerful source of evidence for this, I think, comes from studies in which we manipulate the *content* of the information and hold the *messenger* constant.  In Cultural Cognition of Scientific Consensus, subjects are recognizing the expertise of a highly credentialed scientist conditional on the position he espouses being consistent with the one that predominates in their group.  At that point, they can't be seen as "choosing" to credit an in-group member on a technical matter  -- the scientist is the only information source on hand, and they are crediting him as someone "who knows what he is talking about" on a technical matter or not depending on whether doing so helps them to persist in holding a group that predominates in their group.   

Or consider They Saw a Protest.  There we did an experiment in which individuals viewed a *digital film* of a political protest & reported seeing acts of intimidation or alternative noncoercive speech conditional on whether the conclusion -- "people who advocate X are violent/reasoned" -- connected them to their groups.  No in-group member telling them anything-- but a form of information processing that was posited to arise from the same mechanisms that are at work in conflicts over risk perception.   

b.  An even more powerful piece of evidence comes from experiments in which we show that the tendency to form group-congruent beliefs originates not in crediting any information source but in a biased use of the sort of reasoning dispositions & capacities that one would have to use to make sense of technical information oneself.   

We've done two experiments like that, both of which are in the nature of follow-ups to our study of how scientific literacy enhances polarization on climate change.  One of these experiments showed that "cognitive reflection," a disposition to use reflection, analytical reasoning as opposed to emotional, heuristic-driven reasoning accentuates ideological polarization when people are assessing a complex conceptual report relating to empirical data.   

The other shows that subjects high in Numeracy, a capacity to reason with quantitative data, use that capacity selectively when drawing inferences from data on an ideologically controversial topic (gun control).  In these cases, again, no one is deferring to a trusted in-group member on a technical matter (I've attached a draft paper in which describe the study; comments welcome!). People are reasoning for themselves, And the ones who we would recognize as being the best reasoners are the ones who are displaying the in-group effect to the greatest extent.   

c. I think it makes perfect sense, sadly, that membership in the sorts of groups who share the "worldviews" we measure would generate a "social" as opposed to an informational effect only on belief formation. 

What we are measuring are outlooks that likely will figure in the bonds of people who are intimately connected with one another.  The benefits people derive from such associations are immense.  The formation of views that could estrange people from those with whom they share those ties, then, could be devastating.   

Meanwhile, for ordinary individuals at least, the cost of forming mistaken understandings on the science of things like climate change is essentially zero.  Nothing they do in their individual lives-- as consumers, as voters, as participants in public discourse -- will have a material impact on risk or on policymaking; they don't matter enough as individuals to have that impact.  So nothing the do in those capacities based on a mistake about the science can affect the risk they or anyone else they care about faces.   

Thus, the cost of being out of line w/ group positions being high, and the cost of being out of line w/ decision relevant science on societal risks being low or zero, I think rational people will form patterns of engaging information situation that more reliably connect them w/ their group than with the best available evidence.  Moreover, the ones who are better at reasoning -- the ones who are higher in science literacy, higher in cognitive reflection, higher in Numeracy-- will be all the more "successful" in using their reason this way. 

5. It is based on this that I would react to the suggestion that connecting cultural cognition to an "in group" effect makes it sound more benign ("less evil").   

I think what I've described is very malign-- very evil!  The sorts of in-group effects here generate a predictable pressure -- one mediated by our own capacity for individual rationally -- that poses a tremendous threat to our collective well-being.   

The entire spectacle, moreover, assaults and insults our reason-- the quality that marks our species as worthy of awe -- and mocks our fitness for self-government-- the form of political life that is in fact the one that our special status as reasoning beings compels we be afforded! 

6.  I'd be in despair, really, except for one thing: I think we can use our reason, too, to address the problem.  The problem -- the denigration of our reason, and the resulting breakdown of processes of enlightened collective action -- is one that the members if all these groups have a stake in solving, since it puts them all at risk.   

Moreover, the problem is one that admits of a solution.  The sort of polarization we see on issues like climate change, nuclear power, the HPV vaccine, guns, etc. -- is not the norm. Usually the strategies we use, including the informational benefit we get from trusting those with whom we have deep affinities, brings us into convergence.  The pathology that generates this very bad, very unusual state, It occurs when something very weird happens -- when a policy-relevant fact that admits of scientific investigation somehow becomes a badge of membership in & loyalty to one of these affinity groups, the state that generates the malign social in-group effect I have described.   

That is not a problem in us, in our reasoning capacity; it is a problem in our science communication environment-- the common deliberative space in which we exercise our normal and normally reliable faculties for recognizing what's known to science. 

Protecting the science communication environment -- thereby enabling culturally diverse people, who of course look to different sources to certify what is known, to converge on the best available evidence -- is exactly what the science of science communication is about.

Some references

Brewer, M.B., Kramer, R.M., Leonardelli, G.J. & Livingston, R.W. Social Cognition, Social Identity, and Intergroup Relations : A Festschrift in Honor of Marilynn Brewer. (Psychology Press, New York; 2011).

Cohen, G.L. Party over Policy: The Dominating Impact of Group Influence on Political Beliefs. J. Personality & Soc. Psych. 85, 808-822 (2003).

Giner-Sorolla, R. & Chaiken, S. Selective Use of Heuristic and Systematic Processing under Defense Motivation. Personality and Social Psychology Bulletin 23, 84-97 (1997).

Giner-Sorolla, R., Chaiken, S. & Lutz, S. Validity Beliefs and Ideology Can Influence Legal Case Judgments Differently. Law and Human Behavior 26, 507-526 (2002).

Kuran, T. Private Truths, Public Lies. (1996).

Mackie, D.M. Systematic and Nonsystematic Processing of Majority and Minority Persuasive Communications. Journal of Personality and Social Psychology 53, 41-52 (1987).


Mackie, D.M. & Skelly, J.J. The Social Cognition Analysis of Social Influence: Contributions to the Understanding of Persuasion and Conformity.  (1994).


Mackie, D.M., Worth, L.T. & Asuncion, A.G. Processing of Persuasive in-Group Messages. Journal of Personality and Social Psychology 58, 812-822 (1990).

Sherman, D.K. & Cohen, G.L. Accepting Threatening Information: Self-Affirmation and the Reduction of Defensive Biases. Current Directions in Psychological Science 11, 119-123 (2002).




The quality of the science communication environment and the vitality of reason

The Motivated Numeracy and Enlightened Self-Government working paper has apparently landed in the middle of an odd, ill-formed debate over the "knowledge deficit theory" and its relevance to climate-science communication. I'm not sure, actually, what that debate is about or who is involved.  But I do know that any discussion framed around the question "Is the knowledge-deficit theory valid?" is too simple to generate insight. There are indeed serious, formidable contending accounts of the the nature of the "science communication problem"--the failure of citizens to converge on the best available evidence on the dangers they face and the efficacy of measures to abate them.  The antagonists in any "knowledge-deficit debate" will at best be stick-figure representations of these positions. 

Below is an excerpt from the concluding sections of the MNESG paper. It reflects how I see the study findings as contributing to the position I find most compelling in the scholarly discussion most meaningfully engaged with the science communication problem. The excerpt can't by itself supply a full account of the nature of the contending positions and the evidence on which they rest (none is wholly without support). But for those who are motivated to engage the genuine and genuinely difficult questions involved, the excerpt might help to identify for them paths of investigation that will lead them to locations much more edifying than the ones in which the issue of "whether the knowledge deficit theory is valid" is thought to be a matter worthy of discussion.

5.2. Ideologically motivated cognition and dual process reasoning generally

The ICT hypothesis corroborated by the experiment in this paper conceptualizes Numeracy as a disposition to engage in deliberate, effortful System 2 reasoning as applied to quantitative information. The results of the experiment thus help to deepen insight into the ongoing exploration of how ideologically motivated reasoning interacts with System 2 information processing generally.

As suggested, dual process reasoning theories typically posit two forms of information processing: a “fast, associative” one “based on low-effort heuristics”, and a “slow, rule based” one that relies on “high-effort systematic reasoning” (Chaiken & Trope 1999, p. ix). Some researchers have assumed (not unreasonably) that ideologically motivated cognition—the tendency selectively to credit or discredit information in patterns that gratify one’s political or cultural predispositions—reflects over-reliance on the heuristic forms of information processing associated with heuristic-driven, System 1 style of information processing (e.g., Lodge & Taber 2013; Marx et al. 2007; Westen, Blagov, Harenski, Kilts, & Hamann, 2006; Weber & Stern 2011; Sunstein 2006).

There is mounting evidence that this assumption is incorrect. It includes observational studies that demonstrate that science literacy, numeracy, and education (Kahan, Peters, Wittlin, Slovic, Ouellette, Braman & Mandel 2012; Hamilton 2012; Hamilton 2011)—all of which it is plausible to see as elements or outgrowths of the critical reasoning capacities associated with System 2 information processing—are associated with more, not less, political division of the kind one would expect if individuals were engaged in motivated reasoning.

Experimental evidence points in the same direction. Individuals who score higher on the Cognitive Reflection Test, for example, have shown an even stronger tendency than ones who score lower to credit evidence selectively in patterns that affirm their political outlooks (Kahan 2013). The evidence being assessed in that study was nonquantitative but involved a degree of complexity that was likely to obscure its ideological implications from subjects inclined to engage the information in a casual or heuristic fashion. The greater polarization of subjects who scored highest on the CRT was consistent with the inference that individuals more disposed to engage systematically with information would be more likely to discern the political significance of it and would use their critical reasoning capacities selectively to affirm or reject it conditional on its congeniality to their political outlooks.

The experimental results we report in this paper display the same interaction between motivated cognition and System 2 information processing. Numeracy predicts how likely individuals are to resort to more systematic as opposed to heuristic engagement with quantitative information essential to valid causal inference. The results in the gun-ban conditions suggest that high Numeracy subjects made use of this System 2 reasoning capacity selectively in a pattern consistent their motivation to form a politically congenial interpretation of the results of the gun-ban experiment.  This outcome is consistent with that of scholars who see both systematic (or System 2) and heuristic (System 1) reasoning as vulnerable to motivated cognition (Cohen 2003; Giner-Sorolla & Chaiken 1997;  Chen, Duckworth & Chaiken 1999).

These findings also bear on whether ideologically motivated cognition is usefully described as a manifestation of “bounded rationality.” Cognitive biases associated with System 1 reasoning are typically characterized that way on the ground that they result from over-reliance on heuristic patterns of information processing that reflect generally adaptive but still demonstrably inferior substitutes for the more effortful and more reliable type of information processing associated with System 2 reasoning (e.g., Kahneman 2003; Jolls, Sunstein & Thaler 1998).

We submit that a form of information processing cannot reliably be identified as “irrational,” “subrational,” “boundedly rational” or the like independent of what an individuals’ aims are in making use of information. It is perfectly rational, from an individual-welfare perspective, for individuals to engage decision-relevant science in a manner that promotes culturally or politically congenial beliefs. Making a mistake about the best-available evidence on an issue like climate change, nuclear waste disposal, or gun control will not increase the risk an ordinary member of the public faces, while forming a belief at odds with the one that predominates on it within important affinity groups of which they are members could expose him or her to an array of highly unpleasant consequences (Kahan 2012). Forms of information processing that reliably promote the stake individuals have in conveying their commitment to identity-defining groups can thus be viewed as manifesting what Anderson (1993) and others (Cohen 2003; Akerlof and Kranton 2000; Hillman 2010; Lessig 1995) have described as expressive rationality.

If ideologically motivated reasoning is expressively rational, then we should expect those individuals who display the highest reasoning capacities to be the ones most powerfully impelled to engage in it (Kahan et al. 2012). This study now joins the rank of a growing list of others that fit this expectation and that thus supports the interpretation that ideologically motivated reasoning is not a form of bounded rationality but instead a sign of how it becomes rational for otherwise intelligent people to use their critical faculties when they find themselves in the unenviable situation of having to choose between crediting the best available evidence or simply being who they are.

6. Conclusion: Protecting the “science-communication environment”

To conclude that ideologically motivated reasoning is expressively rational obviously does not imply that it is socially or morally desirable (Lessig 1995). Indeed, the implicit conflation of individual rationality and collective wellbeing has long been recognized to be a recipe for confusion, one that not only distorts inquiry into the mechanisms of individual decisionmaking but also impedes the identification of social institutions that remove any conflict between those mechanisms and attainment of the public good (Olson 1965). Accounts that misunderstand the expressive rationality of ideologically motivated cognition are unlikely to generate reliable insights into strategies for counteracting the particular threat that persistent political conflict over decision-relevant science poses to enlightened democratic policymaking.

Commentators who subscribe to what we have called the Science Comprehension Thesis typically propose one of two courses of action. The first is to strengthen science education and the teaching of critical reasoning skills, in order better to equip the public for the cognitive demands of democratic citizenship in a society where technological risk is becoming an increasingly important focus of public policymaking (Miller & Pardo 2000). The second is to dramatically shrink the scope of the public’s role in government by transferring responsibility for risk regulation and other forms of science-informed policymaking to politically insulated expert regulators (Breyer 1993). This is the program advocated by commentators who believe that the public’s overreliance on heuristic-driven forms of reasoning is too elemental to human psychology be corrected by any form of education (Sunstein 2005).

Because it rejects the empirical premise of the Science Comprehension Thesis, the Identity-protective Cognition Thesis takes issue with both of these prescriptions. The reason that citizens remain divided over risks in the face of compelling and widely accessible scientific evidence, this account suggest, is not that that they are insufficiently rational; it is that the that they are too rational in extracting from information on these issues the evidence that matters most for them in their everyday lives. In an environment in which positions on particular policy-relevant facts become widely understood as symbols of individuals’ membership in and loyalty to opposing cultural groups, it will promote people’s individual interests to attend to evidence about those facts in a manner that reliably conforms their beliefs to the ones that predominate in the groups they are members of. Indeed, the tendency to process information in this fashion will be strongest among individuals who display the reasoning capacities most strongly associated with science comprehension.

Thus, improving public understanding of science and propagating critical reasoning skills—while immensely important, both intrinsically and practically (Dewey 1910)—cannot be expected to dissipate persistent public conflict over decision-relevant science. Only removing the source of the motivation to process scientific evidence in an identity-protective fashion can. The conditions that generate symbolic associations between positions on risk and like facts, on the one hand, and cultural identities, on the other, must be neutralized in order to assure that citizens make use of their capacity for science comprehension.[1]

In a deliberative environment protected from the entanglement of cultural meanings and policy-relevant facts, moreover, there is little reason to assume that ordinary citizens will be unable to make an intelligent contribution to public policymaking. The amount of decision-relevant science that individuals reliably make use of in their everyday lives far exceeds what any of them (even scientists, particularly when acting outside of the domain of their particular specialty) are capable of understanding on an expert level. They are able to accomplish this feat because they are experts at something else: identifying who knows what about what (Keil 2010), a form of rational processing of information that features consulting others whose basic outlooks individuals share and whose knowledge and insights they can therefore reliably gauge (Kahan, Braman, Cohen, Gastil & Slovic 2010).

These normal and normally reliable processes of knowledge transmission break down when risk or like facts are transformed (whether through strategic calculation or misadventure and accident) into divisive symbols of cultural identity. The solution to this problem is not—or certainly not necessarily!—to divest citizens of the power to contribute to the formation of public policy. It is to adopt measures that effectively shield decision-relevant science from the influences that generate this reason-disabling state (Kahan et al. 2006).

Just as individual well-being depends on the quality of the natural environment, so the collective welfare of democracy depends on the quality of a science communication environment hospitable to the exercise of the ordinarily reliable reasoning faculties that ordinary citizens use to discern what is collectively known. Identifying strategies for protecting the science communication environment from antagonistic cultural meanings—and for decontaminating it when such protective measures fail—is the most critical contribution that decision science can make to the practice of democratic government.

[1] We would add, however, that we do not believe that the results of this or any other study we know of rule out the existence of cognitive dispositions that do effectively mitigate the tendency to display ideologically motivated reasoning. Research on the existence of such dispositions is ongoing and important (Baron 1995; Lavine, Johnston & Steenbergen, 2012). Existing research, however, suggests that the incidence of any such disposition in the general population is small and is distinct from the forms of critical reasoning disposition—ones associated with constructs such as science literacy, cognitive reflection, and numeracy—that are otherwise indispensable to science comprehension. In addition, we submit that the best current understanding of the study of science communication indicates that the low incidence of this capacity, if it exists, is not the source of persistent conflict over decision-relevant science. Individuals endowed with perfectly ordinary capacities for comprehending science can be expected reliably to use them to identify the best available scientific evidence so long as risks and like policy-relevant facts are shielded from antagonistic cultural meanings.


Motivated Numeracy (new paper)!

Here's a new paper. I'll probably blog about it soon, but if you'd like to comment on it now, please do!



The NRA's "expressive-rope-a-dope-trick"

The NRA gets science communication.

In fact, it understands something that many groups that at least purport to be committed to promoting constructive public engagement with the best available scientific evidence don’t.

Of course, it uses what it understands for a purpose very distinct from promoting such engagement. Indeed, it uses its knowledge about how diverse, ordinary people ordinarily come to know what they know about decision-relevant science in a manner that effectively impedes their convergence on evidence essential to their common welfare.

This makes the NRA a truly evil entity—a kind of syndicalist element subversive of the Constitution of the Liberal Republic of Science

But one can still actually learn something from seeing what it knows and what it does.

The point the NRA gets—and that many other groups that I think have admirable aims don’t, and that makes them tend to do a bad job—is that effective communication of decision-relevant science depends on the quality of the science communication environment.

The science communication environment is the sum total of cues, influences, and process that enable people to recognize as known by science so many more things than they could possibly form a meaningful understanding of for themselves. The number of things that fit into that category is immense—from the contribution that antibiotics make to treating diseases to the validity of modern telecommunications technologies they rely on to transmit data, from the reliability of their vehicle’s GPS systems to the public health benefits of pasteurization of raw milk, from the nontoxicity of pressed wood products manufactured subject to state and federal formaldehyde limits to the nutritional value of food products (massive amounts of them in the US) that are prepared with GM technology.

One of the most vital constituents of the science communication environment is the existence of authoritative networks of certification.

I’m talking, really, just about the role that the utterly ordinary, every-day communities individuals inhabits—the ones that comprise their neighbors, their friends, their trusted coworkers, and myriad professions they rely on, from doctors to auto mechanics to accountants to insurance adjusters.

These communities are flush with reliable, valuable guidance that individuals can use to determine what’s known to science.  Of course, they are also coursing with bogus information too—unsupported and unsupportable claims about the dangers of everyday products (“watch out—cell phone radiation causes brain tumors!”) and absurd claims about health remedies (“ach—don’t do chemotherapy for your breast cancer; yoga will do the trick!”)

People sort out one from the other—again, not because they are experts on the claims that are being made what science knows, but because they are experts at something else: figuring out who actually knows what they are talking about, and can be relied upon to transmit the best available evidence in a reliable and accurate manner.

This is the key to understanding why the transmission of knowledge tends to have a culturally insular quality to it.

The communities of certification people tend to resort to orient themselves appropriately with respect to decision-relevant science are ones made up of people who share basic outlooks on the good life.  People enjoy spending time with people like that and tend to form important projects with them. They can read those people more easily—and distinguish the genuinely knowledgeable from the bullshitters among them more readily—than they can when they are engaging people whose cultural orientation is very different from their own.

We live in a society that tolerates and celebrates cultural diversity (a fact that is actually essential to the progress of scientific discovery), and therefore the number of communities people rely on to perform this certification function is large.

But that’s generally not a problem.  These communities are all in touch with what science knows.  They all generally lead their members to the same conclusions.

Indeed, if there was a community that consistently misled its members on what science knows, the members of that group, given how important decision-relevant science is to their own well-being, wouldn’t last very long.

Nevertheless, every once in a while a risk or other policy-relevant fact becomes engaged in antagonistic cultural meanings that convert positions on it, in effect, into badges of membership in and loyal to opposing cultural groups. 

When that happens, members of diverse cultural groups won’t converge on the best available evidence.  Instead—using the very same normal, and normally reliable cues to ascertain what’s known to science—they will polarize.

The stake that any ordinary person has in protecting the status of, and his or her standing in, one of these groups tends to exceed the significance of the stake that person has, as an individual, in forming scientifically informed personal beliefs. As a result, individuals, in this circumstance, will predictably engage information in a manner more reliably geared to forming beliefs that match the ones the position identified with their group than the ones most supported by the best available scientific evidence.  

Indeed, in these circumstances, individuals endowed with the capacities and dispositions most strongly associated with science comprehension will use these abilities in an opportunistic fashion to serve the goal have to conform the evidence the encounter or actively seek out to the position that is predominant in their cultural group.

These antagonistic meanings can be likened to a form of pollution in the science communication environment.  Their existence disables the faculties that ordinary members of the public use to recognize what science knows. 

That’s what the NRA knows.  That’s the insight into the science of science communication that it ruthlessly exploits—not to promote convergence on the best available evidence but to cultivate a state of persistent, knowledge-disabling antagonism.

The NRA is in the business of science miscommunication.  And its most potent weapon is not the dissemination of studies that purport to show that crime rates go down when people are allowed to carry concealed handguns. 

It’s the steady stream of pollution that it emits into the science communication environment through actions calculated to sustain and invigorate the culturally antagonistic meanings that surround guns in American society.

Really, the NRA is an ingenious science communication environment polluter.

It’s most creative, successful, and insidious technique involves what I will call the “expressive-rope-a-dope” maneuver.

This trick involves proposing a law that in fact has zero behavioral consequence but that is bristling with cultural meanings that one can expect to antagonize another cultural groups.  The effect is achieved, though, not by antagonizing the other group (I suppose the NRA or some other group using this tactic might take pleasure in that) but by provoking the opposing group into denouncing the law in terms that are similarly suffused with culturally assaultive language.

The result of the violent collision of these meanings is a mushroom cloud of toxic, culturally partisan recrimination that blankets the public in the radiation of identity threat.  Whatever science content is being transmitted by anyone’s messages is drowned out but the much clearer, much more intense, much more consequential signal that the positions at stake here are ones that are symbols of membership your group; deviate from that position at your peril!

Consider two examples of the NRA using this trick.

The first involved its campaign to push for adoption of “stand your ground” self-defense laws.  These laws state that a person needn’t retreat before using deadly force to repel a threat of death or great bodily harm.

From the beginning, the enactment of these laws has drawn high profile, incensed denunciations of “wild west,” “shoot first,” “vigilante justice”—along with completely untenable, absurd claims about how this “sharp turn in American law” increased homicide rates.

The simple truth is that these laws were not a departure, radical or otherwise, from existing law. The right to “stand one’s ground” had been the majority rule in the U.S. for over a century, and was already on the books in most of the states that adopted them!

The absurdity of media reports blaming “relaxation” of self-defense standards for increased homicides was comically inflated by the incompetence of publicity-hungry scholars pedaling econometric models purporting to quantify how much “reducing the legal price” for homicide in states that never changed their law increased the “return” on resorting to deadly violence!

The aim of getting states to enact them wasn’t to create a legal safe haven for individuals who forgo a physical one in lieu of blowing away a deadly attacker—a scenario that one is hard-pressed to find instances of except in lawschool hypotheticals.

Rather, as I’ve discussed previously, the effort was a calculated strategy to reactivate the focus of a long dormant, largely sectional conflict between proponents of opposing cultural styles—one stressing values such as individual honor and self-reliance and the other the democratic ideal of reasoned, nonviolent resolution of conflict and the duty of universal concern, on the other—who saw the contest over enactment of these laws as symbolic contest between their competing visions.

Mission accomplished for the NRA, which has parleyed the recurring attacks on “stand your ground laws”—the most recent in connection with the Trayvon Martin case, in which that law played no role in the defense theory—into a sense of indignation and defiant pride on the part of those who recognize in the tone and idiom of the critiques contempt for their identities.   

The second involves legislation now pending in Missouri that would make it a crime for federal agents to enforce federal gun legislation in the state. The NRA is not playing an open role in backing the legislation, but it frequently orchestrates symbolic legislation of this sort behind the scences. Predictably, the law has provoked a ear-splitting clang of alarm bells from NRA critics in the national media warning that the legislation, if passed, will become a model for “nullification” of federal gun laws across the Nation. 

They should save their breath.  Such laws are a dead letter under the Supremacy Clause of the U.S. Constitution.  There is zero likelihood that any state prosecutor would even try to enforce one, much less that a federal court (to which any such prosecution would be subject to “removal” or transfer under federal law) would uphold its constitutionality.

But of course, the contrived panic is music to the NRA’s ears.  It supplies them with even more vivid and dramatic materials with which to feed the sense of cultural encirclement that drive those whose identities are promiscously assaulted by gun-control advocates to donate money to the organization. 

The biggest threat to the NRA isn’t gun legislation. It is apathy.

Gun ownership is the strongest predictor (not surprisingly) of resistance to gun control legislation.  Over time, the percentage of Americans owning guns as declined.

Halting that trend, the NRA recognizes, depends on sustaining the vitality of the cultural meanings that have always made guns so popular with a large segment of the American public.

The surest way to do that is to manufacture dramatic instances of expressive conflict over guns, thereby reinvigorating opposition to gun control as a symbol of cultural identity and bombarding the communities in which that cultural style is prevalent with the signal that having a strong position against regulation of guns continues to be something that those whom they interact in their daily lives will use to judge their character.

But there is in fact a way effectively to oppose this strategy.

The expressive-rope-a-dope maneuver requires a dope—a loud, aggressive, ill-informed opposition that doesn’t get that the laws its attacking are purely expressive, or that the contribution those laws make to maintaining the gun as a symbol of identity depends on attacking them in culturally assaultive way.

Don't do that. Don't take the bait. Don't give the NRA what it wants by pretending symbolic gestures have real and dire consequences and then making opposition to them the occassion for amplifying the signal of cultural hostility that fills otherwise ordinary citizens with resentment and fury.

There’s no meaningful political theater if only half the cast shows up.

Indeed, this is something that lots of groups that are committed to promoting constructive engagement with decision-relevant science could benefit from learning.  The NRA isn't the only group that knows how to rope dopes.

This assumes, of course, that the groups getting roped really want to protect the quality of the science communication environment from culturally partisan meanings.

Some of them likely value the chance to engage risk issues in a manner that fills the science communication environment with culturally partisan meanings.

If so, then they aren't being dopes when they snap at the bait and make their own contribution to the toxic fog of cultural recrimination surrounding the American gun question or other issues that feature persistent polarization over decision-relevant science.

In that case, they are being tapeworms of cognitive illiberalism, just like the NRA.



Science and the craft norms of science journalism, Part 2: Making craft norms evidence based

This is the second in a series that will be between 3 and 14,321 posts on the connection between science and the craft norms of science journalism.

The point of the series, actually, is that there isn’t—ironically—the sort of connection there should be.

I myself revere science journalists. To me, they perform a kind of magic, making it possible for me, as someone of ordinary science intelligence to catch a glimpse of, and be filled with the genuine wonder and awe inspired by, seeing what we have come to know about the workings of the universe by use of science.

This isn’t really magic, of course, because there’s no such thing as magic, and it would insult anyone who accepts science’s way of knowing as the best—the only valid—way of knowing to say that what he or she is doing amounts to “magic” if the person saying this weren’t being ironic or whimsical (I could imagine describing something as “magic” in a tone of rebuke or contempt: e.g., “Freudian psychoanalysis is a form of magic.”).

But what science journalists do is amazing and hard to fathom. They perform an astonishing task of translation, achieving a practical, workable commensurability between the system of rational apprehension that ordinary people use to make sense of the phenomena that they must recognize and handle appropriately in the domain of everyday life and the system of rational apprehension that scientists in a particular field must use to make sense of the phenomena in their professional domain.

Both systems are stocked with prototypes finely turned to enable the sort of recognition that negotiating the respective domains requires. 

But those prototypes are vastly different; or in any case, the ones the experts use are absent and very distinct from anything that exists in the inventory of patterns and templates of the ordinary, intelligent person. 

These special-purpose expert prototypes (acquired through training and professionalization and experience) are what allow the expert to see reliably what others in his or her field see, and thus to participate in the sharing and advancement of knowledge in that expert domain.

But enabling the ordinary nonexpert to see the things that science comes to know as experts use their specialized professional judgment is the whole point of science journalism!  

Necessarily science journalists must find some means of bridging the gap between the prototypes of the expert scientist and the everyday ones of the curious nonexpert so that the latter can form a meaningful apprehension of the amazing, and awe-inspiring insights that the former glean through science's methods of knowing.

This isn’t magic, in fact.

It is craft. Of the most impressive and admirable sort. 

It comprises norms that reliably populate the mind of the science journalist with prototypes and patterns of communication practices that achieve the amazing commensurability I’m talking about.

Science journalists generate these craft norms through their collective activity, and acquire them through experience.

But they aren’t static.  They evolve.

Moreover, they aren’t invisible.  They are matters that science journalists, like any other professionals, become keenly and acutely aware of as they do their jobs, and do them in concert with others with whom they discuss, and from whom they learn, their craft.

And like other professionals, science journalists are keenly interested in whether their craft norms are in order

In the account I’m giving, craft norms are the medium by which professional judgment is formed and through which it operates.

Like a method of scientific measurement, professional judgments need to be reliable: they must enable consistent, replicable, shared apprehension of the phenomena that are of consequence to members of the profession.

But like methods of scientific measurement they must also be valid.  The thing they are enabling those who possess them reliably, collectively, to apprehend and form judgments about must genuinely be the thing that those in the profession are trying to see.

In the case of the science journalist, that thing that must be seen—not just reliably but accurately—is how to make it possible for the nonexpert of ordinary science intelligence to form the most meaningful, authentic, true picture of the awesome things that are genuinely known to science.

Science journalists, like other professionals, are constantly arguing about whether their norms are valid in this sense. "Are we really doing what we want to do as best we can?," they ask themselves.

Actually, there is no sense of crisis in the profession (as far as I can tell). They know full well that in the main their craft norms are reliably guiding them to ways of communicating that actually work.

But there are plenty of particular matters—ones of genuine consequence—that they worry about, that they have different opinions on, that relate to whether particular things they are doing might actually be working less well than some alternative or maybe even frustrating their goals.

The last post touched on one of those things: In it I discussed Andrew Gelman’s critique of the passivity of science journalists in reporting on “WTF!” social science studies—ones that report remarkable, astonishing, unbelievable results that, in Gelman’s view, almost inevitably are shown to rest on a very basic methodological defect.

It’s not as if science journalists aren’t aware of that issue & filled with views about it!

What’s more, Gelman proposed a solution: interview lots of additional experts besides the study authors and find out if they think the study is valid.

Actually, science journalists talk about this too!  The issue isn’t just whether this is a feasible idea but whether it is actually a sound one given what the aim of science journalism is trying to do.

Gelman didn’t recognize that his prescription is bound up with the controversy over whether “balanced coverage”—a norm that enjoins science journalists to cover “both sides” and evince a posture of “neutrality” toward disputed scientific claims—actually contravenes the objective of helping the public form an accurate perception of what’s known by science, particularly on controversial issues like, say, evolution or climate change.

Which gets to another thing that I think was missing, not just from Gelman’s (excellent!) essay but from the discussion that science journalists, as a professional community, are constantly having.

The matters they are debating when they reflect on the validity of their craft norms are very often empirical ones.

The admit of empirical investigation.  Indeed, they demand it: members of a profession are no more able to determine through simple debate which of multiple plausible accounts of a phenomenon is true than are scientists

Scientists don’t just debate in that situation. They collect empirical evidence!

That’s what science journalists need to do too. 

They need to make their profession evidence-based—the need to create procedures for identifying craft-norm issues that admit of empirical testing and mechanisms institutions for collecting that evidence, transmitting, and reflecting in common on what that evidence reveals.

Not as a substitute for their craft-norm informed professional judgment—but as a self-consciously managed source of knowledge that they can use as they do what they participate in the process by which their craft norms are formed, evolve and are transmitted.

The need for an evidence-based culture in science journalism is one of the things I had in mind when I said that the points of connection between science journalism and science itself need to be strengthened.

In fact, it is the most important.  But there are other points worth mentioning—ones that it will be easier to explain now that this point is out there.

So I will say more. Later.                               

But the one last thing I will say is that science journalism is not the only profession that is committed to the transmission of scientific knowledge that, to its disadvantage, fails to use science’s way of knowing to advance its knowledge of how to transmit what science knows.

Indeed, science journalists are in a position to do a tremendous favor for those other professions by showing them how to remedy this problem.

Some might think, after decades of aggressive inattention to the science of science communication by those responsible for transmitting decision-relevant science in our democracy, that nothing short of magic will ever remedy our democracy’s deficit in science communication intelligence.

If so, then science journalists are the ones we need to show us how to pull this trick off.