follow CCP

Recent blog entries
Wednesday
Nov072012

Hey, the problem *isn't* that people are irrational, proof #6276: Prop 37 fails in Calif

From Andy Revkin on dotearth:

California’s Proposition 37, or #Prop37 as it was known on Twitter, failed last night by a substantial margin — 53 percent to 47 percent. The ballot initiative would have required labeling for some genetically engineered foods. (Click here for an illuminating interactive county-by-county map of the vote. Upscale urban and coastal regions wanted it; inland areas mostly rejected it.)

As I said on Tumblr this morning, I’m glad that the sloppyunscientific andprotectionist initiative failed, but glad an important discussion of transparency in food sourcing has begun....

There’s more on Dot Earth on relevant issues....

Tuesday
Nov062012

It's journal club time (episode 391)! Lewandowsky et al. on scientific consensus

Thanks to the many friends who sent me emails, made late night phone calls, or showed up at my front door (during the time when the storm had knocked out internet & phone service) to make sure I saw Lewandowsky, Gignac, & Vaughan's The pivotal role of perceived scientific consensus in acceptance of science in Nature Climate Change. It's a really cool paper!

LGV present observational and experimental evidence relating to public perceptions of scientific consensus on climate change and other issues. CCP did a study on scientific consensus a couple yrs ago,  -- Kahan, D.M., Jenkins-Smith, H. & Braman, D. Cultural Cognition of Scientific Consensus. J. Risk Res. 14, 147-174 (2011)--which is one of the reasons my friends wanted to be sure I saw this one.  

The paper presents two basic findings. I'll say something about each one.

Finding 1: Perceptions of scientific consensus determine public beliefs about climate change--and in essentially the same way that they determine it on other risk issues.

In the observational study, the respondents (200 individuals who were solicted to participate in person in downtown Perth, Australia) indicated their beliefs about (a) the link between human CO2 emissions and climate change (anthropogenic global warming or "AGW"), (b) the link between the HIV virus and AIDS, and (c) the link between smoking and lung cancer.  The respondents also estimated the degree to which scientists believed in such links. LGV then fit a structural equation model to the data and found that a single "latent" factor -- perception of scientific consensus with respect to the link in question -- explained the respondents' beliefs, and "fit" the data better than models that posited independent relationships between respondents' beliefs and their perceptions of scientific consensus on these matters. So basically, people believe what they think experts believe about all these risks.

Surprised? "Of course not. That's obvious!"

Shame on you, if that is how you reacted. It would have been just as "obvious!" I think, if they had found that perceptions of scientific consensus didn't explain variance in perceptions of beliefs in AGW, or that such perceptions bear a relationship to AGW distinct from the ones on other risks. That's because lots of people believe that skepticism about climate change is associated with unwillingness to trust or believe scientists. If that were true, then then the difference between skeptics and believers wouldn't be explained by what they think scientific consensus is; it would be explained by their willing to defer to that consensus.

Most social science consists in deciding between competing plausible conjectures. In the case of climate change conflict, two plausible conjectures are (1) that people are divided on the authority of science and (2) that people agree on the authority of science but disagree about what science is saying on climate change. LGV furnish more evidence more supportive of (2) than (1). (BTW, if you are curious about how divided Australians are on climate change, check this out.)

from Kahan, Jenkins-Smith & Braman (2011)In that regard, moreover, their finding is exactly in line with the CCP one. Using a large (N = 1500) nationally representative sample of US adults, we measured perceptions of scientific consensus on climate change, nuclear power risks, and gun control.  These are highly contentious issues, on which American citizens are culturally divided. Nevertheless, we found that no cultural group perceives that the view that is predominant among its own members is contrary to scientific consensus. (We also found that all the groups were as likely to be mistaken as correct about scientific consensus across the run of issues, at least if we treated the "expert national consensus reports" of the National Academy of Sciences as the the authority on what that consensus is.)

So next time you hear someone saying "climate skeptics are anti-science," "the climate change controversy reflects the diminishing authority of/trust in scientists" etc., say "oh, really? What's your evidence for that? And how does it relate to the LGV and CCP studies?"

Finding no. 2: When advised that there is overwhelming scientific consensus in favor of AGW, people are more likely to believe in AGW -- and this goes for "individualists," just like everyone else.

The experiment subjects (100 individuals also solicited to participate in person in Perth, Australia) indicated their AGW beliefs after being randomly assigned to one of two conditions: a "consensus information" group, which was advised by the experimenters that there is overwhelming scientific consensus (97%) on AGW; and a "no information" group, which was not supplied any information on the state of scientific opinion.  

LGV found, first, that subjects in the consensus-information group were more likely to express belief in AGW. This result adds even more weight to the surmise that popular division over climate change rests not on a division over the authority or credibility of scientists but on a division over perceptions of scientific consensus.

from Lewandowsky, Gignac & Vaugh (2012)Second, LGV found that the impact of consensus-information exposure had a stronger effect on subjects as their scores on a "free-market individualism" worldview measure increased. In other words, relative to their counterparts in the no-information condition, subjects who scored high in "individualism" were particularly likely to form a stronger belief in AGW when exposed to scientific-consensus information.

Although also perfectly plausible, this finding should definitely raise informed eyebrows.

Public opinion on climate change in  Australia, as in the US, is culturally divided.  Consistent with other studies, LGV found that individualism generally predicted skepticism about AGW.

We know (in the sense of "believe provisionally, based on the best available evidence and subject to any valid contrary evidence that might in the future be adduced"; that's all one can ever mean by "know" if one actually gets the logic of scientific discovery) that individualist skepticism toward AGW is not based on skepticism toward the authority of science. Both the observational component of the LGV study and the earlier CCP study support the view that individualists are skeptical because they aren't convinced that there is a scientific consensus on AGW.

Well, why? What explains cultural division over perceptions of scientific consensus?

One conjecture -- let's call it "cultural information skew" or the CIS -- would be that individualists and communitarians (i.e., non-individualists) are exposed to different sources of information, and the information the former receives represents scientific consensus to be lower than does the information the latter receives.

But another conjecture -- call it "culturally biased assimilation" or CBA -- would be that individualists and communitarians are culturally predisposed to credit evidence of scientific consensus selectively in patterns that fit their predisposition to form and maintain beliefs consistent with the ones that prevail within their cultural groups. CBA doesn't imply that individualists and communitarians are necessarily getting the same information. But it would predict disagreement on what consensus is even when people with those predispositoins are supplied with the same evidence.

CBA is one of the mechanisms comprised by cultural cogniton.

from Kahan, Jenkins-Smith, Braman (2011)The same CCP study on scientific consensus furnished experimental evidence supportive of CBA. When subjects were asked to assess whether a scientist (one with elite credentials) was an "expert"-- one whose views should be afforded weight -- subjects tended to say "yes" or "no" depending on whether the featured scientist was depicted as espousing the position consistent with or opposed to the one that predominated among people who shared the subjects' values. 

In other words, subjects recognized the positions of elite scientists as evidence of what "experts" believe selectively, in patterns that fit their cultural predispositions on the risk issues (climate change, nuclear power, and gun control) in question. If this is how people outside the lab treat evidence of what "expert consensus" is, they can be expected to end up culturally divided even when they are exposed to the very same evidence.

At least one more research team has made a comparable finding. Adam Corner, Lorraine Whitmarsh, &  Dimitrios Xenias published an excellent paper a few months ago in Climatic Change that showed that subjects displayed biased assimilation with respect to claims made in newspaper editorials, crediting or discrediting them depending on whether the claims they made were consistent with what the subjects already believed about AGW. That's not culturally biased assimilation necessarily but the upshot is the same:  one can't expect to generate public consensus simply by bombarding people with "more information" on scientific consensus.

The LGV finding, though, appears inconsistent with biased assimilation, cultural or otherwise. The subjects in the consensus-information group were being supplied with evidence -- in the form of information provided by experimenters -- that suggested scientific consensus on AGW is very high (higher, apparently, than even subjects who believe in AGW tend to think).

The CBA prediction would be that more individualistic subjects would simply dismiss such evidence as non-credible -- in the same way that subjects in the CCP study rejected the credibility of scientists who furnished them with information contrary to their cultural predispositions. Having been given no credible information in support of revising their assessment of scientific consensus, LGV's individualist subjects would not (under the CBA view) be expected to revise their assessments of AGW.

But apparently they did! That's a result more in keeping with the "information skew" (CIS) account of why individualists disagree with communitarians.  So it turns out after all that all we need to do is un-skew things. As LGV put it, their study "underscores the vital role of highlighting a scientific consensus when communicating scientific facts," particularly when the underlying issues are "difficult to grasp or are hotly debated or challenge people’s world views."

So do I "accept" LGV as evidence against CBA, and as evidence for being less skeptical about a communication strategy that focuses on simply "highlighting scientific consensus"?  For sure!

But I don't see the evidence as super strong -- and certainly not strong enough to change my mind on these matters given the sum total of the evidence, including but not limited to the previous CCP & Corner et al. studies. In Bayesian terms, I give LGV a likelihood ratio of 0.77 in favor of CBA (or 1.3 in favor of the alternative, CIS hypothesis).

The reason I am not inclined to assign more decisive weight to the LGV finding is that I'm not convinced that people in the real world will be nearly so willing to accept real-world information on scientific consensus as the LGV study apparently were to accept the LGV experimenters' representations.

If individualists in the real world were that receptive to information "highlighting" scientific consensus, I'm very confident they would have gotten the message by now. You really have to be off the grid -- off the planet, even -- not to have heard over & over & over that there is "overwhelming scientific consensus" on AGW. One either accepts that information when it is presented -- on tv, in newspapers, by people one talks to on the street corner -- or one just doesn't. And obviously a good segment of the population just doesn't.

Basically, I'm taking the fact that "some people credit reports of scientific consensus on AGW yet many don't" as the starting point for investigation, and trying to figure out who sees what & why. Again, the CCP experimental result is evidence, in my view, that people are motivated to selectively credit or dismiss evidence of scientific consensus in ways that fit their cultural prepositions (CBA). 

Now in fact, I am surprised that individualistic subjects in the LGV study apparently did put so much confidence in the word of the experimenters. But that they did makes me question whether the situation those subjects were in is really comparable to one of people who are engaging real-world information sources.

I'm inclined to say that in this regard I think the CCP experiment was more realistic. We -- the experimenters -- made no representations to our subjects about the state of scientific consensus. Rather, we showed them some evidence -- a scientist taking a position -- and let them decide for themselves what weight to attach to it. They told us that they viewed what we were showing them as valid evidence of "what experts believe" only when that evidence was consistent with the position that predominated in their group.  

I think that's closer to the situation that we can anticipate people will be in outside the lab when real-world people -- from journalists to advocates to individual scientists to their fellow citizens -- try to "highlight" AGW consensus to them. The expectation that people in that setting will be dismissive toward representations that challenge their predispositions is strongly supported by Corner, Whitmarsh, & Xenias (2012) as well.

Actually, LGV come pretty darn close to saying they agree with this point. They write:

At first glance, our results challenge the results of Kahan and colleagues, that perceived consensus operates like any other fact that is equally subject to dismissal as other evidence surrounding AGW. However, on closer inspection, the study by Kahan did not provide socially-normative information about a consensus (that is, ‘97 out of 100’) but instead presented participants with an informational vignette, attributed to a fictional expert, that either described the risk from climate change or downplayed it. Because this manipulation provided anecdotal rather than social-norming information, it is not surprising that participants rated the source as less trustworthy if the message was worldview dissonant. Normative information, by contrast, is widely assumed to be more resilient to ideologically-motivated dismissal ....

Right: if one provides information that people view as "socially normative" -- i.e., as worthy of being believed --  they'll accept it. But the issue is how to make people view that information as "socially normative" when it is contrary to their cultural predispositions? I just find it implausible to believe that people in the world are as open to real-world evidence (including media accounts & the like) purporting to tell them that they & all their peers are wrong about scientific consensus on AGW as the subjects in the LGV experiment apparently were when the experimenters told them  "97 out of 100 climate scientists believe in AGW." 

My skepticism, however, is not a reason for anyone, including me, to dismiss the significance of LGV's experimental finding.  Only a person who doesn't really understand how empirical study enlarges knowledge would think that one can find a study compelling, insightful, and challenging only if one is "convinced" by the conclusion.

Indeed, if you get how empirical inquiry works, then you'll know how I or LGV or anyone else should respond to the questions I've raised: not by putting this paper aside, but by getting a firm grip on it & trying to reciprocate its contribution to knowledge by doing additional studies that take aim at exactly what is giving me pause here.

E.g., if one embedded the statement "97 out of 100 scientists accept AGW" in a NY Times newspaper story, would individualists react the same way as the ones in this study did? Would they be just as likely to believe that representation as they would be to accept the representation that "only 3" -- or more plausibly for experimental purposes, "only 43" or "only 47"--"of 100" scientists believe in AGW? Would egalitarian communitarian subjects likewise credit just as readily either representation on the state of consensus on AGW? Same for safety of nuclear power?

Show me that -- a result that essentially replicates LGV in a in the Corner, Whitmarsh, & Xenias (2012) design -- & I'll definitely be revising my priors on CBA by a humongous amount!

But I won't have to wait for that result (or the opposite of it) to get the benefits of both knowing more and having more to puzzle over as a result of this paper.

I think it's cool! Read it & tell me what you think!

References

Corner, A., Whitmarsh, L. & Xenias, D. Uncertainty, scepticism and attitudes towards climate change: biased assimilation and attitude polarisation. Climatic Change (2012), on-line advance publication at http://dx.doi.org/10.1007/s10584-012-0424-6

Kahan, D.M., Jenkins-Smith, H. & Braman, D. Cultural Cognition of Scientific Consensus. J. Risk Res. 14, 147-174 (2011)

Lewandowsky, S., Gignac, G.E. & Vaughan, S. The pivotal role of perceived scientific consensus in acceptance of science. Nature Climate Change (2012).

Monday
Nov052012

2 tropes in Proposition 37 debate

As election day approaches, citizens around the Nation are excitedly debating the most consequential of the issues to be resolved on Tuesday: whether Californians should vote "yes" or "no" on Proposition 37, which would require that GM foodstuffs bear a label that states "Genetically Engineered'" or "Produced with Genetic Engineering" in "clear and conspicuous words ... on the front of the package."

I've already explained, in 3 previous posts (1, 2, 3) the general reasons why I don't like Proposition 37.

In fact, I'm certainly open to counter-arguments and have been following the back-and-forths to see if I catch sight of anything in the debate that gives me reason to reconsider.

So far, I haven't.

The main thing I've seen from the proponents are two recurring tropes -- argument bits, essentially. They seem to be effective debating points -- or at least they draw many appreciative nods and cheers from those who already accept the "yes" position -- but they aren't helping me at all to see what I might be missing.  

I'm going to explain why I find the tropes unhelpful, not because I want to change anyone's mind but because I do want those who might want to change mine to see why these points just aren't responsive to my concerns.

Trope 1: "All proposition 37 does is furnish information. What could possibly be wrong with that?"

My reaction:  

a. The communicative impact of the label isn't necessarily confined to the words on it ("GENETICALLY ENGINEERED," on "the front of the package" etc); it includes "there's a reason for you to worry about this -- or else we wouldn't bother to tell you, would we?..."  People process information that way, & it makes perfectly good sense for them to do so. So if in fact there's not something for them to worry about, then labels like this either risk steering them away from things that aren't dangerous or diluting the significance they give to warnings. There's ample literature on both effects--and on how complicated it is to design labels that inform rather than misinform consumers. As a busy person who makes sense of information in the same way as everyone else, I prefer a more considered & systematic approach to how my "warning environment" is populated. 

b. Even more important, the labeling referendum is a communicative focal point for messages that are radiating with cultural "us vs. them/whose side are you on" meanings of the sort that make people see (literally, see; cultural cognition works that way) risks in a way that divides them into warring tribes. The proponents of Proposition 37 are already making unfounded claims about the science on GM food risks, and I worry that passage of this provision will be used strategically and rhetorically as part of a continuing campaign to create a fog (smog, even) of motivated reasoning that interferes with the ability of diverse groups to recognize and converge on the best available evidence as it accumulates.

Stigmatizing a technology can degrade the quality of science-communication environment, making it harder for people to communicate constructively w/ one another & figure out what to do. That's happened in the US with various technologies, including nuclear power. It has happened specifically w/ GM foods in Europe. I'm worried about that here.

c. I accept, too, that some of my fellow citizens are simply interested in knowing whether GMOs are in food, either because they are worried about as-yet undetected health risks or because philosophically/morally they think there is something untoward about genetic engineering.  But they can get that information without making me and others bear the empty-alarm clutter of a state-mandated advisory label, since non-GMO producers are free to put a "GMO-free label" on their products.

Trope 2: "The issue is democracy & scientifically informed decisionmaking!"

My reaction:

Sigh...

Like you, I'm for democracy. Like you, I'm for informed decisionmaking, individual & collective.

That's why I'm worried about Proposition 37.  

The capacity of a democracy to make enlightened decisions turns on the quality of its science communcation environment. The issue here is whether Proposition 37 is a kind of pollution of that environment. I think it is.

The scientific jury is always out on risk -- which is to say, we must always and forever continue to collect evidence and be open to the best scientifically available information on the hazards we face and how to abate them, both with respect to new technologies and existing ones.

But as we can see from the contentious, unconstructive, unenglightened and unenlightening disputes today over climate change, it is a huge huge mistake to take for granted the conditions that assure we'll be able to recognize what the scientific jury is saying as it makes its reports. My concern is that Proposition 37 is part and parcel of a style of political advocacy that destroys those very conditions. 

Again, I might be wrong here, and I'd be interested in figuring out why plenty of reflective people disagree with me.  

But when those who support Prop 37 intone "democracy ... freedom ... right to know ... information!" -- not to mention "profit mongers vs. the people!" etc. (the campaign for Propsition 37 is  funded by industries interested in making profits, too; that's just the way it goes) -- the only thing I learn is that they either aren't getting or don't care what worries reflective people on the other side.

Thursday
Nov012012

A "teachable moment" for science communication: Mayor Bloomberg shows how it's done

Our climate is changing. And while the increase in extreme weather we have experienced in New York City and around the world may or may not be the result of it, the risk that it may be — given the devastation it is wreaking — should be enough to compel all elected leaders to take immediate action.

Reported in latest dotearth post on the foreseeably polarizing "Sandy-causation-teaching-moment" meme.

That said, I do wish Bloomberg would stop trying to make people drink small sodas & breast feed their infants!

BTW, by calling Bloomberg's statement a "teachable moment" for science communication, I recognize that I risk insulting the many  many many people who have been urging that Sandy be seized as a "teachable moment" for those communicating climate science to the public. The problem with this phrase is that that it conveys a certain attitude; it comes off sounding as if one views those who need to be "taught" something as dimwitted school children. I'd recommend a different "strategy" -- like, say, treating (even truly regarding) the people to whom one is purporting to communicate science as thinking citizens who are entitled to get information in a form and under conditions that enable them to use their reason.

I promise not to use this obnoxious idiom anymore if you do. Deal?

Wednesday
Oct312012

"Climate change caused ...": linguistics, empirics, & reasoned discourse

Here are some reflections occasioned by (1) Andy Revkin's excellent dotearth blog post on the relationship between climate change and Sandy; (2) the anger that Revkin's post aroused among at least some climate-change-policy advocates, who proposed the laughable but still disturbing idea that Revkin be publicly censured in some way (others, it should be noted, responded in a critical but reasoned way to issues about advocacy and science information that are admittedly complex); and (3) a columin in the Huffington Post by George Lakoff, who has figured out that the problem here is confusion over language, which if used properly resolves important practical, empirical issues without the need to consult evidence (including evidence of how the public engages with climate science)...

There are 3 issues here: (1) one relating to whether "climate changed caused x" in general; (2) another to whether it caused Sandy in particular; and (3) a final one to the polemics of "caused by climate change..."

1. General.  Hansen et al. are much more helpful than Lakoff on this issue. When we ask the question, "did climate change cause x," the issue is not "semantic" but practical & empirical: we want to understand what the physical relationship is between climate change & particular events &  what effect to assign to particular events in trying to asses the impact of climate change.  The issue isn't what word or phrase to use; it's what to do. Hansen et al. tell us *exactly* what we need to know: climate change shifts the normal distribution of weather events, making certain outcomes more likely & hence more frequent than they otherwise would have been. Accordingly, we can say that climate change made a particular event "y times more likely" than it would have been, if that's useful (Hansen et al. identify events for which y is very very very high!); but more importantly we can speak instructively to the question of what the impact of climate change is in practical terms-- "more events like this per yr/decade, which will cost $z billion, kill q x 10^3 people  etc".  

It's ironic that Lakoff refers to black-lung & cigarettes. The law got over being confused on causation for mass torts when it stopped trying to wring practical guidance from vague, impressionistic, & ultimately question-begging/-obscuring concepts like "direct vs. indirect," & w/ the help of toxicologists, epidemiologists, economists & others started to think in practical, empirical terms akin to those being proposed by Hansen et al. At that point, the law adopted doctrines that made companies that manufacture products that increase the incidence of some harm pay the price of that extra harm w/o getting tied in conceptual knots about whether this particular actor "caused" that particular injury.

 2. Sandy. I myself am still not sure whether climate change increased the likelihood of a weather event like Sandy. I would have thought the answer was clearly yes. But Revkin's excellent dotearth post showed me that there is at least a division of opinion on this among experts; whether Sandy belongs to the class of events the likelihood of which was increased to a very high degree by climate change is not as cut & dried, apparently, as whether climate change increased the likelihood of, say, persistence of summer heat waves, or wild fires in western US.

But I might be misunderstanding & I'm sure there is more to day. I'd like to hear it. But I won't if people agree to talk the way Lakoff does: whatever one thinks of his "systemic causation" linguistically, the way he uses it ("Global warming systemically caused the huge and ferocious Hurricane Sandy...") begs the practical/empirical question.

3. Polemics of "causation." The furor over Revkin's column & over the use of the word "cause" in general is bound up with strategic political & moral issues. Many of those who want to create public engagement with climate change believe that it is essential to be able to say "climate change caused x" extreme event. I can see why; "causation" implies responsibility, and we are  motivated (very appropriately) to regulate and otherwise hold accountable those responsible for harm. But is it really clear that one can't get the responsibility/accountability point across by saying "climate change makes x event times more likely"-- particularly where y is astronomically high? Or by saying (if we have evidence for saying it) that "climate change means we can expect to see an x event -- one that kills q x 10^3 people & costs $billion -- every 2/5/10 yrs" etc? I'm not sure; be interesting to try to test these things. Lakoff criticizes Hansen's communication skills, but doesn't himself present any evidence to support his assertion that his own proposed way of using terms would promote public comprehension or engagement. I, at least, can think of some pretty plausible counter-hypotheses.

One thing I am sure about, though: those who are insisting that science journalists or others use the term "causation" in a way that avoids even asking the practical, empirical question & getting an answer to it (not to mention those advocates who have proposed attacking Revkin as a way to create a "teaching moment" for journalists & other reflective participants in public discussion) believe in a form of democratic deliberation that involves a smaller role for appeals to citizens' reason than I am comfortable with. 


 
Monday
Oct292012

The science communication problem: one good explanation, four not so good ones, and a fitting solution

I was on a panel Saturday on “public policy and science” at the CSICon conference in Nashville. My friend Chris Mooney was on it, too. I didn’t speak from a text, but this is pretty close to what I rember saying; slides here.

I’m going to discuss the “science communication problem” – the failure of sound, widely disseminated science to settle public controversies over risks and other policy-relevant facts that admit of scientific investigation.

What makes this problem perplexing isn’t that we have no sensible explanation it. Rather it’s that we have too many.

There are always more plausible accounts of social phenomena than are actually true.  Empirical observation and meansurement are necessary--not just to enlarge collective knowledge but also to steer people away from deadends as they search for effective solutions to the society’s problems.

In this evidence-based spirit, I’ll identify what I regard as one good explanation for the science communication problem and four plausible but not so good ones. Then I’ll identify a “fitting solution”—that is, a solution that fits the evidence that makes the good explanation better than the others.

One good explanation: identity-protective cognition

Identity-protective cognition (a species of motivated reasoning) reflects the tendency of individuals to form perceptions of fact that promote their connection to, and standing in, important groups.

There are lots of instances of this. Consider sports fans who genuinely see contentious officiating calls as correct or incorrect depending on whether those calls go for or against their favorite team.

The cultural cognition thesis posits that many contested issues of risk—from climate change to nuclear power, from gun control to the HPV vaccine—involve this same dynamic. The “teams,” in this setting, are the groups that subscribe to one or another of the cultural worldviews associated with “hierarchy-egalitarianism” and “individualism-communitarianism.”

CCP has performed many studies to test this hypothesis. In one, we examined perceptions of scientific consensus. Like fans who see the disputed calls of a referree as correct depending on whether they favor their team or its opponent, the subjects in our study perceived scientists as credible experts depending on whether the scientists’conclusions supported the position favored by members of the subjects’ cultural group or the one favored by the members of a rival one on climate change, nuclear power, and gun control.

Not very good explanation # 1: Science denialism

“Science denialism” posits that we see disputes over risks in the US because there is a significant portion of the populatin that doesn’t accept that the authority of science as a giude for policymaking.

The same study of the cultural cognition of scientific consenesus suggests that this isn’t so. No cultural group favors policies that diverge from scientific consensus on climate change, nuclear power, or gun control. But as a result of idenity-protective cognitoin, they are culturally polarized over what the scientific consensus is on those issues.

Moreover, no group is any better at discerning what scientific consensus is than any other. Ones that seem to have it right, e.g., on climate change are the most likely to get it wrong on deep geologic isolation of nuclear wastes, and vice versa.

Not very good explanation #2: Misinformation

I certainly don’t dispute that there’s a lot of misinformation out there. But I do question whether it’s causing public controversy over policy-relevant science. Indeed, causation likely runs the other way.

Again, consider our scientific consensus study. If the sort of “biased sampling” we observed in our subjects is typical of the way people outside the lab assess evidence on culturally contested issues, there won’t be any need to mislead them: they’ll systematiclly misinform themselves on the state of scientific opinion.

Still, we can be sure they’ll very much appreciate the efforts of anyone who is willing to help them out. Thus, their motivation to find evidence supportive of erroneous but culturally congenial beliefs will spawn a cadre of misinformers, who will garner esteem and profit rather than ridicule for misrepresenting what’s known to science.

The “misinformation thesis” has got things upsidedown.

Not very good explanation #3: “Bounded rationality”

Some people blame controversy over policy-relevant science on deficits in the public’s reasoning capacities.  Ordinary members of the public, on this view, know too little science and can’t understand it anyway because they use error-prone, heuristic stratetgies for interpsteing risk information.

Plausible, sure. But wrong, it turns out, as an explantion for the science communication problem: higher levels of science literacy and quantiative reasoning ability, a CCP study found, don’t quiet cultural polarization on issues like climate change and nuclear power; they magnify it.

Makes sense given identity-protective cognition. People who are motivated to form perceptions that fit their cultural identities can be expected to use their greater knowledge and technical reasoning facility to help accomplish that—even if generates erroneous beliefs about societal risks.

Not very good explanation #4: Authoritarian personality

The original authoritarian-personality of Adorno and his colleagues is often dismissed as an exercise in polemics disguised as social science.

But in recent years, a serious body of scholarship has emerged on correlations between dogmatism, closed-mindedness, and like personality traits, on the one hand, and conservative ideology, on the other. This work is insigthfully synthesized in Mooney’s The Republic Brain.

Does this revitalized “authoritarian personality” position explain public controversy over policy-relevant science?

It’s odd to think it does, given the role that identity-protective cognition plays in such controversies. Identity-protective cognition affects all types of perception (not just evaluations of evidence but brute sense impressions) relating to all manner of group affinities (not just politics but college sports-team allegiances). So why would the impact of identity-protective cognition be linked to a personality trait found in political conservatives?

But the point is, we should just test things – with valid study designs. Is the score on an “open mindendess” test a valid predictor of the sort of identity-protective reasoning that generates disputes over climate change, the HPV vaccine, nuclear power, guns?

I did a study recently designed to answer to this question. I examined whether liberal Democrats and conservative Republicans would displayed identity-protective cogntion in assessing evidence of the validity of the Cognitive Reflection Test (CRT)—which is in fact a valid measure of reflective, open-minded engagement with information.

They both did, and to the same degree.  When told that climate-skeptics got a higher CRT score (and here were presumably more open-minded), liberal Democrats were much less likely to view the test as valid than when they were told that climate-believes got a higher score (indicating they were more open-minded). The mirror-image pattern emerged for conservative Republicans.

What’s more, this effect was magnified by the disposition measured by CRT. That is, the subjects most inclined to employ conscious, reflective reasoning were the most prone to identity-protective cogniton—a result consistent with our findings in the Nature Climate Change study.

The  new “authoritarian personality” work might be identifying real differences between liberals and conservatives. But there’s little reason to think that what it’s telling us about them has any connection to identity-protective cognition—the dynamic that has been shown with direct evidence to play a significant role in the science communication problem.

A fitting solution: The separation of meaning and fact

Identity-protetive cognition is the problem. It affects liberals and conservatives, interferes with the judgment of even the most scientifically literate and reflective citizens, and feeds off even sound information as it creates an appetite for bad.

We need a solution, then, fitted to counteracting it. The one I propose is the formation of "science communication environment" protection capacity in our society.

Policy-consequential facts don’t inevitably become the source of cultural conflict. Indeed, they do only in the rare cases where they become suffosed with highly charged and antagonistic cultural meanings.

These meanings are a kind of pollution in the science communication environment, one that interferes with the usually reliable faculty ordinary people employ to figure out who knows that about what.

The sources of such pollution are myriad. Strategic behavior is one. But simple miscalculation and misadventure also play a huge role.

The well-being of a democratic society requires protecting the science communication environment from toxic meanings. We thus need to use our knowledege to understanding how such meanings are formed. And we need to devote our political resolve to developing procedures and norms that counteract the forms of behavior—intentional and inadvertent—that generate this form of pollution.

A wall of separation between cultural meaning and scientific fact is integral to the constitution of the Liberal Republic of Science.

Tuesday
Oct232012

WSMD? JA! Episode 2: cultural polarization on death penalty & climate change, 2006 vs. 2012

This is the second episode of CCP's already insanely popular new feature, "Wanna see more data? Just ask!" -- or "WSMD? JA!" (For contest rules and conditions, see here.)

In this episode, we answer the question, posed by students in Jeff Fagan's Capital Punishment seminar at Columbia Law School, "have you applied the cultural cognition scales to capital punishment?"

The answer is ... "why, yes -- in a survey just last month! And in another back in December 2006."

It's pretty interesting to compare the two sets of survey results, both on the issue of capital punishment and on the issue of global warming.

The items were the same in both the 2006 & 2012 studies:

How strongly do you oppose or support ... [1] stricter carbon emission standards to reduce global warming ... [2] the death penalty for murder?

The items both used six-point Likert measures: (1) strongly oppose; (2) modestly oppose; (3) slightly oppose; (4) slightly support; (5) modestly support; (6) strongly support.

Both of these issues -- the death penalty and climate change -- are ones on which hierarchical individualists and egalitarian communitarians are most intensely divided. I used ordered logit regression models to simulate how likely a "typical" hierarchical individualist (one whose scores on the "Hierarchy" and "Individualism" scales are both set at +1 standard deviation) and a "typical" egalitarian communitarian (-1 SD on each scale) were to "support" the indicated policy at some level (either slightly, moderately, or strongly) in the two studies.

Here's the outcome from the Dec. 2006 study (a 1500-person, nationally representive sample):

Basically, mirror images: the egalitarian communitarian is over 90% likely to support stricter carbon emission limits, and about 60% likely to support the death penalty; the hierarchical individualist is about 90% likely to support the death penalty and about 60% likely to support stricter carbon emission limits

Now here are the results in September 2012 (from an 800-person, nationally representative sample): 

Wow! An amazing increase in the degree of polarization on carbon-emission limits, with the egalitarian communitarian squeezing up close to 100% and the hierarch individualist dropping down to about 10% likely to support that policy. There's more polarization on the death penalty too: while the hierarch individualist of 2012 is hanging in at about 90% likely to support, the egalitarian communitarian is down from around 60% in 2006 to around 40% today.

I didn't expect to see such stark results -- on either issue, really.

Maybe I should have? On climate change, the common view is that the issue has become increasingly partisan. Remember, too, that 2006 was the year that Al Gore's movie came out, an event that many see as having helped brand the climate change issue ideologically.

I had thought the idea that climate change polarization was more recent was overstated. Well, I was right to remember climate change being highly polarized, culturally speaking, in 2006, but these data support the view that people (ordinary, not particulary partisan ones, remember) are much much more divided now! (Some think that's changing; yet our studies over the last yr haven't shown any abatement in cultural polarization.)

On capital punishment, it's pretty well known that support for the death penalty is generally declining. Consider this trend in Gallup's national polling:

The divide was 65% for, 28% against in Oct. 2006. Five years later, the divide had narrowed to 61% for to 35% against. That's something, and if you go back a bit further, the contemporary trend seems even more noticeable, albeit modest.

From our data, it looks like most of the action is coming from egalitarian communitarians, who moved from being more likely to support to more likely to oppose. Hierarch individualists don't seem to have budged!

I think this is pretty interesting (almost as interesting as learning that today's army has fewer horses and bayonets than it did in 1916, news the President announced in the middle of my writing this post; yet another shock!).

But I have to say, I myself am not so interested in the policy positions favored by people with opposing cultural outlooks.  One can't have a policy position -- on anything -- without making a judgment of value. Not surprisingly, people with different values tend to support different policies (although as I said, in this case the changing strength of the conflict on carbon emission limits did surprise me).

What's more interesting -- to me, at least! -- is the contribution that cultural values make to perceptions of risk and related facts. Cultural cognition is about how people's cultural outlooks shape processing of various types of information--from scientific findings to expert opinions to images captured in a video

What I wish we had collected data on last month and also back to 2006 was the relationship between our subjects' cultural outlooks and their perceptions of whether the death penalty deters. That's one of the classic examples of an empirical issue that's driven by symbolic or expressive, cultural outlooks. Indeed, every schoolboy & -girl knows  Lord, Ross & Lepper's classic biased assimilation study, which found that people conformed their assessments of studies on the deterrent effect of the death penalty to their pre-existing positions.

But there's really interesting evidence that people in general are becoming less convinced that the death penalty deters without changing their mind on the death penalty. Consider this from Gallup:

If you go back to '85 -- back when the death penalty was a big deal (remember Willie Horton? that was from the 1988 presidential campaign; the issue is dead as ... well, it's no longer relevant at all to national political divisions) & closer to when Lord, Ross & Lepper did their study, 62% believed the death penalty deterred, and only 31% that it didn't. Today the proportions are close to reversed. Yet support the death penalty has not tailed off nearly so dramatically.

What's going on? ... Maybe because the issue is less salient as a focus for cultural contestation (again, it's been off the national political state for a quarter century!), people don't feel the same pressure to conform their consequentialist rationales to their cultural evaluations of the death penalty; in other words, motivated consequentialism might be associated most strongly with culturally polarizing issues...

Just a conjecture! I really am perplexed!

And I like feeling that way.  Thanks Fagan Capital Punishment students for a really good question.

That's all for this episode of "WSMD? JA!"  See you next time!

Some references:

Ellsworth, P.C. & Gross, S.R. Hardening of the Attitudes: Americans’ Views on the Death Penalty. J. Soc. Issues 50, 19 (1994).

 

Saturday
Oct202012

Outline of position on (attitude about) how to improve policy-supportive science communication 

Had a conversation w/ a really smart scholarly friend who shares my basic orientation toward science communication & who is doing cool things to advance it. For his benefit, after we were done I reduced my thoughts to a small annotated outline. Figured I might as well put the memo up on the blog. It's the internet equivalent, I suppose, of a guy on a desert island putting a message in a bottle & tossing it into the ocean--the nice thing being that there are *so many* other islands out there on the net that the hope the bottle will end up washing onto the shore of someone who finds its contents useful is not nearly so farfetched or desperate!

0.  Polarization does not stem from a deficit in the public's comprehension of
     science 
(or the exploitation of any such deficit by self-interested actors)

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

Kahan, D. Fixing the Communications Failure. Nature 463, 296-297 (2010).

Misinformation and climate change conflict

1. On how to make sense of cultural cognition, science comprehension, and cultural
    polarization:

The problem isn’t the mode of comprehending science; it’s the contamination of the “science communication environment” in which cultural cognition (or like mechanisms) can be expected to & usually do reliably lead diverse, ordinary people to converge on best science. The contamination consists in the attachment of antagonistic cultural meanings to facts that admit of scientific investigation.

Kahan, D. Why we are poles apart on climate change. Nature 488, 255 (2012).

Nullius in verba? Surely you are joking, Mr. Hooke! (or Why cultural cognition is not a bias, part 1) 

The cultural certification of truth in the Liberal Republic of Science (or part 2 of why cultural cognition is not a bias)

2. On what to do                                                                                                        

a. Protect science communication environment: We need to perfect the knowledge we have for forecasting potential contamination—on, say, novel issues like nanotechnology, synbio, or GMOs—and implement procedures (say, govt review of “science communication impact” of govt-funded science research & of regulatory decisionmaking) to use that knowledge to preempt such contamination.

The science of science communication: an environmental protection conception (Lecture at National Academy of Sciences Sackler Colloquium, May 22, 2012)

b.  Decontaminate already polluted environments: Hard to do but not impossible. Involves figuring out how through conscious reorientation of meaning cues—identity of advocates, narrative frames for conveying info, etc.—so that toxic associations get broken down.

Kahan D.M., Jenkins-Smith, J., Tarantola, T., Silva C., & Braman, D., Geoengineering and the Science Communication Environment: a Cross-cultural Study, CCP Working Paper No. 92 (Jan. 9, 2012).

c.  Select policy/engagement locations in manner that exploits relative quality of scicom environments. The cues that determine what issues mean are highly sensitive to context, including what the policy question is, who is involved in the discussion, & where it is occurring. If one context is bad, then see if you can find another.

E.g., climate: The national-level “mitigation” discussion is highly polluted; the local, adaptation focused one is not.

The "local-adaptation science communication environment": the precarious opportunity

Go local, and bring empirical toolkit: Presentation to US Global Change Research Program

3. How to do it: scientifically

We have knowledge on these dynamics.  So just guessing what will work to promote constructive, nonpolarized public engagement with scientific information—without looking at & trying to make informed conjectures based on that knowledge—is a huge mistake (an ironic one, too, since it is an utterly unscientific way to do things).

An even bigger mistake is to do scicom w/o collecting information. Disciplined observation & measurement can be used to calibrate & improve knowledge-informed strategies as a communication effort (say, an attempt to build support for sensible use of climate science in an adaptation setting) unfolds. But just as important, the collection of information generated by these means is critical to extending practical knowledge of how to do effective communication in field settings. What’s learned every time people engage in scientifically informed science communication is more information that can be used to help improve the conducting of such activity in the future.

Thus, people who engage in policy-supportive science communication efforts w/o systematic information collection protocols – including ones that test effectiveness of their methods in promoting open-minded enagement—are casually dissipating & wasting a knowledge resource of tremendous value. They are in fact unwittingly aiding & abetting entropy--an act of treason in the Liberal Republic of Science!

Wild wild horses couldn't drag me away: four "principles" for science communication and policymaking 

Honest, constructive & ethically approved response template for science communication researchers replying to "what do I do?" inquiries from science communicators

 

Wednesday
Oct172012

Wanna see more data? Just ask! Episode 1: another helping of GM food

Okay, here's a new feature for the blog: "Wanna see more data? Just ask!"  

The way it works is that if anyone sees interesting data in one of my posts, or in any of our studies (assuming it was one I worked on; for others, I'll pass on requests but don't necessarily expect an answer; some of my colleagues have actual lives), and has some interesting question that could be addressed by additional analyses, that person can post a request (in comments section or by email to me) & I'll do the analyses and post the results.

Now notice I said the question has to be "interesting." Whether it meets that standard is something I'll decide, using personal judgement, etc. But here are some general, overlapping, related criteria:

1.  The request has to be motivated by some conjecture or question.  Basically, you have have some sort of theoretically grounded hypothesis in mind that can be tested by the analysis you'd like to see. The most obvious candidate would be a conjecture/question/hypothesis that's in the sprit of a plausible alternative explanation for whatever conclusion it was that I reached (or the study did) in presenting the data in the first place. But in any case, give some indication (can be brief; should be!) of what the question/hypothesis/conjecture that you are curious about is & why. 

2. Tell me how I can do the analysis and why doing it that way can be expected to generate some result that gives us more reason to accept a particular answer to the motivating question, or more rason to accept or reject the motivating hypothesis, than we would have had without the analysis.  The "how to do" part obviously will be constrained by what sorts of variables are in the dataset. Usually we have lots of demographic data as well as our cultural outlook variables and so forth. The "why" question requires specifying the nature of the causal inference that you think can be drawn from the analysis.  It's gotta make sense to be interesting.

3. No friggin' fishin trips! Don't ask me to correlate global warming with the price of cottage cheese just because you think that would be an amusing thing to do.

4. Don't even think of asking me to plug every conceivable variable into the right-hand side of a regression and see what sort of gibberish pops out. Of course, I'm happy to do multivariate analyses, but each variable has to be justified as part of a model that relates in a specifiable way to the interesting conjecture motivating the request and to the nature of the inference that can be drawn from the analysis. Or to put it another way, the analysis has to reflect a cogent modelling strategy. Overspecified regression analyses are usually a signature of the lack of a hypothesis -- people just see what turns out to be significant (something always will with enough variables) & then construct a post-hoc, just-so story for the result. In addition, the coefficients for overspecified models are often meaningless phantoms-- the impact of influences "holding constant" influences that in the real world are never "constant" in relation to those influences.... I'll write another post on why "over-controlling" is such a pernicious, mindless practice....

Okay. This first installment is responsive to questions posed in response to "part 3" of the GM food risk series. Disccusants there were curious about whether the "middling" mean score for the GM food risk item was best understood as "not sure; huh?," as I proposed, or as a genuine, mid-level of concern. One suggested seeing some more raw data might help, and on reflection I can think of some ways to look at them that might, at least a bit.

Consider these histograms, which reflect the distribution of responses to the 8-point industrial-strength risk perception item for "Global warming" (left) and "Genetically modified foods" (right):

Here are some things to note. First, GM food distribution is much more "normal" -- bell shaped -- than the global warming distribution. Indeed, if you compare the black line -- the statistical "normal density distribution" given the mean & SD for the global warming data --with the red one -- the kernel density plot, which "fits" a locally weighted regression to the data-- you can see that the distribution for global warming risk perceptions is closer to bimodal, meaning that the subjects are actually pretty divided between those who see "low risk" and those who see "high."  There's not so much division for GM foods.

Second, the GM foods distribution has a kind of a fat mid-point (low kurtosis). That's because a lot of survey respondents picked "3," "4," & "5." Because an excess of "middle choices" is a signature of "umm, not sure" for risk perception measures of this sort, I am now even more persuaded that the 800 members of this nationally representative sample didn't really have strong views about GM foods in relation to the other risks, all of which were ones that displayed substantial cultural polarization.

But my confidence in this conclusion is only modest.  The cases in which a middling mean signifies generalized "don't know" often have much more dramatic concentrations of responses toward the middle of the scale (high kurtosis); indeed, the labels that were assigned to each point on the likert item risk-perception measure were designed to mitigate the middle/don't-know effect, which is usually associated with scales that ask respondents to estimate a probability for some contingency (in which case people who don't know mean to convey that with "50%.").

Now consider these two figures:

These are the kernel density estimates for responses to these two risk-perception items when the sample is split at the mean of the "individualism-communitarianism" scale. Basically, the figures allow us to compare how "individualists" and "communitarians" are divied on global warming (left) and GM foods (right).

Do you see what I do? The individualists and communitarians are starkly divided on climate change: the latter is skewed strongly toward high risk, and the former toward low (although perhaps a bit less so; if I looked at "hierarch individualists," you'd really see skewing). That division (which, again, is compounded when the hierarchical disposition of the subjects is taken into account as well) is the source of the semi-bimodal distribution of responses to the global warming item. 

Now look at individualists & communitarians on GM foods. They see more or less eye-to-eye. This is corroboration of my conclusion in the last post that there isn't, at least not yet, any meaningful cultural division over GM foods. (BTW, the pictures would look the same if I had divided the subjects into "hierarchs" and "egalitarians"; I picked one of the two worldview dimensions for the sake of convenience and clarity).

Whaddya think? Wanna see some more? Just ask!

 Reference

de Bruin, W.B., Fischhoff, B., Millstein, S.G. & Halpern-Felsher, B.L. Verbal and Numerical Expressions of Probability: “It's a Fifty–Fifty Chance”. Organizational Behav. & Human Decision Processes 81, 115-131 (2000).

 

Monday
Oct152012

Timely resistance to pollution of the science communication environment: Genetically modified foods in the US, part 3

Okay: some data already!

As explained in parts one & two of this series, I’ve been interested in the intensifying campaign to raise the public profile of—and raise the state of public concern over—GM foods in the US.

That campaign, in my view, reflects a calculated effort to infuse the issue of GM-food risks with the same types of antagonistic meanings that have generated persistent states of cultural polarization on issues like climate change, nuclear power, the HPV vaccine, and gun control.  To me, that counts as pollution of the science communication environment, because it creates conditions that systematically disable the faculty that culturally diverse citizens use (ordinarily with great reliability) to figure out what is known to science.

But as I commented in the last post, the campaign has provoked articulate and spirited resistance from professional science communicators in the media. I view that as an extremely heartening development, because it furnishes us with what amounts to a model of how professional norms might contribute to protecting the science communication environment from toxic cultural meanings. Democratic societies need both scientific insight into how the science communication environment works and institutional mechanisms for protecting it if they are to make effective use of the immense knowledge at their disposal for advancing their citizens' common welfare.

But where exactly do things stand now in the US?  Historically, at least, the issue of GM-food risks has aroused much less attention, much less concern, than it has in Europe. That could change as a result of culturally partisan communications of the sort we are now observing, but has it changed yet or even started to?

John Timmer, the science editor for Ars Technica, actually posed more or less this question to me in a twitter exchange, asking whether there really is “anything like” the sort of cultural conflict toward GM foods risks that we see toward climate-change risks in this country. Questions like that deserve data-informed answers.

So here’s some data from a recent (end of September) survey. The sample was a nationally representative one of 800 individuals. One part of the survey asked them to rank on a scale of 0-7 “how serious” they viewed a diverse set of risks (I call this the “industrial strength risk perception measure”). 

The question, essentially, is whether GM foods are at risk of acquiring the sorts of cultural meanings that divide “hierarchical individualists” and “egalitarian communitarians” on various issues. Accordingly, I have constructed statistical models that permit us to see not only how GM-food risks rank in relation to others for the American population as a whole but also whether and strongly GM-food risks divide those two segments of the population.

 

There are a number of things one could say here.

One is—holy smokes, the US public is apparently more worried about GM-food risks than they are about global warming, nuclear power, and guns! The “average American” would assign a ranking of 4.3 to GM foods (just above “moderately risky”) but only 3.9 for global warming (just below), 4.0 (spot on) for nuclear, and 2.9 (between “low” and “moderate”) for guns.

But that wouldn’t be the way I’d read these results. First of all, while it’s true that GM foods are apparently more scary for the “average” American than guns, nuclear power, and climate change, the striking thing is just how unconcerned that “person” is with any of those risks. “High rates of taxation for businesses” are apparently much more worrisome for the "mean" member of the American population than the earth overheating or people being shot. Given how unconcerned this guy/gal is with all these other risks, should we get all that excited that he/she is a bit more more concerned about GM foods?

Notice too that the "mean" member of the population isn't as concerned with GM foods as with high business tax rates (4.5)—or as illegal immigration (4.7) or government spending (5.3)? What to make of that?...

But second and more important, look at the cultural variance on these risks.  Global warming turns out to be the most serious risk for egalitarian communitarians. Indeed, that group sees nuclear power as much riskier, too, than either business tax rates, illegal immigration, or “government spending,” which are about as scary for that group as gun risks.  Hierarchical individualists have diametrically opposed perceptions of the dangers posed by all of these particular risk sources.

Bear in mind, hierarch individualists and egalitarian communitarians aren’t rare, or unusual people. They are pretty recognizable in lots of respects—including their political affiliations, which amount to “Independent leans Republican” and “Independent leans Democrat,” respectively.

Given this, it’s not clear that it makes much sense to assign meaning to the “average” or “population mean” scores on these risks. Because real people have particular rather than "mean" cultural outlooks, we should ask not how the "average" person perceives culturally contested risks, but how someone like this see those risks as opposed to someone like that?

Yet note, the risks posed by GM foods are not culturally contested. We are all, in effect, "average" there.  Moreover, for both cultural hierarchical individualists and egalitarian communitarians, GM-food risks are in the “middle” of the range of risk sources they evaluated.

So what I’d say, first, is that there is definitely no cultural conflict for GM foods in the US—at least not of the sort that we see for climate change, nuclear power, guns, etc.

Second, I’d say that I don’t think there’s very much concern about GM foods generally. The “middling” score likely just means that members of the sample didn’t feel nearly as strongly about GM foods as they felt—one way or the other—about the other risks. So they assigned a middling rating.

But third, and most important, I’d say that this is exactly the time to be worried about cultural polarization over GM foods.

As I said at the outset of this series, putative risk sources aren’t borne with antagonistic cultural meanings. They acquire them.

But once they have them, they are very very very hard to get rid of. 

In both parts, I likened culturally antagonistic meanings to “pollution” of the “science communication environment.”  Given how hard it is to change cultural meanings, it’s got to be a lot easier and more effective to keep that sort of contamination out—to deflect antagonistic meanings away from novel technologies or ones that otherwise haven’t acquired such resonances—than it is to “clean it up” once an issue has become statured with such meanings.

Consider the debate over climate change, which is highly resistant to simple “reframings” strategies. Perhaps it would have worked to have put Nancy Pelosi and Newt Gingrich on a couch together before 2006. But today, the simple recommendation “use ideologically diverse messengers!” is not particularly helpful.

So I believe the data-informed answer to John Timmer's question is, no, GM foods don't provoke anything like the sort of antagonistic meanings that climate change expresses.

And for that reason, I'd argue, the efforts of reflective science journalists and others to resist the release of such contaminants into the science communication environment is as timely as it is commendable.

Part one in this series.

Part two.

Sunday
Oct142012

Resisting (watching) pollution of the science communication environment in real time: genetically modified foods in the US, part 2

Just as the health of individual human beings depends on the quality of natural environment, the well-being of a democratic society depends on the quality of the science communication environment.

The science communication environment is the sum total of cues, influences, and processes that ordinary members of the public rely on to participate in the collective knowledge society enjoys by virtue of science.

No one (not even scientists) can personally comprehend nearly as much of what is known to science as it makes sense for them—as consumers, as health-care recipients, as democratic citizens—to accept as known by science. To participate in that knowledge, then, they must accurately identify who knows what about what.

When the science communication environment is in good working order, even people who have only rudimentary understandings of science will be able to make judgments of that kind with remarkable accuracy. When it is not, even citizens with high levels of scientific knowledge will be disabled from reliably identifying who knows what about what, and will thus form conflicting perceptions of what is known by science—to their individual and collective detriment.

Among the most toxic threats to the quality of a society’s science communication environment are antagonistic cultural meanings: emotional resonances that become attached to risks or other policy-relevant facts and that selectively affirm and denigrate the commitments of opposing cultural groups.

Ordinary individuals are accustomed to exercising the faculties required to determine who knows what about what within such groups, whose members, by virtue of their common outlooks and experiences, interact comfortably with one another and share information without misunderstanding or conflict. Because antagonistic cultural meanings create strong psychic pressures for members of opposing groups to form and persist in conflicting sets of factual beliefs, such resonances enfeeble the reliable functioning of the faculties ordinary people (including highly science literate ones) use to participate in what is known by science.

Antagonistic cultural meanings are thus a form of pollution in the science communication environment. Their propagation accounts for myriad divisive and counterproductive policy conflicts—including ones over climate change, nuclear power, and private gun ownership.

In part one of this series, I described the complex of economic and political forces that have infused the issue of genetically modified (GM) foods with culturally antagonistic meanings in Europe.

I also noted the signs, including the campaign behind the pending GM food-labeling referendum in California, that suggest the potential spread of this contaminant to the US science-communication environment.

What makes the campaign a pollutant in this regard has nothing to do with whether GM foods are in fact a health hazard (there’s a wealth of scientific data on that; readers who are interested in them should check out David Tribe’s blog). Rather, it has to do with the deliberate use of narrative-framing devices—stock characters, dramatic themes, allusions to already familiar conflicts, and the like—calculated to tap into exactly the culturally inflected resonances that pervade climate change, nuclear power, guns, and various other issues that polarize more egalitarian and communitarian citizens, on the one hand, and more hierarchical and individualistic ones, on the other.

But as I adverted to, there is at least one countervailing influence that didn’t exist in Europe before it became a site of political controversy over GM foods but that does exist today in the US: consciousness of the way in which dynamics such as these can distort constructive democratic engagement with valid science, and a strong degree of resolve on the part of many science communicators to counteract them.

Science commentators like Keith Kloor and David Ropeik, e.g., have conspicuously criticized the propagation of what they view as unsubstantiated claims about GM Food health risks.

Both of these writers have been outspoken in criticizing ungrounded attacks on the validity of science on climate change, too. Indeed, Kloor recently blasted GM food opponents as the “climate skeptics of the Left.

Precisely because they have conspicuously criticized distortions of science aimed at discounting environmental risks in the past, their denunciation of those whom they see as distorting science to exaggerate environmental risks here reduces the likelihood that GM foods risks will become culturally branded.

Science journalists, too, have been quick to respond to what they see as the complicity of their own in participating in dissemination of questionable science claims on GM foods.

In one still-simmering controversy, a large number of journalists accepted an offer of advance access to an alarming study on GM-food risks in return for refraining from seeking the opinion of other scientists before publishing their “scoop” stories. Timed for release in conjunction with a popular book and a TV documentary, the study, conducted by a scientists with a high profile as supporter of GM-food regulation, was in fact thereafter dismissed as non-credible by expert toxicologists—although not before the alarming headlines were seized on by proponents of the California labeling proposition as well as European regulators.

Writing about the controversy, NY Times writer Carl Zimmer blasted the affair as a “rancid, corrupt way to report about science.” It was clear to the participating reporters, Zimmer observed, that the authors of the study were seeking to exclude any critical appraisal from the initial burst of attention” in the media, thereby “reinforcing opposition to genetically modified foods.” “We need to live up to our principles, and we need to do a better job of calling out bad behavior…. [Y]ou all agreed to do bad journalism, just to get your hands on a paper. For shame.”

Ars Technica editor John Timmer amplified Zimmer’s response. “Very little of the public gets their information directly from scientists or the publications they write,” Timmer pointed out. “Instead, most of us rely on accounts in the media, which means reporters play a key role in highlighting and filtering science for the public.” In this case, Timmer objected, “the press wasn't reporting about science at all. It was simply being used as a tool for political ends.”

One reason to be impressed by these sorts of reactions to GM foods is that they suggest the possibility of using professional norms as a more general device for protecting the quality of the science communication environment.

As I indicated in my last post, there is nothing inevitable about the process by which a risk issue becomes suffused with antagonistic cultural meanings.  Those kinds of toxic associations are made, not born.

It follows that we should make protection of the science communication environment a matter of self-conscious study and self-conscious action.  The natural environment cannot be expected to protect itself from pollution itself without scientifically informed action on our part. And the same goes for the quality of the science communication environment.

I’m of the view that the sorts of collective action that protection of the science communication environment requires will have to come from various sources, including government, universities, and NGOs.

But clearly one of the sources will have to be professional science communicators. Timmer is clearly right about the critical role it plays—not just in translating what’s known by science into terms that enable curious people to experience the thrill of sharing in the wondrous insights acquired through our collective intelligence (I myself am so so grateful to them for that!), but in certifying who knows what about what so that as democratic citizens people can reliably gain access to the knowledge they need to contribute to intelligent collective decisionmaking.

Animated by diverse motivations—commercial and ideological—actors intent on disabling the faculty culturally diverse citizens use to discern who knows what about what can thus be expected to strategically target the media. Strong professional norms are a kind of warning system that can help science journalists recognize and repel efforts to use them as instruments for polluting the science communication environment.

Unlike centrally administered rules or codes, norms operate as informal, spontaneous guides for collective behavior. They get their force from internalized emotional commitments both to abide by shared standards of conduct and to contribute to enforcement of them by censure and condemnation of violators. Norms are propogated as members of a community observe examples of behavior that express those commitments and see others responding with admiration and reciprocation. That all seems to be happening here. 

This unusual opportunity to watch an attempt to inject a new toxic meaning into the science communication environment also furnishes a unique opportunity to learn something about who can protect that environment from pollution and how.

Oh!  I said I would share some data on cultural perceptions of GM food risks in the US in this installment of the series. But don’t you agree that I’ve already gone on more than long enough? So I’ll just have to present the data next time—in the third, and I promise final, post in this (now) 3-part (I actually imagined only one when I started) series.  (But here's a sneak preview.)

Part one in this series.

Part three.

Friday
Oct122012

Watching (resisting) pollution of the science communication environment in real time: genetically modified foods in the US, part 1

Putative risk sources are not born with culturally divisive meanings. They acquire them.

Something happens—as a result perhaps of strategic manipulation but also possibly of accident or misadventure—that imbues some technology with resonances that selectively affirm and denigrate the outlooks of opposing groups. At that point, how to regard the risks the technology poses—not only what to do to ameliorate them, but whether even to acknowledge them as real—becomes a predicable occasion for disagreement between the groups’ respective members.

By highlighting the association between competing positions and competing cultural identities, such disagreement itself sharpens the antagonistic meanings that the technology bears—thereby magnifying its potential to generate conflict. And so forth & so on.

But the thing that imbues a technology (or a form of behavior or a type of policy) with culturally antagonistic meanings doesn’t have to happen. There’s nothing necessary or inevitable about this process.

It’s this contingency that explains why one putative risk source (say, the HPV vaccine) can provoke intense cultural conflict while another, seemingly equivalent source (say, the H1N1 vaccine) doesn’t. It explains, too, why one and the same risk (nuclear power, e.g., or “mad cow disease”) can provoke division in one society but indifference in another, seemingly similar society.

Consider genetically modified (GM) foods. Historically, GM foods—e.g., soybeans altered to resist diseases that otherwise would be controlled by chemical pesticides, or potatoes engineered to withstand early frosts—have not provoked nearly as much concern in the US as in Europe.  Such products can be found in upwards of 70% of US manufactured foods.  In Europe, the figure is supposedly closer to 5%, due to enactment of progressively stricter regulations over the last decade and a half.

But it’s certainly possible that something could happen to make US public attitudes and regulations move in the direction of Europe’s. Indeed, it could be happening.

There is now a concerted effort underway to raise the risk profile of GM foods. The most conspicuous manifestation of it is a California ballot proposition to mandate that all foodstuffs containing GM foods bear a label that advises (warns) consumers of this fact.

The proposition is supported by organic food producers and sellers, who funded the effort to get the initiative on the ballot and are now funding the campaign to secure its approval, as well as by certain environmental groups, which are playing a conspicuous public advocacy role.

A label is not a ban. But it can definitely be a precursor to something more restrictive.

Consumers logically infer from “advisory” labels that there is a reason for them to be concerned.

Psychologically they tend to greatly overreact to any information about chemical risks—including information that tries to prevent them from overreacting by characterizing the risks in question as small or uncertain.  It’s thus in the nature of modest, informational regulations to breed concerns capable of supporting stronger, substantive regulation.

Dynamics of cultural cognition, moreover, can fuel this sort of escalation. If the source of initial concern is transmitted in a manner that conveys antagonistic resonances, then the resulting division of opinion among members of different groups can feed on itself.

The movement to promote concern with GM foods seems ripe with antagonistic meanings of this sort. The information being disseminated to promote attention to the risks of GM foods in general, and to promote support for the California initiative in particular, are suffused with cues (stock characters, distinctive idioms, links to already familiar sources of conflict such as nuclear power & climate change) that are likely to resonate with those who harbor an egalitarian-communitarian cultural style and antagonize those with a more a hierarchical and individualistic one.

This framing of the issue could thus end up pitting members of these two groups—already at odds over climate change, nuclear power, gun control, and various other risks—against one another.

In that case, the US will have arrived at a state of cultural conflict over GM foods that seems to be the same one European nations followed. There, small, local farmers took the lead in proclaiming the health risks of GM food products, which were being supplied by their larger, transnational industrial-farm rivals.

Egalitarian environmental activists enthusiastically assimilated this charge into their broader indictment of industry for putting profits ahead of human welfare. Among the ironies here was the impetus such political activity imparted to blocking the production of so-called “golden rice,” a nutritionally enhanced GM grain that public health advocates hailed for the contribution it could make to combating afflictions (including preventable blindness) in malnourished children in the developing world.

I don’t know or even have a particularly strong intuition on what risks GM foods pose.

But I do have a very strong opinion that a state of cultural polarization over GM food risks would be a bad thing. As myriad controversies--from the nuclear power debate of the 1980s to the climate change debate of today--have made clear, when risk issues become infused with antagonistic cultural meanings, democratic societies are less likely to enact policies that reflects the best scientific evidence, whatever it might be.

Okay. Enough for now. In the next in this two part series, I will identify one important countervailing influence that didn’t exist in Europe before it became a site of conflict over GM food risks but that does exist now. I’ll also report some data that bears on the current degree of cultural polarization that exists in the US over the risks that GM foods present.

Parts two & three in this series.

References

Anderson, K., Damania, Richard and Jackson, Lee Ann,  (September 2004). World Bank Policy Research Working Paper No. 3395. Available at SSRN: http://ssrn.com/abstract=625272 in World Bank Policy Research Working Paper No. 3395. (ed. W. Bank)2004).

Ferrari, M. Risk perception, culture, and legal change : a comparative study on food safety in the wake of the mad cow crisis. (Ashgate Pub., Farnham, Surrey, England ; Burlington, VT; 2009).

Finucane, M.L. & Holup, J.L. Psychosocial and cultural factors affecting the perceived risk of genetically modified food: an overview of the literature. Soc Sci Med 60, 1603-1612 (2005).

Kahan, D. Fixing the Communications Failure. Nature 463, 296-297 (2010).

Kahan, D.M. Gentle Nudges vs. Hard Shoves: Solving the Sticky Norms Problem. U. Chi. L. Rev. 67, 607-45 (2000).

Kurzer, P. & Cooper, A. What's for Dinner? Comparative Political Studies 40, 1035-1058 (2007).

Slovic, P., Flynn, J., Mertz, C.K., Poumadere, M. & Mays, C. in Cross-Cultural Risk Perception: A Survey of Empirical Studies. (eds. O. Renn & B. Rohrmann) 55-102 (Kluwer Academic, Dordrecht, The Netherlands; 2000).

Sunstein, C.R. Laws of Fear: Beyond the Precautionary Principle. (Cambridge University Press, Cambridge, UK ; New York; 2005).

Monday
Oct082012

More R^2 envy!

To add to your "it's-not-how-big-but-what-you-do-with-itR2 file, this from Andrew Gelman:

 

Sunday
Oct072012

Checking in on the "white male effect" for risk perception 

I read a couple of interesting studies of risk and the “white male effect” recently, one by McCright and Dunlap published (advance on-line) in the Journal of Risk Research and another in working paper form by Julie Nelson, an economist at the University of Massachusetts at Boston.

The “white male effect” (WME) refers to the observed tendency of white males to be less concerned with all manner of risk than are women and minorities.  The phenomenon was first observed (and the term coined) in a study by Flynn, Slovic & Mertz in 1994 and has been poked and prodded by risk-perception researchers ever since.

WME was the focus of one early Cultural Cognition Project study. Extending study findings by Finucane, Slovic, Mertz, Flynn & Satterfield, a CCP research team (which included WME veterans Slovic & Mertz!) found that WME could be largely attributed to the interaction of cultural worldviews with race and gender. The WME was not so much a “white male effect” as a “white hierarchical and individualistic male effect” reflecting the extreme risk skepticism of men with those worldviews.

The design and hypotheses of the CCP study reflected the surmise that WME was in fact a product of “identity protective cognition.” Identity-protective cognition is a species of motivated reasoning that reflects the tendency of people to form perceptions of risk and other facts that protect the status of, and their standing in, self-defining groups.  White hierarchical individualistic males were motivated to resist claims of environmental and certain other risks, we conjectured, because the wide-spread acceptance of those claims would justify restrictions on markets, commerce, and industry—activities important (emotionally and psychically, as well as materially) to the status of white men with those outlooks.

The McCright and Dunlap article corroborates and strengthens this basic account of WME. Using political ideology rather than cultural worldviews to measure the latent motivating disposition, M&D find that the interaction of conservatism with race and gender explains a wide range of environmental risks (thus enlarging on an earlier study of their own, too, in which they focused on climate change).

M&D suggest that WME can be seen as being generated jointly by identity-protective cognition and “system justification,” a psychological dynamic that is said to generate attitudes and beliefs supportive of the political “status quo.” They defend this claim convincingly with the evidence that they collected. But I myself would be interested to see a study that tried to pit these two mechanisms against each other, since I think they are in fact not one and the same and could well be seen as rival accounts of many phenomena featuring public controversy over risks and related policy-consequential facts.

Nelson’s paper presents a comprehensive literature review and re-analysis of various studies—not just from the field of risk perception but from economics and decision theory, too—purporting to find greater “risk aversion” among women than men.

Actually, she pretty much demolishes this claim. The idea that gender has some generic effect on risk perception, she shows, is inconsistent with the disparity in the size of the effects reported across various settings. Even more important, it doesn’t persist in the fact of experimental manipulations that are more in keeping with explanations based on a variety of context-specific or culturally grounded dynamics (such as stereotype threat).

Nelson hints that the ubiquity of the “female risk aversion” claim in economics might well reflect the influence of a culturally grounded expectation or prototype on the part of researchers and reviewers—an argument that she in fact explicitly (ruthlessly!) develops in a companion essay to her study.

I got so excited by the papers that I felt like I had to so some data analysis of my own using responses from a nationally representative sample of 800 subjects who participated in a CCP study in late September.

The top figure, which reflects a regression model that includes only gender and race, shows the classic WME for climate change (the outcome variable is the “industrial strength risk perception” measure, which I’ve normalized via z-score). 

The bottom figure graphs the outcome once the worldview measures and appropriate race/gender/cultural interaction terms are added.  It reveals that WME is in truth a “white hierarchical individualistic male effect”: once the intense risk skepticism associated with being a white, hierarchical individualistic male is taken into account, there’s no meaningful gender or race variance in climate change risk perceptions to speak of.

For fun (and because the risk perception battery in the study also had this item in it), I also ran a model  for “the risk that high tax rates for businesses poses to human health, safety, or prosperity” in American society. Relative to the ones displayed in climate-change risk perceptions, the results are inverted:

In other words, white males are more worried about this particular risk, although again the gender-race difference is an artifact of the intensity of the perceptions of white hierarchical individualistic males.

That these characteristics predict more risk concern here is consistent with the identity-protective cognition thesis: because it burdens an activity connected to status-enhancing roles for individuals with this cultural identity, white hierarchical individualistic males can be expected to form the perception that high tax rates on business will harm society generally.

This finding also bears out Nelson’s most interesting point, in my view, since it confirms that men are more risk tolerant than women” only if some unexamined premise about what counts as a "risk" excludes from assessment the sorts of things that scare the pants off of white men (or at least hierarchical, individualistic ones).

Cool papers, cool topic!

References

Finucane, M., Slovic, P., Mertz, C.K., Flynn, J. & Satterfield, T.A. Gender, Race, and Perceived Risk: The "White Male" Effect. Health, Risk, & Soc'y 3, 159-172 (2000).

Flynn, J., Slovic, P. & Mertz, C.K. Gender, Race, and Perception of Environmental Health Risk. Risk Analysis 14, 1101-1108 (1994).

Kahan, D.M., Braman, D., Gastil, J., Slovic, P. & Mertz, C.K. Culture and Identity-Protective Cognition: Explaining the White-Male Effect in Risk Perception. Journal of Empirical Legal Studies 4, 465-505 (2007).

McCright, A.M. & Dunlap, R.E. Bringing ideology in: the conservative white male effect on worry about environmental problems in the USA. J Risk Res, doi:   (2012).

McCright, A.M. & Dunlap, R.E. Cool dudes: The denial of climate change among conservative white males in the United States. Global Environmental Change 21, 1163-1172 (2011).

Nelson, Julie.  Are Women Really More Risk-Averse than Men?, INET Researcn Note (Sept., 2012)

Nelson, Julie.  Is Dismissing the Precautionary Principle the Manly Thing to Do? Gender and the Economics of Climate Change, INET Research Note (Sept. 2012)

 

Friday
Oct052012

ASTAR: bringing the culture of science to law--and the culture of law to science

Last weekend I attended and made a presentation at the Advanced Science & Technology Adjudication Resource Center (ASTAR).

ASTAR is an amazing concept. The goal of the program is to train a cadre of “science & technology resource” judges with the knowledge needed to preside over cases involving highly complex scientific issues.

Prospective ASTAR resource judges are awarded training scholarships after being nominated by the judiciaries in their state (or by one of the two participating federal courts). They then must complete 120-hours of training, including 60 hours of participation in regularly convened sessions that focus on one or another specific area of science. Once they get through that—if, really; there’s a “Ranger School” aspect to this—they are deemed ASTAR Fellows, and play an active role in the conduct of the program in addition to serving as their jurisdictions’ “resource judges.”

As impressive as all this sounds, seeing it in action is even more awe-inspiring.

The topic for this session was “Management Of Complex Cases Involving Environmental Hazards.” There were 22 (I think; I lost count!) 3/4-hour sessions crammed into the weekend. Most of them involved nuclear radiation and were taught—very expertly—by scientists from the Los Alamos National Laboratory (where I got to give a talk on Monday; more on that later).

The judges were dedicated students—bombarding the lecturers with insightful questions (many of which related to the readings the judges were assigned to do before arrival).

In my session, I talked about “risk perception & science communication.” The basic message I tried to impart was that cultural cognition is something that has to be understood by those who manage any process of fact-finding, particularly one involving laypeople (or experts, for that matter, making decisions outside of their own domains of expertise).

Obviously judges fit that description, and in addition to reviewing studies that show the impact of cultural cognition on public risk perceptions generally, I also showed how the same dynamics can affect jurors’ perceptions of trial evidence. (Slides here.)

Actually, courts are in many respects way ahead of other institutions, in & out of government, in preparing themselves to play an intelligent role in managing the impact of cultural cognition on factfinding.

Judges know that valid evidence doesn’t establish its own validity or otherwise ineluctably guide people to the truth.

They know that ordinary people likely can make accurate and defensible factual determinations on matters that turn on scientific and other forms of evidence-- but only if information is presented to them in a form and (just as important) an environment suited to the faculties ordinary people use to identify who knows what about what.

They also know that how to assure information gets presented in such a manner is not something that one can just figure out by personal hunch & speculation. Fitting the presentation of scientific & other evidence to the reasoning faculties of ordinary people is a topic that admits of--indeed, demands-- scientific investigation.

Accordingly, judges (or at least the best ones, like those who are part of & support ASTAR) want to be sure they keep up with the what’s scientifically known about how to promote reliable factfinding.

That’s not the case, I pointed out to the ASTAR judges, for many other actors whose job it is to help ordinary members of the public figure out facts—ones essential to planning their financial futures, to making intelligent decisions as consumers of health care, and to making informed decisions as citizens of a self-governing society.

To illustrate this point, I told the judges about the horrendous and inexcusable science-communication misadventure surrounding the HPV vaccine. Combining our CCP study on HPV vaccine risk perceptions with media reports, I reviewed how the vaccine came to be stigmatized with the divisive cultural meanings that continue to suppress vaccination rates

Merck polluted the science communication environment on that one. But that happened only because the FDA and CDC didn’t even know that the path the company was urging on them was one that would fill the atmosphere with disorienting partisan meanings. They didn’t know it was actually their job to make sure that didn’t happen.

And the reason they didn’t know those things, I’m sure, is that they were (and I’m worried likely remain) entirely innocent of the science of science communication.

In its monumentally important report on the state of forensic science, the National Academy of Sciences called for courts and legislators, law-enforcement agencies and universities all to combine to bring the “culture of science to law.”

The ASTAR judges are doing that.

What's more, they are doing it in a way that reflects the signature virtues of their own profession—including its insistence that lawyers and judges become familiar with expert or technical forms of knowledge essential to the performance of their own work, and that judges in particular assume responsibility for securing the conditions most conducive to informed decisionmaking in the courtroom.

Bringing these aspects of the culture of law to science would go a long way to remedying the institutional deficits in science communication that prevent our society from making full use of the vast bounty of knowledge at its disposal.

Thursday
Oct042012

Graphing interactions so that curious people can actually *understand* them

A friend & collaborator asked me,

So...could you send me a quick tip/reference on how to best graph interactions in regression? I'm just thinking of simple line-charts, comparing divergent slopes for two or three different groups after controlling for the other vars in the equation. I'm *sure* this is easily done, but I'm blanking on how. I mean, it's easy enough to draw the slope based on the unstandardized coefficient. And the Y-intercept to start that line from is...what? the B of the constant?

My response:

I'm sure you are asking b/c you are unsatisfied, understandably in my view, w/ the graphing recommendations that appear in references like Aiken, L.S., West, S.G. & Reno, R.R. Multiple Regression: Testing and Interpreting Interactions. (Sage Publications, Newbury Park, Calif.; 1991) &  Jaccard, J. & Turrisi, R. Interaction Effects in Multiple Regression, Edn. 2nd. (Sage Publications, Thousand Oaks, Calif.; 2003) -- even though those are definitely the best references for understanding the statistical logic of interactions & making intelligent modeling choices.

There are excellent papers that reflect general disatisfaction w/ how social scientists tend to graphically report (or not) the results of multivariate regression models. They include:
  • Gelman, A., Pasarica, C. & Dodhia, R. Let's Practice What We Preach: Turning Tables into Graphs. Am Stat 56, 121-130 (2002).
  • King, G., Tomz, M. & Wittenberg., J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am. J. Pol. Sci 44, 347-361 (2000).
  • Kastellec, J.P. & Leoni, E.L. Using Graphs Instead of Tables in Political Science. Perspectives on Politics 5, 755-771 (2007).
They don't deal w/ interactions per se, but b/c they address the objective of how to make regression model results intelligible in general, you can easily derive from them ideas about strategies that work w/ models that include cross-product interaction terms.

I'll show you some examples below but here are some general tips I'd give: 

a. *don't* graph data after splitting sample (e.g., into "high," "medium" & "low" in political sophistication)... Graph the results of the model that includes all the relevant predictors & cross-product interaction terms as applied to the entire sample; those are the results you are trying to display & splitting sample will change/bias the parameter estimates.

b. consider z-score normalization for the outcome variable: you won't have to worry about the intercept (it should be zero, of course), you'll avoid lots of meaningless "white space" if values within of +/-1 or +/-2 SDs (the end points for y-axis) are concentrated within a middling portion of the  outcome measure. Also for most readers, reporting the impact in terms of SDs of the outcome variable will be more intelligible than differences in raw units of some arbitrary scale (the sort you'd get by summing the likert items to form a composite likert scale, e.g.)

c. rather than graphing *slopes*, consider plotting regression estimates based on sensibly contrasting values for the predictors (and corresponding values for the cross-product interaction term); the "practical effect" of the interaction is likely to be easier to grasp that way than comparison of visual differences in slopes

d. if you are using OLS to model responses to a likert item, consider using ordered logit instead -- maybe you should be doing this anyway, but in any case, probabilities of responding at particular level (or maybe range of levels; say "agree either slight, moderately or strongly vs disagres slighly, modreately, or strongly") conditional on levels of predictor & moderator are graphically more intelligible  than estimated values on an arbitrary continuous scale.

e. consider graphing estimated *differences* (& corresponding CIs) in the outcome variable at different levels of moderator; e.g, if difference increases between subjects who are from different groups (or who vary  +/- 1 SD on some continuous predictor) conditional on whether the value of some continuous moderator, then use bar graph w/ CIs or some such to show how much greater the estimated difference between the two groups is at the two levels of the moderator 

f. consider monte carlo simulation of estimated impact of contrasting sets of predictors & moderators (& associated interactions); do kernel-density plots for 1,000 or 2,000 values of each -- it's a *really* good way to show both the contrast in the estimates & the precision of the estimates (much better than standard CIs). See King et al. above 

g. usually prefer connected lines to bar graphs to display contrasts; former are more comprehensible

h. in general, don't use standardized regression coefficients but do center continuous predictors (or convert them to z-scores) so that people who are reading the table can more readily interpret them

Have attached [reproduced below] a bunch of CCP study examples that reflect one or another of these strategies or related ones. BTW, of course, all of these reflect things that I learned  to do from collaborating w/ Don [Braman], who like all great teachers teaches people how to teach themselves.

note: all examples below are clickable thumbnails that expand to larger size for closer inspection

 

 

 

Tuesday
Oct022012

The aporetic judge

Judge Mark Kravitz, of the Federal District Court for the District of Connecticut died yesterday from Lou Gehrig’s disease. He was 62.

In my 2011 Harvard Law Review Foreword, I described a style of judicial reasoning and opinion writing that I characterized as “aporetic.”

Aporia is an ancient Greek term referring to a mode of argumentative engagement that evinces comprehension of an issue’s ineradicable complexity.

Aporia is not a state of uncertainty or equivocation (indeed, it’s not really anything that can be described by a single English word I can think of). One can reach a definitive conclusion about a problem and still be aporetic in assessing it.

But if one adopts a position that denies or purports to dispel the difficulty that a truly difficult issue poses, or that fails to recognize the undeniable weight of the opposing considerations on either side, then one isn’t being aporetic. Indeed, in that case, one is actually not getting the issue at hand, no matter how one resolves it.  The effacement of real complexity signifies a deficiency in intellectual character.

Judicial reasoning—of the sort that is expressed openly in court opinions—tends not to be aporetic. Of course, most of the issues that courts resolve are not fraught with complexity. But even in those that really are, judges tend to effect a posture of unqualified, untroubled confidence.

This form of comic overstatement is most conspicuous in Supreme Court opinions. Every relevant source of guidance (text, purpose, precedent, policy, tradition, “common sense” etc.) indisputably, undeniably converges on a single conclusion, the Justices emphatically insist. We are supposed to believe this even though the Court’s primary criterion for review is the existence of an issue that has divided lower courts, and even though the Justices themselves often disagree about which outcome in a particular case is supported indisputably, undeniably by every conceivable consideration.

But actually, there’s nothing funny about such puffing. On the contrary, it’s disturbing.

Hyperbolic certitude diminishes the legitimacy of the law by conveying to those who are disappointed by the outcome of a case that that the judge who decided it was biased, and intent on deception.

It also denigrates reason. It embodies in the law an attitude that breeds cynicism and dulls reflection.

In my Foreword, I defended an alternative, aporetic idiom of judicial reasoning that recognizes rather than effaces genuine complexity. Aporia in judicial reasoning, I argued, should be seen as a judicial virtue—because in fact it is. Being able to see complexity and being moved to engage it openly are character dispositions, and they conduce to being a good judge.  A judge who is committed to being just will experience aporia when he or she must decide a genuinely complex case; and by resort to aporetic reasoning in his or her opinion, that judge assures citizens generally that their rights are being determined by someone committed to judging impartially.

Mark Kravitz had this virtue. In fact, for me, he was and remains the model of it.  Before I had occasion to observe him as a judge, I had (despite many years studying and practicing law) only a dim, inchoate sense of judicial aporia; when I try to make the picture as vivid and compelling for others as it now is for me, I try to describe Mark Kravitz.

Last April, Judge Kravitz decided a case—one of his last—in which members of the Occupy protest movement brought a suit to try to halt the imminent, forcible removal of their tent city from the New Haven Green. He denied their motion for an injunction.

No one can read his opinion, though, and escape the conclusion that the issues it presented were difficult.  Indeed, in a tone that was rare in his opinions, Judge Kravitz expressed anger at the city’s attorneys for attempting to avoid—and thus for seeking to tempt the court to avoid—acknowledging the seriousness of the Occupy protestors’ position. Dismissing the city attorneys’ argument that the protestors’ encampment did not qualify as “speech” protected by the First Amendment, the judge wrote: “One would have to have lived in a bubble for the past year to accept Defendants' claim that Occupy's tents ‘could simply mean that the plaintiffs enjoy camping.’ ”

The Occupy movement, in New Haven as elsewhere, aims to exemplify its message: to express the desire that the economically disenfranchised become more central to American public life by literally placing the economically disenfranchised in the center of America's public spaces. Defendants need not deny the obvious political expressivity of this act in order to argue that reasonable limits on acts like this may still be necessary and appropriate.

The protestors deserved an opinion that acknowledged their dignity and public spirit. As disappointed, moreover, as they no doubt were to lose the case, I suspect they likely will be able to make good use of the portion of the opinion I’ve quoted (likely they will see the value, e.g., of including it in a demonstration-permit application, something the New Haven protestors denied the authority of the City to require as a condition of convening a protest on the Green).

Those inclined to distrust the City deserved to know that its stated reasons for ending the protest were being scrutinized by a decisionmaker intent on being fair. They got that, too, from the quoted language, and from the numerous points in the opinion that acknowledged the force and seriousness of the protestors’ arguments even in the course of deciding against them....

We all deserve judges who are unafraid to see, and unafraid to tell us they see, genuine complexity. We have one less judge of this character today than we had yesterday. But by furnishing us such a clear and inspiring picture of what this judicial virtue looks like, Mark Kravitz gave us a resource we can use to assure that there are many, many more aporetic judges in the future than we have ever had in the past.

 

Tuesday
Sep252012

Tragedy of the Science-Communications Commons

Giving lecture today at Hampshire College. Here's the summary:

Culture, Rationality, and Risk Perception: the Tragedy of the Science-Communication Commons

From climate change to the HPV vaccine to gun control, public controversy over the nature of policy-relevant science is today a conspicuous feature of democratic politics in America. A common view attributes this phenomenon to the public’s limited comprehension of science, and to its resulting vulnerability to manipulation by economically motivated purveyors of misinformation. In my talk, I will offer an alternative account. The problem, I will suggest, is not a deficit in rationality but a conflict between what’s rational at the individual and collective levels: ordinary members of the public face strong incentives – social, psychological, and economic – to conform their personal beliefs about societal risk to the positions that predominate within their cultural groups; yet when members of diverse cultural groups all form their perceptions of risk in this fashion, democratic institutions are less likely to converge on scientifically informed policies essential to the welfare of all. I will discuss empirical evidence that supports this analysis--and that suggests potential strategies for securing the collective good associated with a science communication environment free of the conflict between knowing what is known and being who we are.

The talk will feature data from our study of science-comprehension and cultural polarization on climate change and our experimental examination of how using geoengineering as a framing device can promote more open-minded engagement with climate science.

Slides here.

Monday
Sep242012

Climate-change risk communication and risk management for businesses

I haven't had a chance to read this really interesting-looking book yet (I just ordered it) but I find the simple existence of it fascinating.

As a national political issue, the issue of global climate change has much more relevance to people as a focus for conveying to themselves & others that they belong to a certain cultural "team" than it does as any sort of thing that might affect their or anyone else's health, welfare, etc. (now or in the future).  As individual consumers, voters, advocates, etc., ordinary members of the public just don't matter enough to have any effect on the risks posed by global climate change (or by ill-considered responses to it). On the other hand, how their beliefs relate to those of others in their community matters a ton. We judge people's characters by the positions they take on whether climate change is a "a global crisis" or a "massive hoax." Being out of sync with those on whom we depend for support -- materially, emotionally, psychically -- can be devestating. So people tend to extract from the "evidence" on climate change the information that really matters: what is someone like me supposed to think.

But this sort of dynamic is peculiar, really, to the framing of "climate change" as a national or global policy issue. When people engage issues of climate-change impact in other settings, the consequences and meanings can be very different.

This is a point I have been stressing recently in advocating more attention to political decisionmaking surrounding local adaptation (here & here, e.g.), where people engage the issue as property owners or scare-resource consumers, where they people they are engaging are their neighbors, and where the language they have for sorting these issues out fits comfortably with their cultural identities. Those are conditions much more hospitable to open-minded, constructive engagement with climate science.

Well, the "business risk" setting is another that has advantages like these. Here people are again engaging the issue not as a symbolic one of significance to their identities as members of tribes or teams but as a financial one that could affect them in their capacity as economic actors. Here too there is a language for addressing the matter that all interested parties share and that doesn't evince hostility to or contempt for the identities of some.

What's more, the very appearance of this sort of engagement with the issue taking place might arguably be expected to have a positive impact on discussion of climate science in other places. It's tangible evidence that people w/ a dollar-and-cents stake & not just a political-ideological one are taking the science very seriously. That in itself supplies a resoruces that can be used, I think, to help counteract the suspicion and distrust that have poisoned the discussion environment at the national level.

Having said all this, I do think the length of the hair of the guy on the book cover might arouse suspicion in the minds of typical hierarchical individualists...

Saturday
Sep222012

What should we teach kids (& others) about cultural conflict over science? And should science education aim to "overcome" cultural cognition?

I got into an interesting email exchange with my friend Mark McCaffrey, the Programs and Policy Director at the National Center for Science Education. (One of the many things for which I'm grateful to Mark for doing is disabusing anyone of the misconception that our Nature Climate Change study on science literacy and cultural polarization implies that science literacy is irrelevant to enlightened democratic decisionmaking.)

In the course of the correspondence, Mark noted

[I]n our arena of education, one of the top three issues for teachers in "what's going to happen and what can we do about it locally?", along with "how do scientists know what they know?" and "how do I deal with controversy in the classroom about climate change?"  Bringing local context (geography and culture) obviously is imperative.

He also stated,

I'm curious, in part due to a conversation [after the recent University of Colorado conference on culture and climate change], on what role education has in shaping and helping transcend cultural frames.

As Mark’s points often do, these ones provoked a chain of reflections on my part, which included these:

A. What to teach kids (and other curious people) about the nature of cultural conflict over science

 The report on the experience & interest of the teachers is fascinating. It makes me think of what a climate scientist told me recently.  He reported that when he talks to members of the public including student audiences, one thing they want to know is why there is such much controversy; that's not what they expect to see, not what they associate with scientific understanding of an issue & they find it mysterious & puzzling. 

I had two reactions:

(1) It's amazing -- inspiring, even -- to see that citizens are curious about this phenomenon, that they want to understand it; they (or some at least) notice and have the same reaction to this peculiar social phenomenon that they (or some) have to an intricate or surprising natural one. That sensibility is one of the most distinctive and admirable characteristics of our species; the commitment to giving people the resources to satisfy this sort of interest -- the education in science, certainly, to be able to comprehend the sort of knowledge that exists but also ready & reliable access to whatever knowledge has been amassed -- is one of the signatures qualities of a good society.

If in fact people -- including high school students (or maybe even younger ones) -- have this reaction to public conflict over science, then I think it would be very very worthwhile to figure out how to give them the resources a curious and intelligent citizen could use to participate in whatever collective knowledge we have about it. Certainly, I'd be happy to give advice to any science educator who thought this was a worthy objective. That's not the sort of education I do, really, but if someone who does do it wanted to have someone to work with who could try to help him or her identify what to try to make comprehensible to people, I'd be delighted to help. 

(2) When I heard this report -- of the citizens (including, again, high school students) who were confused about this issue, it made me think too that the chance to answer the question is itself a sort of civic opportunity to contribute to a climate for discussion that itself helps, if not to dispel the confusion, at least to ameliorate the negative impact it has on common engagement with contested science issues.

In response to the scientist who reported on the curiosity of the public to make sense of the climate-change controversy, another person in the conversation noted that there are people strongly committed to misleading members of the public and who were filling the media with misinformation.

I don't deny that but I don't think it is the aspect of the problem that it is most important or useful for these curious people to understand. What is is that the conflict about climate change is the signature of a kind of degradation of the science communication environment the quality of which is essential to the interest we all have in being able reliably to ascertain what is collectively known.

There will always be more that is collectively known (through science) than we can meaningfully comprehend (life is short, and the world complex enough to demand specialization). As a result, we have to make use of our ability to identify and properly interpret the signs around us about who knows what about what.

Ordinarily we are great at that; but sometimes something goes wrong -- maybe from deliberate efforts to confuse but often simply as a result of misadventure and miscalculation -- that creates conflict and chaos in those signs. That sort of state is something that inevitably confuses all of us; it is something we are all vulnerable to; and it is something the avoidance of which is critical to our common interests--however we feel about climate change, and however we feel about moral and social issues of the sort that inevitably divide people who enjoy freedom of thought.

We have to try to figure out how to respond to climate change as natural phenomenon, and as an issue that divides us. But we need to think more generally  about what we can do to protect our science communication environment from the sort of contamination that accounts for this peculiar and pernicious form of conflict over what we know.... 

B. Does knowing what is known by science require teaching students to "overcome" cultural cognition?

On overcoming cultural frames with education ... My reaction is "yes & no."  

Yes in the sense that I think the sort of influences associated with cultural cognition are not "all there is" -- by any means! -- to engaging scientific information, and are qualified in particular by "professional habits of mind." That is, I think part of the nature of professionalization is that it imbues in those who are subject to it a set of conceptual frameworks, a collection of reasoning skills, and also a cluster of dispositions (some reflective & conscious but others more or less automatic and even emotional) that help them reliably to engage information in the manner suited to accomplishment of the expert reasoning task at hand.

This is so for scientists; but it is true for doctors, lawyers, journalists etc. These habits of mind will usually steer professionals away from the sorts errors they might make were they to engage information through the mechanisms distinctive of cultural cognition.

But I don't think that it is feasible for everyone to attain the professional habits of mind of the expert with respect to every domain in which they will need to participate in or have access to expert knowledge. Even those who have experts' professional habits of mind in one domain will need to make use of information outside of it, in ones in which the habits of mind most suited to engaging information are different from the ones they use in theirs.

Here, then, is where I come to the "no" part of the answer. Because we must in fact participate in or apprehend what is known in domains in which we lack the substantive knowledge and habits of mind distinctive of those who produce it, we will -- all of us-- need to exercise a distinct faculty suited to ascertaining what is collectively known (one that often involves being able to identify who knows what they are talking about).

This is conjecture on my part but I am of the view that the dynamics associated with cultural cognition are integral to the operation of that faculty. We figure out what is known by accessing cues of certification that are native to affinity groups within which we are comfortable and socially competent. The groups are diverse (how could they not be in a pluralistic society?); but they are all generally *reliable* in guiding their members to an accurate understanding of what is collectively known -- by science and by other expert ways of knowing (which groups could possibly persist that failed to put their members in touch with such knowledge, which is critical, in fact, to individual well-being).

So "cultural frames" are not something to be overcome in the interest of making us able to know things; they are vital pieces of equipment that we need in order to participate in what is collectively known. The most one could do, I suppose, is replace them with something else -- but the other thing would not be professional habits of mind, since those will always be out of reach for most and in any case domain specific -- but rather some other regime of social certification.

I don't see cultural cognition as a bias, or even as a "heuristic." It is an intrinsic component of human rationality. But its reliable operation presupposes certain conditions -- what I would characterize, have characterized already in this msg, as an uncontaminated or clean "science communication environment." The goal is not to "overcome culture"; it is to protect the conditions in which culture can make the valuable -- and amazing -- contribution that it does to our being rational beings capable of acquiring knowledge through aggregated, cumulative inquiry into the workings of nature.

****

Mark responded, predictably and characeristically, with additional thoughtful comments relating to whether these sorts of ideas (which I think he himself might qualify or revise; he is the one with the professional habits of mind suited to educating people, including science educators) might be turned into concrete directives and materials relevant to science education. That would be fantastic in my view. I'd certainly be willing to help him or other science-education experts explore this possibility!