follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

Tuesday
Jul052016

Travel report: Has liberal democracy lost its power to motivate?

This is a belated postcard from stop on recent around the world (I know it was because it included both Cambridges—UK and US—with lots of stops in between) tour. . . . It reports on one of two talks I gave at the annual Breakthrough Institute Dialogue series.  This one was part of a panel on “Progress Problems,” in which the question that I and the other panelists, who included Max Roser and Lydia Powell, addressed was “why are so many of the richest and most privileged people on earth, despite reaping such extraordinary benefits,  convinced that progress is a mirage and modernity must inevitably end badly?My remarks, as best as I can recall them, were as follows (slides here) . . . .

So the question as I understand it is --

Have liberal democratic ideals lost the power to motivate the citizens of liberal democracies?

Can we summon their attention to the common challenges they face by invoking their shared commitment to self-government, civil liberties, and free markets? Or are the animating ideals of liberal democracy now themselves a source of estrangement and division that ennervate public spiritedness?

My answer to these qustions will take a dialectical form. That is, like Clint Eastwood at the Republican Convention of 2012, I will treat you to a disagreeable dialogue with myself, in which I will radically change direction at least twice.

But insofar as I will get the last word, I’m confident that I’ll ultimately come out on top in the exchange.

So to start, with . . .

                Thesis: Sure, those ideals can motivate! I’ll show you . . . .”

I’ll show you, that is, an experiment (Kahan, Jenkins-Smith et al. 2015), one in which invoking the spirit of liberal democratic institutions sharpened apprehension of, and magnified the will to address, a collective challenge –namely the one posed by human-caused global warming.

In the experiment, we measured the willingness of subjects (members of two separate nationally representatives samples, one from  American and the other from England) to engage open mindedly with a climate-change study.

A composite of two real studies (Allen et al. 2009; Solomon et al. 2009), the one featured in our experiment—call it the “Nature-Science study”—told a bleak story. Scientists, it reported, had overestimated the speed with which carbon dioxide dissipates. As a result, even if human beings were to cease generating all greenhouse gasses tomorrow, past emissions would guarantee continued increases in global temperatures along with devastating consequences—from catastrophic flooding of coastal regions, to production-ending droughts in agricultural ones—for decades to come.

We also measured our subjects cultural worldviews along the two dimensions—hierarchy-egalitarianism and individualism-communitarianism—featured in studies of the cultural cognition of risk (Kahan 2012).

As we anticipated, experiment subjects of a hierarchical, individualistic orientation—the ones most predisposed to climate skepticism—were preemptively dismissive of the Nature-Science study results.

This, however, was in a control condition, in which subjects, before grappling with the Nature-Science study, read a news article about a town meeting over a proposal to install traffic lights in the vicinity of a new residential development.

Subjects in two other conditions were assigned to read different news articles: in one, a story about how a national association of preeminent scientists had issued a statement calling for increased limits on human CO2 emissions to combat global warming; and in the other, a story about how the same association had issued a statement calling for research on geoengineering to offset the effects of past and future emissions. We labeled these the “anti-pollution” and “geoengineering” conditions.

Logically, there’s no reason why subjects assigned to either of these conditions should have formed different views on the validity of the Nature-Science study: the validity of the evidence for an asserted problem doesn’t turn on whether someone approves or disapproves of any particular solution for it.

But psychologically, the solution might well matter.

The “anti-pollution” and “geoengineering” stories embed the problem of climate change in different narratives and thus invest it with alternative social meanings (Lessig 1995).

The former narrative is about the inevitable limits on technological ingenuity and the consequences for having too long ignored them. Against the background of the “anti-pollution story,” the message of the Nature-Science story is  “game over” and “I told you so.”

Individuals of a hierarchical, individualistic cultural outlook revere commerce and industry, not just for what they do but for what they signify about human resourcefulness and the welfare-enhancing consequences of spontaneous private orderings and the stratified systems of authority that they spawn.  They are motivated, unconsciously, to resist evidence of the existence and impact of human-caused climate change precisely because they know that if society credits such evidence it will call into question the premises of their preferred way of life.

The social meanings in the anti-pollution story reinforce that perception, and hence amplify the motivation to resist the evidence.

But the message of the “geoengineering story” is very different.

We are not the stupid animal, this narrative goes, who when it reaches the top of the Malthusian curve comes crashing down ass over tincups.  We shift the f***ing curve!

Drinking your own shit, you say? No problem! Try modern sanitation & you can increase the density of cities 10 fold relative to what a bunch of tight-sphinctered naysayers once told us was the “natural limit,” enforced by the dreaded penalty of cholera outbreaks.

Well, it’s time to shift the curve again! This time with mirror-coated nanotechnology flying saucers that magically—hell, not magically; by rational intention & design—spontaneously assemble at just the right attitude to cool the atmosphere to a predetermined, geo-thermostatically determined level.

Not “game over” but more of the same!

“I told you so”? Unh uh! Try, yes we can!

Whereas the social meanings implicit in the “anti-pollution” story narrative threaten and denigrate the identity of the hierarch individualist, the meanings implicit in “geoengineering” affirm and gratify his vision of the best life and its prospects

The result is an abatement of the unconscious, reflexive resistance to evidence that there is in fact a problem to be addressed—by one means or another.

In any case, that was the conjecture we wanted to test in our experiment.

And it was the result that we in fact observed.

Relative to the control condition, hierarchical individualists in assigned to read the “anti-pollution” news story first became even more skeptical, even more dismissive of the validity of the Nature-Science study, increasing polarization within the study sample.

But those who read the “geoengineering” story first were decidedly more receptive to the evidence in the Nature-Science study; they didn’t dismiss its findings out of hand. 

As a result, polarization, over the validity of the study and over the reality of human-caused climate change, both decreased.

So there you go!

The ideals of liberal democracy include the  confidence that people have that technology, human ingenuity, private orderings, and individual strivings can in the course of freeing us from the limits of nature, devise effective solutions for problems of their own making.

Invoking these ideas, narratively, can inspire, can summon attention to common problems and the will to address them!

I’ve shown you!

 Antithesis: “No they can't! Take a closer look.”

Hold on.

Sorry.

What you’ve shown us is that liberal democratic ideals can’t genuinely motivate the citizens of liberal democratic regimes.  Just take a closer look at your own data, and you’ll see.

Yes, relative to their counterparts in the “antipollution” condition subjects in the “geoengineering” one became more open-minded about climate change.

But egalitarian communitarian subjects—people of the sort who normally are “climate concerned”—became less so.

They are the citizens who bridle at the self-centered acquisitiveness implicit in market institutions and in liberal conceptions of individual rights.

For them, the meanings of boundless individual ingenuity and permanent technological progress that pervade the narrative implicit in the “geoengineering” condition threatened and denigrate their identity.

If the meaning of climate change is “yes, we can” and “more of the same!,” then they want none of it.

Or least they want less.  Things aren’t that bad, egalitarian communitarian subjects assigned to the “geoengineering” condition said after reading the Nature-Science study.  We can still make “progress” by shutting down industry, turning off modern agricultural techniques, and simply retreating into a pre-modern style of economic life.

The scientists who wrote this study are biased, are relying on unproven computer models, are furnishing us with evidence that it would be precipitous to use in policymaking without a lot more corroboration etc etc.

Sound familiar? These are the tropes of skepticism—now from the mouths of those most inclined to be embrace climate change.

Why? Because what they really care about, what motivates them, is not  “evidence” (they definitely lack the science literacy to understand it) but the social meaning of “I told you!” and “game over!” (“This changes everything” blah blah) that informs the default narrative on climate change.

Change the narrative and they change their tune.

Just look at your data: They show that, relative to their counterparts in the anti-pollution condition, egalitarian communitarians became skeptical about climate change science in the in the geoengineering condition.

That, plus the greater receptivity to the Nature-Science study data on the part of hierarch individualists, was why there was less polarization in the “geoengineering” condition!

The motivation that invoking liberal democratic ideals, in the form of narratives of limitless technological progress and the self-corrective, self-redemptive power of private orderings and markets, is offset by the resistance that doing so motivates in that portion of or factionalized body politic that has come to despise individual striving, technology, and markets.

You can’t inspire with these ideals!

Invoking them on behalf of one cause of another is a zero sum game. 

Synthesis: “Liberal democratic ideals can indeed inspire--if you just stop obsessively looking at them.

I’m sorry.

You—both of you—are just playing a game.

What is this angst over the loss of the inspirational force of liberal democratic ideals, private markets included?  I mean really, what are you talking about?

Or better why are you focusing so much on talk—by such a small, small group of people who bother to theorize about these things?

Just look a tthe behavior of people—hierarch individualists, egalitarian communitarians, demoKrats/RepubliKans, “liberals,” “conservatives” or whatever.

Ambivalence about technology? Disaffection with consumption?

Ask Apple or Netflix or Amazon if that’s what their bottom lines tell them.

Yes your buddy’s new “environmental studies major” girlfriend is telling you about how environmentally destructive new information technologies are. But she’s showing you on her IPad, which has a “Green Party for Bernie Sander’s” IPad “skin”!

Against capitalism, Naomi Klein? Seriously? (Any chance you’ll show us your tax returns?)

Look.  This s a fashion statement:

And so is this:

Just as these opinions are:

 

Now here’s a worldview:

It’s real.

It’s anti-liberal.

It’s anti-market.

It’s anti-democratic.

And it’s just not on the table, in this society

No one around here finds this genuine repudiation of liberal democratic ideals the least bit inspiring.

The only thing that’s on the table here are the tokens of a demeaning, petty symbolic status competition driven by intellectually juvenile, self-promoting conflict entrepreneurs. . . . 

(Actually, you two guys both have garishly expensive but ridiculously dated sensibilities about fashion; in-your-face black is so 1999! You ain't no Johnny Cash, that's for sure.)

So don’t play that stupid game.

Stop looking & looking & looking at it.

“Messaging”/arguing liberal democracy doesn’t motivate people.

Living it does.


References:

Allen, M.R., Frame, D.J., Huntingford, C., Jones, C.D., Lowe, J.A., Meinshausen, M. & Meinshausen, N. Warming caused by cumulative carbon emissions towards the trillionth tonne. Nature 458, 1163-1166 (2009).

Kahan, D.M., Hank, J.-S., Tarantola, T., Silva, C. & Braman, D. Geoengineering and Climate Change Polarization: Testing a Two-Channel Model of Science Communication. Annals of the American Academy of Political and Social Science 658, 192-222 (2015).

Lessig, L. The Regulation of Social Meaning. U. Chi. L. Rev. 62, 943-1045 (1995).

 Solomon, S., Plattner, G.-K., Knutti, R. & Friedlingstein, P. Irreversible climate change due to carbon dioxide emissions. Proceedings of the National Academy of Sciences 106, 1704-1709 (2009).

 

 

Sunday
Jul032016

Weekend update: Scarier than Nanotechnology? Episode # 532

ROBOMANDER!

He or she not only emulates the facial expression of a real salamander but also "slithers [locomotes] just like the real thing"!

So is it cute or scary?

Will it disgust those who fear guns or those who fear drones?!

Only time will tell ... & only time will tell whether it grows/morphs into . . .

ROBOFROG!

 

But for now, keep locomoting, little fellah!

 

Friday
Jul012016

The "Science Communication Problem": Two (of four) false starts

From On the Sources of Ordinary Science Knowledge--and Exraordinary Science Ignorance (in press):

2.2. A related false start blames scientists [for the Science Communication Problem--the failure of valid, compelling, widely disseminated science to disepel public disagreement over policy-relevant facts].

If members of the public aren’t converging on some policy-relevant facts despite the clarity of the evidence, the reason must be that scientists are failing to convey the evidence clearly enough (e.g., Brownell, Price & Steinman 2013). Or maybe they are speaking out too clearly, crossing the line from factfinder to policy advocate in a manner that compromises their credibility. Or perhaps what is compromising their credibility is how cagily they are hiding their advocacy by implausibly asserting that the  facts uniquely determine particular policy outcomes (e.g., Fischoff 2007).

While one can make a compelling normative case for either clearer (Olson 2009; Dean 2009) or less opinionated speech (Lempert, Groves & Fischbach 2013) by scientists, the idea that how scientists talk is the cause of the Science Communication Problem is palpably unconvincing. 

Again, all one has to do is look at science issues that don’t provoke persistent controversy. How about raw milk (Sci., Media, & Public Res. Group 2016)? Is there some reason to believe biologists have been doing a better job explaining pasteurization than climate scientists have been doing explaining the greenhouse effect? What folksy idioms or tropes did the former use that were so effective in quieting political polarization?  Or was it that they just were more genuinely neutral on whether people should drink their milk straight up from the cow’s udder?

Here, obviously, I’m relying on a pile of rhetorical questions in lieu of evidence. But the absence of evidence is my evidence.

No one has ever thought it worthwhile to “regress” the difference in public acceptance of, say, the scientific consensus on the dangers of ozone depletion and the scientific consensus on human-caused climate change on the clarity and policy-neutrality of the National Academy of Sciences’ respective reports on those issues (e.g, National Research Council 1976, 1982, 2008, 2011); or the difference between how rapidly and near-universally states adopted the proposed addition of the adolescent HBV vaccinationand how persistently they have resisted adoption of the HPV vaccine  (Kahan 2013) on the clarity and policy-neutrality of the American Academy of Pediatrics’ endorsements of both (American Academy of Pediatrics 1992, 2007).  

Likely no one has because it’s clear to the naked ear that what these groups of scientists had to say on the uncontested members of these societal-risk pairs was no less obscure and no less opinionated than what they had to say about the contested members of them. But whatever the source of the omission, the inclusion of only contested cases in the “sample” necessarily defeats any valid inference from the “obscurity” or “partisanship” of how scientists speak to why any particular policy-relevant fact is affected by the Science Communication Problem.

2.3.  Another false-start account of the Science Communication Problem attributes it to growing resistance to the authority of science itself. Along with widespread disbelief in evolution, political conflict over global warming or other issues is variously depicted as evidence of either the “anti-science” sensibilities of a particular segment of the public or of a creeping anti-science strain in American culture generally (e.g., Frank 2013). 

Anyone who manages to divert his gaze from the Science Communication Problem for even an instant is sure to spy evidence massively out of keeping with this account. In its biennial Science Indicators series, for example, the National Science Foundation  (2016) includes survey measures that consistently evince effusive degrees of confidence in and support for science (Figure 2). These levels of support do not vary meaningfully across groups defined by their political outlooks or degrees of religiosity (Figure 3). Indeed, the levels of support are so high that it would be impossible for them to harbor practically significant levels of variance across groups of any substantial size.

For behavioral validation of these sensibilities, all one has to do is look up from one’s desk (away from one’s monitor) to see the care-free confidence individuals evince in science when making decisions both mundane (the ingestion of a pill to preempt hair loss) and vital (submission to radiation therapy for cancer).

 

Because this evidence is so obvious, it’s less likely proponents of the “age of denial” thesis don’t see it than that they see it as irrelevant. On this view, confusion over or outright rejection of the admittedly authoritative evidence that science has collected on human-caused climate change or human evolution just is evidence of a deficit in the cultural authority of science.

Fine. But at that point what started out as an explanation for the Science Communication Problem has transmuted, ironically, into a piece of evidence-impervious dogma that rules out contrary proof  by definitional fiat. 

References

American Academy of Pediatrics, Committee on Infectious Diseases. Universal hepatitis B immunization. Pediatrics 8, 795-800 (1992).

American Academy of Pediatrics. HPV Vaccine Does Not Lead to Increased Sexual Activity (2012), http://tinyurl.com/jrjx37o

American Academy of Pediatrics. Prevention of Human Papillomavirus Infection: Provisional Recommendations for Immunization of Girls and Women With Quadrivalent Human Papillomavirus Vaccine. Pediatrics 120, 666-668 (2007).

Brownell, S.E., Price, J.V. & Steinman, L. Science communication to the general public: why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. J. Undergraduate Neuroscience Educ. 12, E6 (2013).

Dean, C. Am I making myself clear? : a scientist's guide to talking to the public (Harvard University Press, Cambridge, Mass., 2009).

Fischhoff, B. Nonpersuasive Communication about Matters of Greatest Urgency: Climate Change. Environmental Science & Technology 41, 7204-7208 (2007).

Frank, A. Welcome to the age of denial. N.Y. Times (Aug. 21, 2013), available at http://www.nytimes.com/2013/08/22/opinion/welcome-to-the-age-of-denial.html?_r=0.

“Green Goo: Nanotechnology Comes Alive.” ETCgroup.org (Feb. 2003), at http://www.etcgroup.org/content/green-goo-nanotechnology-comes-alive.

Lempert, R.J., Groves, D.G. & Fischbach, J.R. Is it Ethical to Use a Single Probability Density Function? (Santa Monica, CA: RAND Corporation, 2013). Available at.

National Research Council (U.S.). Board on Atmospheric Sciences and Climate. & National Research Council (U.S.). Committee on Stabilization Targets for Atmospheric Greenhouse Gas Concentrations. Climate stabilization targets : emissions, concentrations, and impacts over decades to millennia (National Academies Press, Washington, D.C., 2011).

National Research Council (U.S.). Committee on Chemistry and Physics of Ozone Depletion. Causes and effects of stratospheric ozone reduction, an update : a report (National Academy Press, Washington, D.C., 1982).

National Research Council (U.S.). Committee on Ecological impacts of Climate Change. Ecological impacts of climate change (National Academies Press, Washington, D.C., 2008).

National Research Council (U.S.). Panel on Atmospheric Chemistry. Halocarbons, environmental effects of chlorofluoromethane release (National Academy of Sciences, Washington, 1976).

National Science Foundation. Science and Engineering Indicators 2016 (Wash. D.C., 2016).

Olson, R. Don't be such a scientist : talking substance in an age of style (Island Press, Washington, DC, 2009).

Science, Media, and the Public Research Group (SCIMEP).. Exploring Public Opinion and Risk Perceptions of Food in Wisconsin. University of Wisconsin-Madison. Madison, WI: Department of Life  Sciences Communication (2016). Available from http://scimep.wisc.edu/projects/reports/

Vance, M.E., Kuiken, T., Vejerano, E.P., McGinnis, S.P., Hochella, M.F., Jr., Rejeski, D. & Hull, M.S. Nanotechnology in the real world: Redeveloping the nanomaterial consumer products inventory. Beilstei

Thursday
Jun232016

Travel report: Is even the vote (not to mention all the voting) on Brexit irrational?...

So I was in Cambridge for the outstanding 7th Annual Risk Studies Summit  at Cambridge University's world-class Centre for Risk Studies this week (now am in San Fran for a couple).

Will write a separate report on that event.  But today on this Brexit business ...

Boy were folks in Cambridge depressed!

People there at least seem to be of one mind on the issue --

But what really seems to bug them is that the issue us up for a vote, and it looks like it will be close (we'll find out presently if so).  How can it be that so many people don't see things the way we do?, they are asking in consternation; what does that mean about the competence of our democratic electorate?

Sound familiar?

Well here's something familiar too:

The Online Privacy Foundation adapted the CCP Motivated Numeracy study to views on Brexit and in a study released yesterday observed the same perverse interactions between predispositions and reasoning proficiency.

I haven't had time to do more the casually peruse the results. But my sense is that there was in fact a finding of symmetry in the effect-- that is, that higher numeracy magnified biased information processing regardless of the subjects' Brexit predispositions.

That doesn't measn that the occasion of the vote isn't entangdeled in reason-effacing pathologies-- on the contary, reason is being effaced-- but it does mean (if I'm right aout the results; I reserve the right to reassess on closer examination & will tell you if I do change my view on this) that only one side is being affected by them.

They have some pretty nice graphis, too, I'll say that:

But that's all I have time for for now!  Others might want to chime in on what they think of the OPF study or generally about Brexit and rationality.

Of course, I'll be tuning in along w/ the 14 billion readers of this blog to get the reasults of the vote later today ...

 

Monday
Jun202016

Travel report: On unpolluted & polluted public health science communcation environments--the cases of the HBV & HPV vaccines (presentation summary & slides)

A rational reconstruction of the talk I gave—in 15' 22" [I can talk 9x faster than the average man, woman, or trained circus animal can read; and I pride myself on the 30-min sentence]—at the truly amazing “How we Can Improve Health Science Communication Conference” at U of Mich.’s amazing Center for Pol. Studies last week. Pretty sure the talks will be or perhaps already are on line.. . . Now am in UK for 7th Annual Cambridge Centre for Risks Studies Confernce – will write a postcard on that soon! Slides for  the U of M talk here.

1. As you know, my paper (in press) is on what I call the Science Communication Problem—the failure of valid, compelling, widely available scientific evidence to quiet public dispute over risks and like facts to which that evidence directly speaks.

The paper argues that our understanding of the Science Communication Problem is being distorted by fixation on conspicuous and specular instances of it—particularly  the conflict over climate change. 

Obviously,  empirical researchers should be focusing on how to decipher and ultimately dispel the Science Communication Problem. My claim, however, is that we won’t achieve these goals if we focus on instances of public dissensus to the near-total disregard of public consensus, which is far and away the norm on decision-relevant science.

 A research program that never diverts its gaze from climate change and other instances of the Science Communication Problem distracts our attention from evidence that would reveal the falsity of many popular accounts of why we have the Problem. It also steers us toward prescriptions that won’t repair the dynamics that ordinarily generate public convergence on the best available evidence and could even, perversely, inflict even more damage upon them.

I won’t rehearse my argument in detail, though. Instead I will try to illustrate it with a specific example not discussed in the paper: public conflict over the HPV vaccine.

2. As I’m sure y’all know, the HPV vaccine confers an imperfect but still important degree of immunity to the human papillomavirus, an extremely common sexually transmitted disease that causes cervical cancer. 

The HPV vaccine also has the distinction of being the only childhood shot recommended for universal administration by the CDC that is not now on the schedule of mandatory school-enrollment immunizations in US states.  Legislative proposals to add it were defeated in dozens of states in the years from 2007 to 2012 as a result of deep, pervasive political controversy over the safety and effectiveness of the vaccine (Kahan 2013).

It’s tempting to think this outcome was inevitable. The vaccine is for a sexually transmitted disease and was to be administered, initially, to pre-pubescent girls as a condition of their eligibility to attend public schools.  Of course, such a proposal would provoke controversy between groups that subscribe to opposing understandings of sexual morality, of parental sovereignty, and of role of the state in securing individual well-being.

But that conclusion—that the HPV-vaccine conflict was inevitable—reflects exactly the tunnel vision I’m attacking.

The HPV vaccine was not the first one that was aimed at a sexually transmitted disease recommended for universal administration to children. The HBV vaccine was.

The HBV vaccine confers immunity to hepatitis-b, which also causes cancer, of the liver.

The CDC proposed it be administered universally to adolescents (now to infants) just a few years before it proposed the  same for the HPV vaccine. With no significant controversy, the HBV vaccine was incorporated into the mandatory, school-enrollment immunization lists of nearly every U.S. state in a wave of approvals that crested just as the HPV-vaccine controversy began. At the time the HPV-vaccine controversy was raging, the HBV vaccine had an national uptake rate of over 90%--compared to the anemic 30% for the HPV Vaccine today (Kahan 2013).

Thus, the HPB vaccine,  is in the “denominator”—the vast class of decision-relevant science issues on which there isn’t public controversy but could be. What it shares with all the other members of that class is the benefit of having become known to the public in an unpolluted science communication environment

The science communication environment, I explain in the paper, consists of the sum total of processes and conventions generative of the cues that normally guide diverse individuals align their behavior with the best available evidence.

The HBV vaccine, like every universal childhood immunization before it, traveled safely through these processes and conventions to the destination of overwhelming public confidence. The vaccine was considered and approved for inclusion in state universal-immunization schedules by non-political public health agencies that have been delegated this expert task by state legislatures. The vast majority of parents thus had occasion to learn of the vaccine for the first time when their consent to administer it was sought from their pediatricians, individuals they had selected b/c they trusted them, who advised the vaccine was safe and a useful addition to the array of prophylactic practices that keep children healthy. Just as important, regardless of who they were—republican or democrat, devout evangelical or atheist or agonistic—all were afforded ample evidence that parents just like them were getting their kids vaccinated for HBV (Kahan 2013).

The decision to follow suit was a no brainer!

In contrast, parents and other citizens learned about the HPV vaccine in what I characterized as a polluted science communication environment.  A polluted science communication environment is one in which some risk or fact has become entangled in antagonistic social meanings that transform them into badges of membership in and loyalty to opposing cultural groups. In those conditions, the same cues that normally guide diverse citizens into convergence on the best available evidence—including what others in their situation are doing and saying-- instead drive them apart.

That’s what happened with the HPV vaccine. To try to establish a dominant position in the market before the approval of a competing HPV vaccine manufactured by its rival Smith-Glaxo Smithkline, Merck--manufacturer of the Gardasil, the HPV shot approved by the FDA in 2006--orchestrated a nationwide campaign to add the vaccine to the state, mandatory school-enrollment schedules by statutes enacted by state legislatures.

What was normally a nonpolitical decision—the updating the of state school-enrollment immunization lists—necessarily became hyper-politicized. People first learned of the vaccine not from their pediatricians but from Fox News, MSNBC, and other political news outlets, who hyped the repressive-in-your-face-religious right vs. the cosmopolitan, communism-of-women-and-children-left showdown on the “STD shot for school girls,” a framing facilitated by Merck’s decision to seek fast-track FDA approval of a girl’s only shot as part of its market-driven plan to sidestep the slower, less politicized approval process.

The result was the entanglement of the HPV vaccine in the sort of antagonistic meanings productive of the most debilitating of all known science-communication pathologies—identity-protective cognition (Kahan 2013).

3. Sarah Gollust and her collaborators (2010, 2013, 2014, 2015) have done a lot of outstanding work to identify and quantify the indicators of this entanglement and its disruptive impact on how ordinary members of the public ordinarily recognize valid science..

The CCP research group did a study on this too back in 2007, just as the process that resulted in this disaster began to unfold. In it we tried to model how different “science communication environments”—unpolluted and polluted—could affect engagement with information on the vaccine’s risks and benefits.

The study (Kahan, Braman, Cohen, Gastil & Slovic 2010) examined how cultural cognition could shape perceptions of the HPV vaccine. Cultural cognition refers to the tendency of people, in effect, to conform their  own perceptions of risk and like facts to the ones that predominate among others who share their cultural identities.

We measured individuals cultural identities with two orthogonal attitudinal scales, hierarchy-egalitarianism and individualism-communitarianism, which can be viewed as forming four types of cultural “affinity groups.”

Next, we exposed them to competing arguments on the balance of risks and benefits of the HPV vaccine from  fictional “public health experts.” The experts were ones we had determined in separate pretests would be tacitly identified by the experimental subjects as having the cultural identifies featured by the cultural cognition worldview scheme.

By crossing the two arguments with the four advocates, we had had 12 HPV “expert-argument matchups.” To assess their impact, we modeled how the proximity of the subjects’ actual cultural outlooks to the experts' tacitly perceived ones affected subjects’ HPV-vaccine risk perceptions.

Our goal  in simultaneously manipulating the array of experts and arguments, on the one hand, and the proximity of the experts' cultural outlooks to the subjects', on th the other, was to simulate the impact of learning about the vaccine under conditions that would themselves vary in how readily they suggested the presence or absence of division of opinion between subjects’ own cultural groups and a rival one.

The simulation suggested that the impact was yuuuuuuuugely consequential.

In effect, there was a continuous polarization effect that tracked subjects' tacit impressions of group conflict.  

We knew, again  from pretesting, that subjects with particular identities had modest predispositions to form one or another impression of the safety and efficacy of the vaccine.

But under the condition least likely to suggest group conflict—the one in which subjects saw an alignment of culturally identifiable experts and arguments contrary to the one they would have expected to see if the issue were in fact dividing groups consistent with subjects’ own predispositions—polarization essentially disappeared.

Where in contrast they saw the alignment most suggestive of such conflict—the one in which an expert with their identity took “their side’s” position and one with opposing identity “the other side’s”—polarization dramatically escalated relative to the level predicted by the subjects’ predispositions alone.

These two points on the continua reflect a pristine and a polluted science communication environment, respectively. The first was the environment in which American parents learned of the HBV vaccine; the latter the one in which they learned of the proposal to add the HPV vaccine to the schedule of mandatory school-enrollment immunizations.

4. Likely this condition could have been avoided. Lots of physicians and others were worried that the manner in which the HPV vaccine was being introduced to the public risked generating a political controversy (Kahan 2013).

But the question now is whether anyone is going to learn from this experience and from research on it, including our study and the penetrating set by Gollust and her collaborators.

The answer, I think, will depend largely on whether members of the public health establishment avoid the mistake of “ignoring the denominator”—the relatively large number of cases in which the public doesn’t polarize but rather converge on the best available scientific evidence. Frankly, I think many of the proposals on how to over come the continuing public ambivalence on the HPV vaccine reflect exactly that mistake.

One prominent proposal is to conduct a large-scale social marketing campaign promoting the vaccine. Thrusting the HPV vaccine back into the limelight in this way would risk exciting the very sorts of sensibilities—and more importantly reigniting the same sort of interest group activity —that bred the initial conflict. Indeed, this idea sounds more or less like a proposal to take out of mothballs the very advertisements that Merck bankrolled during its disastrous campaign to secure legislative mandates.

Just look at the denominator!

There wasn’t any social marketing campaign on HBV vaccine, just as there wasn’t any on the myriad other science issues—from medical x-rays to nanotechnology—on which diverse members of the public now have aligned their behavior appropriately by science.

The mechanism, moreover, in those cases hasn't been the public's reflective processing of detailed bits of medical or other scientific information. It has been their attention to the cue emitted by the words and behavior of others who have evinced their confidence  by words and deeds showing that they have confidence in the underlying science.

An unpolluted science communication environment is not bustiling with broadcast messages. On the contrary, it comprises a host of persitent low key signals that assure that people that doing things that rely on what is in fact the best available evidence is mundane, banal normal.

The question is how to promote this sort of normality to people's engagement with the HPV vaccine.

I’ll give you a hint on the answer.

The one state, Rhode Island, that has adopted an HPV-vaccine school-enrollment mandate in the years since the initial political firestorm over this proposal abated did so without particular fanfare—by resort to the nonpolitical administrative process that is  actually the norm for updating state mandatory vaccination regimes. 

Parents in RI aren't now learning about the HPV vaccine from media reports on a contested legislative mandate for an STD shot for their pre-teen children; they didn’t learn about it from a weird and very likely counterproductive (Nyhan et al. 2014) social marketing campaign. 

Rather they are getting the information in the normal way—from talking to their pediatricians about it at the same time they discuss other immunizations that their children are required to get, and from seeing that other parents just like them, after having done the same, are making decisions to get their kids vaccinated for HPV—just as they are making the decision (at rates well over 90%) to do the same for HBV, MMR and all the other childhood diseases from which their kids and lots of others too are protected by universal immunizations.

It's a no brainer!

It's no big deal.

Refs

Gollust, S.E. & Cappella, J.N. Understanding public resistance to messages about health disparities. Journal of health communication 19, 493-510 (2014).

Gollust, S.E., Attanasio, L., Dempsey, A., Benson, A.M. & Fowler, E.F. Political and News Media Factors Shaping Public Awareness of the HPV Vaccine. Women's Health Issues 23, e143-e151 (2013).

Gollust, S.E., Dempsey, A.F., Lantz, P.M., Ubel, P.A. & Fowler, E.F. Controversy undermines support for state mandates on the human papillomavirus vaccine. Health Affair 29, 2041-2046 (2010).

Gollust, S.E., LoRusso, S.M., Nagler, R.H. & Fowler, E.F. Understanding the role of the news media in HPV vaccine uptake in the United States: Synthesis and commentary. Human vaccines & immunotherapeutics, 1-5 (2015).

Kahan, D., Braman, D., Cohen, G., Gastil, J. & Slovic, P. Who Fears the HPV Vaccine, Who Doesn’t, and Why? An Experimental Study of the Mechanisms of Cultural Cognition. Law Human Behav 34, 501-516 (2010).

Kahan, D.M. A Risky Science Communication Environment for Vaccines. Science 342, 53-54 (2013).

Kahan, D.M. On the sources of ordinary science intelligence and ignorance. Oxford Handbook on the Science of Science Communicatoin (in press).

Nyhan, B., Reifler, J., Richey, S. & Freed, G.L. Effective messages in vaccine promotion: A Randomized Trial. Pediatrics 133, e835-e842 (2014).

Thursday
Jun162016

The fourth of "four theses on ordinary science intelligence" ... a fragment

I posted the first two "yesterday"; if you want to read the third, then just download   On the Sources of Ordinary Science Knowledge and Ignorance . . .

IV. “The recognition problem [that generates conflict over decision relevant science] is a polluted science communication environment.”

The species of pattern recognition that ordinary members of the public normally use to recognize valid science enables them to get the benefit of substantially more scientific insight than any could possibly hope to genuinely comprehend.  The evidence I described in the last section, however, evinces the disablement of this critical capacity.  The final “thesis of ordinary science knowledge” identifies the source of this disablement: a polluted science communication environment.

Popper (1962b), I noted, attributes the acquisition and exercise of the capacity for science-recognition to immersion in a set of social processes and conventions. When I refer to the science communication environment, I mean to refer to the sum total of the processes and conventions that enable recognition of valid science in this way (Kahan 2015b). Any influence that impairs or impedes the operation of these social practices will necessarily degrade the power of free, reasoning citizens to recognize valid science, and hence to realize the full benefits of it. As a result, we may understand any such influence to be a form of pollution in the science communication environment.

The sorts of influences that can generate such disablement are no doubt numerous and diverse. But I will focus on one, which degrades an especially consequential cue of science validity.

Of all the sources of ordinary science knowledge, by far the most significant will be individuals’ interactions with others with whom they a share cultural commitments or basic understanding of the best way to live. The suggestion that direct communication with scientists is more consequential reflects either the First or Second False Start or both: individuals have neither the time nor the capacity to extract information directly from scientists.  Much more accessible, and much more readily subject to meaningful interpretation, are words and actions of other ordinary people, whose use of DRS vouches for their confidence in it as a basis for decision. 

Indeed, it vouches as effectively when nothing is said about it as it does when something is. Nothing—including a new National Academy of Sciences expert consensus report (National Research Council 2016) that few members of the public will even be dimly aware exists—will assure an ordinary person that it is safe to eat GM corn chips as will watching his best friend and his brother-in-law and his officemate eating them without giving the matter a second’s thought, the “all clear” signal that obviates the need for the vast majority of Americans even to bother learning that the corn chips they are eating contain GM foods (Hallman, Cuite & Morin 2013).

Of course, ordinary citizens don’t interact only with those with whom they share important cultural commitments. But they interact with them much more than they interact with others, for the simple reason that they find their company more congenial and more productive of all manner of profitable intercourse. They are also less likely to was time squabbling with these people, and can also read them more reliably, distinguishing who really does know what science knows and who is only a poser. It is perfectly rational for them consciously to seek out guidance from such individuals, then, or to form unconscious habits of mind that privilege them as sources of guidance on what science knows (Kahan 2015b).

Figure 5 ... click on it: it will increase your proportion of fast-twitch to slow-twich mucle fibers by 23.6%!This process is admittedly insular, but it clearly works in the main. All of the major cultural groups in which this process operate are amply stocked both with members high in science comprehension and with intact social processes for transmitting what they know. No group that lacked these qualities—and that as a result regularly misled its members on the content of valid DRS—would last very long! On issues that don’t display the profile of the Science Communication Paradox, moreover, individuals highest in science proficiency do tend to converge on the best available evidence, and no doubt pull the other members of their groups along in their wake (Figure 5).

But such a system is vulnerable to a distinctive pathology: identity-protective cognition (IPC). IPC occurs when a policy-relevant fact that admits of empirical inquiry becomes entangled in antagonistic social meanings that transform positions on them into badges of identity in, and loyalty to, competing cultural groups (Kahan 2010, 2012). The cost under those conditions of forming factually incorrect beliefs on matters like whether humans are heating up the earth or whether fracking will extinguish or contaminate drinking water sources is essentially zero: individuals’ personal views and actions are not consequential enough to affect the level of risk they face, or the likely adoption of ameliorating (or simply pointless or even perverse) regulatory responses. But given what beliefs on these subjects (correct or incorrect) have come to signify about the kind of person one is—about whose side on is on, in what has become a struggle for status among competing cultural groups—the personal cost of forming the wrong ones in relation to one’s own cultural identity could be high indeed (Kahan, Peters et al. 2012).

In such circumstances, individuals can be expected to use their reason to form and persist in beliefs that reliably vouch for their group identities regardless of whether those beliefs are factually accurate.  This conclusion is consistent with numerous studies, observational (Bolsen & Druckman 2015; Bolsen, Druckman & Cook 2014; Gollust, LaRussao et al. 2015; Gollust, Dempsey, et al. 2010) and experimental (Kahan, Braman, et al. 2009, 2010), that link IPC to the Science Communication Problem’s signature forms of polarization.  Indeed, individuals who enjoy the highest level of proficiency will display this form of motivated reasoning to the greatest extent, precisely because they will be the most adept at using their reasoning proficiency secure the interest that they share to form identity-expressive beliefs (Kahan in press).

In sum, the antagonistic social meanings that trigger IPC are a toxic form of pollution in the science communication environment of culturally pluralistic societies.  They disable individuals’ science-recognition capacities, not by degrading their reason but by conscripting it into the service of advancing their group’s cause in a demeaning form of cultural status competition. IPC does not create the role that social influences play in popular recognition of what science knows. Rather it corrupts them, transforming the role that  spontaneous, everyday social interactions play from an engine of convergence on the beset available evidence into a relentlessly aggressive agent of public dissensus over what scientific consensus really is. 

References

Bolsen, T. & Druckman, J.N. Counteracting the Politicization of Science. Journal of Communication 65, 745-769 (2015).

Bolsen, T., Druckman, J.N. & Cook, F.L. The influence of partisan motivated reasoning on public opinion. Polit Behav 36, 235-262 (2014).

Gollust, S.E., Dempsey, A.F., Lantz, P.M., Ubel, P.A. & Fowler, E.F. Controversy undermines support for state mandates on the human papillomavirus vaccine. Health Affair 29, 2041-2046 (2010).

Gollust, S.E., LoRusso, S.M., Nagler, R.H. & Fowler, E.F. Understanding the role of the news media in HPV vaccine uptake in the United States: Synthesis and commentary. Human vaccines & immunotherapeutics, 1-5 (2015).

Hallman, W., Cuite, C. & Morin, X. Public Perceptions of Labeling Genetically Modified Foods. Rutgers School of Environ. Sci. Working Paper 2013-2001, available at http://humeco.rutgers.edu/documents_PDF/news/GMlabelingperceptions.pdf.

Kahan, D. Fixing the Communications Failure. Nature 463, 296-297 (2010).

Kahan, D. Why we are poles apart on climate change. Nature 488, 255 (2012).

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

National Research Council (U.S.). Committee on Genetically Engineered Crops. Genetically Engineered Crops: Experiences and Prospects  (National Academies Press, Washington, D.C., 2016).

Popper, K.R. On the Sources of Knowledge and of Ignorance. in Conjectures and Refutations 3-40 (Oxford University Press London, 1962b).

 

 

Tuesday
Jun142016

Two of "Four Theses on Ordinary Science Knowledge" . . . a fragment

From On the Sources of Ordinary Science Knowledge and Ignorance . . .

I. “Individuals must accept as known more decision relevant science (DRS) than they can possibly understand or verify for themselves.”

The motto of the Royal Society is Nullius in verba, which translates literally into “take no one’s word for it.” But something—namely, any pretense of being a helpful guide to getting the benefits of scientific knowledge—is definitely lost in a translation that literal.

If you aren’t nodding your head violently up and down, then consider this possibility. You learn next week that you have endocrinological deficit that can be effectively treated but only if you submit to a regimen of daily medications.  You certainly will do enough research to satisfy yourself—to satisfy any reasonable person in your situation—that this recommendation is sound before you undertake such treatment.

But what will you do? Will you carefully read and evaluate all the studies that inform your physician’s recommendation? If those studies refer, as they inevitably will, to previous ones the methods of which aren’t reproduced in those papers, will you read those, too? If the studies you read refer to concepts with which you aren’t familiar, or use methods which you have no current facility, will you enroll in a professional training program to acquire the necessary knowledge and skills? And once you’ve done that, will you redo the experiments—all of them; not just the ones reported in the papers that support the prescribed treatment but in any those studies relied on and extended—so you can avoid taking anyone’s word on what the results of such studies actually were as well?

Of course not. Because by the time you do those things, you’ll be dead. To live well—or just to live—individuals (including scientists) must accept much more DRS than they can ever hope to make sense of on their own.

Science’s way of knowing involves crediting as true only inferences rationally drawn from observation. This was—still is—a radical alternative to other ways of knowing that feature truths revealed by some mystic source to a privileged few, who alone enjoy the authority to certify the veracity of such insights. That system is what the founders of the Royal Society had in mind when they boldly formulated their injunction to “take no one’s word for it.” But it remains the case that to get the benefits of the distinctive, and distinctively penetrating, mode of ascertaining knowledge they devised, we must take the word of those who know what’s been ascertained by those means—while being sure not to take the word of anyone else (Shapin 1994).

II. “Individuals acquire the insights of DRS by reliably recognizing it.”

But how exactly does one do that? How do reasonable, reasoning people who need to use science for an important decision but who cannot plausibly figure out what science knows for themselves figure out who does know what science knows?

We can rule out one possibility right away: that members of the public figure out who genuinely possesses knowledge of what science knows by evaluating the correctness of what putative experts believe. To do that, members of the public would have to become experts in the relevant domain of knowledge themselves. We have already determined (by simply acknowledging the undeniable) that they lack both the capacity and time to do that.

Instead they have to become experts at something else: recognizing valid sources of science. They become experts at that, moreover, in the same way they become experts at recognizing anything else: by using a conglomeration of cues, which operate not as necessary and sufficient conditions, but as elements of prototypical representations (“cat,” “advantageous chess position,” “ice cream sandwich,” “expert”) that are summoned to mind by mental processes, largely unconscious, that rapidly assimilate the case at hand to a large inventory of prototypes acquired through experience.  In a word (or two words), they use pattern recognition (Margolis 1993).

This is equivalent to the answer that Popper gave (in an essay the title, and much more, of which are the inspiration for this one) in answering the near-identical question about how we come to know what is known by science. Popper’s target was a cultural trope of sensory empiricism that treated as “scientific knowledge” only that which one has observed for oneself. After impaling this view on the spear tips of a series of reductios, Popper explains that most things we know”—i.e., know to be known to science—“we have learnt by example, by being told.” In appraising the conformity of any such piece of information to the qualities that invest it with the status of scientific knowledge, moreover, an individual must rely on “his knowledge of persons, places, things, linguistic usages, social conventions, and so on” (ibid., p. 30).

To be sure, powers of critical reasoning play a role. We must calibrate this facility of recognition by “learning how to criticize, how to take and to accept criticism, how to respect truth” (ibid, p. 36), a view Baron (1993) and Keil (2012) both develop systematically.

But the objects of the resulting power to discern valid science are not the qualities that make it valid: those are simply far too “complex,” far too “difficult for the average person to understand (Baron, 1993, p, 193). What this faculty attends to instead are the signifiers of validity implicit in informal, everyday social processes that vouch for the good sense of relying on the relevant information in making important decisions (Keil 2010, 2012). Popper characterizes the aggregation of these processes as “tradition,” which he describes as “by far the most important source of our knowledge” (1962b, p. 36).

It is worth noting that although Popper here is referring to the process by which ordinary science knowledge disseminates to nonscientists, there is no reason to think that scientists are any less in need of a valid-knowledge recognition capacity, or that they acquire or exercise it in a fundamentally different way. Indeed, there is ample reason to think that it couldn’t possibly differ from the faculty that members of the public use to recognize valid science (Shapin 1994) aside from its being more finely calibrated to the particular insights and  methods needed to be competent in the production of the same (Margolis 1987, 1996).

 “How do we gain our knowledge about how to analyze data?” ask Andrew Gelman and Keith O’Rourke (2015, pp., 161-2).  By “informal heuristic reasoning,” they reply, of the sort that enables those immersed in a set of practice to see the correctness of an answer to a problem before, and often without ever being able to give a fully cogent account of, why.

References

Baron, J. Why Teach Thinking? An Essay. Applied Psychology 42, 191-214 (1993).

Gelman, A. & O’Rourke, K. Convincing Evidence. in Roles, Trust, and Reputation in Social Media Knowledge Markets: Theory and Methods (ed. E. Bertino & A.S. Matei) 161-165 (Springer International Publishing, Cham, 2015).

Keil, F.C. Running on Empty? How Folk Science Gets By With Less. Current Directions in Psychological Science 21, 329-334 (2012).

Keil, F.C. The feasibility of folk science. Cognitive science 34, 826-862 (2010).

Margolis, H. Patterns, thinking, and cognition : a theory of judgment (University of Chicago Press, Chicago, 1987).

Margolis, H. Paradigms and Barriers (University of Chicago Press, Chicago, 1993).

Margolis, H. Dealing with risk : why the public and the experts disagree on environmental issues (University of Chicago Press, Chicago, IL, 1996).

Popper, K.R. On the Sources of Knowledge and of Ignorance. in Conjectures and Refutations 3-40 (Oxford University Press London, 1962b). 

Shapin, S. A social history of truth : civility and science in seventeenth-century England (University of Chicago Press, Chicago, 1994).

 

 

Monday
Jun132016

On the Sources of Ordinary Science Knowledge and Ignorance (new paper)

A short little conference paper ...

 for this really cool conference:

 

First person to figure out the significance of the title before reading the paper wins a cool prize.

Wednesday
Jun082016

Slovic elected to NAS!! 

An inspiration to billions for his pathbreaking studies of public risk perceptions!

And to hundreds of millions of those same billions for his messy desk & office -- if he can overcome come that handicap, so can we dammit!

UofO issued a press statement that focuses on the messy desk. Understandable, because that's the obvious human-interest angle on the story.

But here are some other, more scholarship-focused details on Slovic's career. 

Although Slovic proved an exception to the rule that scholars of public risk perception tend to do their best work by age 15, his early hybrid lab-field study strategies remain high on his list of achievements-- and of course generated the $millions that underwrote his later, even more influential research efforts:

 "No one checked my id," Slovic explained in connection with one of his early papers, a jr-high final project on the peculiar risk perceptions of Las Vegas gamblers.

Researcher Sarah Lichtenstein, pictured here with Slovic, opened a hedge fund based on early team research insights, only a fraction of which have ever been revealed to the public. She was quoted recently as saying, "Black & Scholes were real chumps. I've made 10^6 as much money by keeping my behavioral-economics based insights into derivative pricing to myself than they got for winning the Nobel Prize in economics..."

It should also be noted that Slovic kicked UO great Steve Prefontaine's ass in a marathon once. "He burned himself out by running the first 10K in 26:59," Slovic explained. "Of course, it also helped that I supplemented my diet with reindeer milk," he added with a wink.

Slovic thereafter published the data underlying his own training regimen, which was derived from study of billions of Boston Marathon entrants.

Wednesday
Jun082016

Cognitive dualism and beliefs as "dispositions to action" ... a fragment

From something I'm working--and working and working and working--on . . .

4.1. Beliefs as action-enabling dispositions

Imagine an astrophysicist who is also a mother and a member of a particular close-knit community.  Like any other competent scientist (or at least any who examines macro- as opposed to quantum-physical processes!), she adopts a Laplacian orientation toward the objects of her professional study. The current state of the universe, she’ll tell you, is simply the penultimate state plus all the laws of nature; the penultimate state, in turn, is nothing more than the anteultimate one plus all those same laws—and so forth and so on all the way back to the big bang (Laplace 1814).  This understanding is gospel for her when she sets out to investigate one or another cosmic anomaly. She hunts for an explanation that fits this picture, for example, in trying to solve the mystery of massive black holes, the size of which defy existing known principles about the age of the universe (Armitage & Natarajan 2002). Nothing under the heavens—or above or within them—enjoys any special exemption from the all-encompassing and deterministic laws of nature.

In her personal life, however, she takes a very different view—at least of human beings. She explains—and judges—them on the assumption that they are the authors of their own actions.  Her attributes her children’s success at school, for example, to their hard work, and is filled with pride. She learns of the marital infidelity of a friend’s spouse and is outraged.

By viewing everything as determined by immutable, mechanistic laws of nature, on the one hand, and by judging people for the choices they make, on the other, is the scientist guilty of self-contradiction? Is she displaying a cognitive bias or some related defect in rationality?

Definitely not. She is using alternative forms of information-processing rationally suited to her ends.  One of her goals is to make sense of how the universe works: the view that everything, human behavior included, is subject to immutable, deterministic natural laws reliably guides her professional investigations. Another of her goals is to live a meaningful life in that part of the universe she inhabits. The form of information processing that attributes agency to persons is indispensable to her capacity to  experience the moral sensibilities integral to being a good parent and a friend.

The question whether there is a contradiction in her stances toward determinstic natural laws and self-determining people is ill-posed. As mental objects at least, these opposing stances don’t exist independently of clusters of mental states—emotions moral judgments, desires, and the like—geared to doing the things she does with them. There is no contradiction in how she is using her reason if the activities that these forms of information processing enable are themselves consistent with one another—as they surely are.

The individual in this example is engaged in cognitive dualism. That is, she is rationally applying to one and the same object—the self-determining power of human beings—alternative beliefs, and corresponding forms of information processing, suited to achieving diverse but compatible goals.

We start with this example for two reasons. One is to emphasize the lineal descent of cognitive dualism from another—the philosophical dualism of Kant (1785, 1787, 1788).  The two “beliefs” about human autonomy we attributed to the astrophysicist are the two perspectives toward the self—the phenomenal and noumenal—that Kant identified as action-enabling perspectives suited to bringing reason to bear on understanding how the world works, on the one hand, and living a meaningful life within it, on the other. Kant saw puzzling over the consistency of the self-perspectives featured by these perspectivs as obtuse because in fact the opposing orientations they embody don’t exist indepedently of the actions they enable—which clearly are fully compatible.

The other reason for starting with the astrophysicist was to remark the ubiquity of this phenomenon. The opposing perspectives that we attributed to her—of the all-encompassing status of deterministic natural laws, on the one hand, and the uniquely self-governing power of human beings, on the other—are commonplace in modern, liberal democratic societies, whose members use the opposing “beliefs” these perspectives embody to do exactly the same things the astrophysicist does with them: make sense of the world and live in it.

Our astrophysicist both does and doesn’t exist.  She’s no one in particular but is in fact everyone in general.

There’s no need to confine ourselves to composites, however.  Decision scientists, it’s true, have paid remarkably little attention to cognitive dualism, misattributing to bounded rationality forms of information processing that aren’t suited for accurate perceptions of particular facts but that are for cultivating identity-expressive affective dispositions (Kahan in press).  In other scholarly domains, however, one can find a richly elaborated chronicle of the existence and rationality of the two forms of information processing that cognitive dualism comprises.

Developmental psychologists, for example, are very familiarity with them. Children, they’ve shown, not only devote considerable cognitive effort to internalizing confidence- and trust-invoking forms of social competence. They also frequently privilege this form of information processing over ones that feature “factual accuracy.” E.g., a child will often choose to defer to an information source with whom she shares some form of social affinity over one whom she recognize has more knowledge—not because she is biased (cognitively or otherwise) but because she has assimilated the kind of decision she is making in that situation to the stake she has in forging and protecting her connections with members of a social group  (Elashi & Mills, 2014;  MacDonald, Schug, Chase & Barth 2013; Landrum, Mills, & Johnston 2013) .

Researchers have also documented the effect of cognitive dualism in studying of how people who “disbelieve” in evolution can both comprehend and use what science knows about the natural history of human beings (Long 2011). Religiously oriented students, e.g., who don’t “believe in” evolution can learn it just as readily as those who do (Lawson & Worsnop 1992). The vast majority of them will make use of that knowledge simply to pass their school exams and then have nothing more to do with it (Herman 2012); but that’s true for the vast majority of their fellow students who say they “believe” in evolution, too (Bishop & Anderson 1990). 

Some small fraction of the latter (the evolution believers) will go on to do something in their life—like become a scientist or a physician—where they will use that knowledge professionally. But so will a small fraction of the former—the students who “don’t believe in” evolution (Hameed 2015; Everhart & Hameed 2013; Hermann 2012). 

These latter individuals—let us call them “science-accepting disbelievers”—are displaying cognitive dualism.  Science-accepting disbelievers are professing—but not just professing, using—disbelief of evolution in their personal lives, where it is a component of a complex of mental states that reliably summon affective-driven behavior that signifies their commitment to a particular community.  But in addition to being people of that sort, they are or aspiring to become science professionals who use belief in evolution to achieve their ends as such (Everhart & Hameed 2013). 

When queried about the “contradiction, science-accepting disbelievers respond in a way that evinces—affectively, if not intellectually—the same attitude Kant had about the contradiction between the phenomenal and noumenal selves. That is, they various stare blankly at the interviewer, shrug their shoulders in bemusement, or explain—some patiently, other exasperatedly—that the evolution they “disbelieve in” at home and the one the “believe in” at work are, despite having the same referent, “entirely different things” because in fact they have no existence, in their lives, apart from the things that they do with them, which are indeed “entirely different” from one another (Everhart & Hameed 2013; Hermann 2012). In a word, they see the idea that there is a contradiction in their opposing states of belief and disbelief in evolution as obtuse.

References

Armitage, P.J. & Natarajan, P. Accretion during the merger of supermassive black holes. The Astrophysical Journal Letters 567, L9 (2002).

Bishop, B.A. & Anderson, C.W. Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching 27, 415-427 (1990).

Elashi, F.B. & Mills, C.M. Do children trust based on group membership or prior accuracy? The role of novel group membership in children’s trust decisions. Journal of experimental child psychology 128, 88-104 (2014).

Hameed, S. Making sense of Islamic creationism in Europe. Public Understanding of Science 24, 388-399 (2015).

Hermann, R.S. Cognitive apartheid: On the manner in which high school students understand evolution without Believing in evolution. Evo Edu Outreach 5, 619-628 (2012).

Kahan, D.M. The Expressive Rationality of Inaccurate Perceptions. Behavioral & Brain Sciences (in press).

Kant, I. & Gregor, M.J. Groundwork of the metaphysics of morals (1785).

Kant, I., Critique of pure reason (1787).

Kant, I.. Critique of practical reason (1788).

Landrum, A.R., Mills, C.M. & Johnston, A.M. When do children trust the expert? Benevolence information influences children's trust more than expertise. Developmental Science 16, 622-638 (2013).

Laplace, P. A Philosophical Essay on Probabilities (1814).

Long, D.E. Evolution and religion in American education : an ethnography (Springer, Dordrecht, 2011).

Lawson, A.E. & Worsnop, W.A. Learning about evolution and rejecting a belief in special creation: Effects of reflective reasoning skill, prior knowledge, prior belief and religious commitment. Journal of Research in Science Teaching 29, 143-166 (1992).

MacDonald, K., Schug, M., Chase, E. & Barth, H. My people, right or wrong? Minimal group membership disrupts preschoolers’ selective trust. Cognitive Development 28, 247-259 (2013).

 

Friday
Jun032016

What does "believing/disbelieving in" add to what one knows is known by science? ... a fragment

From something I'm working on (and related to "yesterday's" post) . . .

4.3. “Believing in” what one knows is known by science

People who use their reason to form identity-expressive beliefs can also use it to acquire and reveal knowledge of what science knows. A bright “evolution disbelieving” high school student intent on being admitted to an undergraduate veterinary program, for example, might readily get a perfect score on an Advanced Placement biology exam (Herman 2012).

It’s tempting, of course, to say that the “knowledge” one evinces in a standardized science test is analytically independent of one's “belief” in the propositions that one “knows.”  This claim isn’t necessarily wrong, but it is highly likely to reflect confusion.  

Imagine a test-taker who says, “I know science’s position on the natural history of human beings: that they evolved from an earlier species of animal. And I’ll tell you something else: I believe it, too.”  What exactly is added by that person’s profession of belief?

The answer “his assent to a factual proposition about the origin of our species” reflects confusion. There is no plausible psychological picture of the contents of the human mind that sees it as containing a belief registry stocked with bare empirical propositions set to “on-off,” or even probabilistic “pr=0.x,” states.  Minds consist of routines—clusters of affective orientations, conscious evaluations, desires, recollections, inferential abilities, and the like—suited for doing things.  Beliefs are elements of such clusters. They are usefully understood as action-enabling states—affective stances toward factual propositions that reliably summon the mental routine geared toward acting in some way that depends on the truth of those propositions (Peirce 1877; Braithwaite 1933, 1946; Hetherington 2011)

In the case of our imagined test-taker, a mental state answering to exactly this description contributed to his supplying the correct response to the assessment item.  If that’s the mental object the test-taker had in mind when he said, “and I believe it, too!,” then his profession of belief furnished no insight into the contents of his mind that we didn’t already have by virtue of his answering the question correctly. So “nothing” is one plausible answer to the question what did it add when he told us he “believed” in evolution.

It’s possible, though, that the statement did add something.  But for the reasons just set forth, the added information would have to relate to some additional action that is enabled by his holding such a belief. One such thing enabled by belief in evolution is being a particular kind of person.  Assent to science’s account of the natural history of human beings has a social meaning that marks a person out has holding certain sorts of attitudes and commitments; a belief in evolution reliably summons behavior evincing such assent on occasions in which a person has a stake in experiencing that identity or enabling others to discern that he does.

Indeed, for the overwhelming majority of people who believe in evolution, having that sort of identity is the only thing they are conveying to us when they profess their belief. They certainly aren’t revealing to us that they possess the mental capacities and motivations necessary to answer even a basic high-school biology exam question on evolution correctly: there is zero correlation between professions of belief and even a rudimentary understanding of random mutation, natural variance, and natural selection (Shtulman 2006; Demastes, Settlage & Good 1995; Bishop & Anderson 1990).

Precisely because one test-taker’s profession of “belief” adds nothing to any assessment of knowledge of what science knows, another's profession of “disbelief” doesn’t subtract anything.  One who correctly answers the exam question has evinced not only knowledge but also her possession of the mental capacities and motivations necessary to convey such knowledge

When a test-taker says “I know what science thinks about the natural history of human beings—but you better realize, I don’t believe it,” then it is pretty obvious what she is doing: expressing her identity as a member of a community for whom disbelief is a defining attribute. The very occasion for doing so might well be that she was put in a position where revealing of her knowledge of what science knows generated doubt about who she is

But it remains the case that the mental states and motivations that she used to learn and convey what science knows, on the one hand, and the mental states and motivations she is using to experience a particular cultural identity, on the other, are entirely different things (Everhart & Hameed 2013; cf. DiSessa 1982).  Neither tells us whether she will use what evolution knows to do other things that can be done only with such knowledge—like become a veterinarian, say, or enjoy a science documentary on evolution (CCP 2016). To figure out if she believes in evolution for those purposes—despite her not believing in it to be who she is—we’d have to observe what she does in the former settings.

All of these same points apply to the response that study subjects give when they respond to a valid measure of their comprehension of climate science.  That is, their professions of “belief” and “disbelief” in the propositions that figure in the assessment items neither add to nor subtract from the inference that they have (or don’t have) the capacities and motivations necessary to answer the question correctly.  Their respective professions  tell us only who they are. 

As expressions of their identities, moreover, their respective professions of “belief” and “disbelief” don’t tell us anything about whether they possess the “beliefs” in human-caused climate change requisite to action informed by what science knows. To figure out if a climate change “skeptic” possesses the action-enabling belief in climate change that figures, say, in using scientific knowledge to protect herself from the harm of human-caused climate change, or in voting for a member of Congress (Republican or Democrat) who will in fact expend even one ounce of political capital pursuing climate-change mitigation policies, we must observe what that skeptical individual does in those settings.  Likewise, only by seeing what a self-proclaimed climate-change believer does in those same settings can we see if he possess the sort of action-enabling belief in human-caused climate change that using science knowledge for those purposes depends on.

References

Bishop, B.A. & Anderson, C.W. Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching 27, 415-427 (1990).

Braithwaite, R.B. The nature of believing. Proceedings of the Aristotelian Society 33, 129-146 (1932).

Braithwaite, R.B. The Inaugural Address: Belief and Action. Proceedings of the Aristotelian Society, Supplementary Volumes 20, 1-19 (1946).

CCP, Evidence-based Science Filmmaking Inititive, Study No. 1 (2016)

Demastes, S.S., Settlage, J. & Good, R. Students' conceptions of natural selection and its role in evolution: Cases of replication and comparison. Journal of Research in Science Teaching 32, 535-550 (1995).

DiSessa, A.A. Unlearning Aristotelian Physics: A Study of Knowledge‐Based Learning*. Cognitive science 6, 37-75 (1982).

Everhart, D. & Hameed, S. Muslims and evolution: a study of Pakistani physicians in the United States. Evo Edu Outreach 6, 1-8 (2013).

Hermann, R.S. Cognitive apartheid: On the manner in which high school students understand evolution without Believing in evolution. Evo Edu Outreach 5, 619-628 (2012).

Hetherington, S.C. How to know : a practicalist conception of knowledge (J. Wiley, Chichester, West Sussex, U.K. ; Malden, MA, 2011).

Peirce, C.S. The Fixaation of Belief. Popular Science Monthly 12, 1-15 (1877).
Tuesday
May312016

"According to climate scientists ..." -- WTF?! (presentation summary, slides)

Gave talk at the annual Association for Psychological Science on Sat.  Was on a panel that featured great presentations by Leaf Van Boven, Rick Larrick & Ed O'Brien. Maybe I'll be able to induce them to do short guest posts on their presentations, although understandably, they might be shy about become instant world-wide celebrities by introducing their work to this sites 14 bilion readers.

Anyway, my talk was on the perplexing, paradoxical effect of "according to climate scientists" or ACS prefix (slides here).

As 6 billion of the readers of this blog know-- the other 8 have by now forgotten b/c of all the other cool things that have been featured on the blog since the last time I mentioned this--attributing positions on the contribution of human beings to global warming, and the consequences thereof, to "climate scientists" magically dispels polarization on responses to cliimate science literacy questions.

Here's what happens when "test takers" (members of a large, nationally representative sample) respond to two such items that lack the magic ACS prefix:

 
Now, compare what happens with the ACS prefix:

 
Does this make sense?

Sure. Questions that solicit respondents’ understanding of what scientists believe about the causes and consequences of human-caused global warming avoid forcing individuals to choose between answers that reveal what they know about what science knows, on the one hand, and ones that express who they are as members of cultural groups, on the other.

Here's a cool ACS prefix corollary:

Notice that the "Nuclear power" question was a lot "harder" than the "Flooding" one once the ACS prefix nuked (as it were) the identity-knowledge confound.  Not surprisingly, only respondents who scored the highest on the Ordinary Science Intelligence assessment were likely to get it right.

But notice too that those same respondents--the ones highest in OSI--were also the most likely to furnish the incorrect identity-expressive responses when the ACS prefix was removed.

Of course! They are the best at supplying both identity-expressive and  science-knowledge-revealing answers.  Which one they supply depends on what they are doing: revealing what they know or being who they are. 

The ACS prefix is the switch that determines which of those things they use their reason for.

Okay but what about this: do rspts of opposing political ordinations agree on whether climate scientists agree on whether human-caused climate change is happening?

Of course not!

 
In modern liberal democratic societies, holding beliefs contrary to the best available scientific evidence is universally understood to be a sign of stupidity. The cultural cogniton of scientific consensus describes the psychic pressure that members of all cultural groups experience, then, to form and persist in the belief that their group’s position on a culturally contested issue is consistent with the best avaialbel scientific evidence.

But that's what creates the "WTF moment"-- also known as a "paradox":


Um ... I dunno!

That's what I asked the participants--my fellow panelists and the audience members (there were only about 50,000 people, because were scheduled against some other pretty cool panels) to help me figure out!

They had lots of good conjectures.

How about you?

Wednesday
May252016

"Repugnance" & reasoned deciscionmaking ... a fragment

From something I working on ...

Disgust-motivated cognition of costs and benefits

 “Repugnance” can figure in an agent’s instrumental reasoning in a number of ways. One would be as an argument in his or her utility function: repugnant states of affairs are ones worth incurring a cost to avoid; the repugnance of an act is a cost that must be balanced against the value of the otherwise desirable states of affairs that the action might help to promote (e.g., Becker 2013). Alternatively, repugnance might be viewed as investing acts or states of affairs with some “taboo” quality that makes them inappropriate objects of cost-benefit calculation (Fiske & Tetlock 1997). I will address a third possibility: that repugnance might unconsciously shape how actors appraise consequences of actions or states of affairs.  Wholly apart from whatever disutility an agent might assign an act or state of affairs on account of its being repugnant, an agent is likely to conform his or her assessment of information about its risks and benefits to the aversion that it excites in her (Finucane, Alhakami, Slovic & Johnson 2000; Douglas 1966). I will survey the psychological mechanisms for this form of “disgust-motivated” reasoning and assess its implications for rational decisionmaking, individual and collective.

Refs

Becker, G.S. The economic approach to human behavior (University of Chicago press, 2013).

Douglas, M. Purity and Danger: An Analysis of Concepts of Pollution and Taboo (1966).

Finucane, M.L., Alhakami, A., Slovic, P. & Johnson, S.M. The Affect Heuristic in Judgments of Risks and Benefits. Journal of Behavioral Decision Making 13, 1-17 (2000).

Fiske, A.P. & Tetlock, P.E. Taboo Trade-offs: Reactions to Transactions That Transgress the Spheres of Justice. Political Psychology 18, 255-297 (1997).

Friday
May202016

Can you spot which "study" result supports the "gateway belief model" and which doesn't? Not if you use a misspecified structural equation model . . .

As promised “yesterday”: a statistical simulation of the defect in the path analysis that van der Linden, Leiserowitz, Feinberg & Maibach (2015) present to support their “gateway belief model.”

VLFM report finding that a consensus message “increased” experiment subjects’ “key beliefs about climate change” and “in turn” their “support for public action” to mitigate it. In support of this claim, they present this structural equation model analysis of their study results:

 

As explained in my paper reanalyzing their results, VLFM’s data don’t support their claims. They nowhere compare the responses of subjects “treated” with a consensus message and those furnished only a “placebo” news story on a Star Wars cartoon series.  In fact, there was no statistically or practically significant difference in the “before and after” responses of these two groups of subjects’ expressions of belief in climate change or support for global warming mitigation.

The VLFM structural equation model obscures this result.  The model is misspecified (or less technically, really messed up) because it contains no variables for examining the impact of the experimental treatment—exposure to a consensus message—on any study outcome variable besides subjects’ estimates of the percentage of climate scientists who adhere to the consensus position on human-caused global warming.

To illustrate how this misspecification masked the failure of the VLFM data to support their announced conclusions, I simulated two studies designed in the same way as VLFM’s. They generated these SEMs:

As can be seen, all the path parameters in the SEMs are positive and significant—just as was true in the VLFM path analysis.  That was the basis of VLFM’s announced conclusion that “all [their] stated hypotheses were confirmed.”

"Study No. 1" -- is this the one that supports the "gateway model"?But by design, only one of the simulated study results supports the VLFM hypotheses.  The other does not; the consensus message changes the subjects’ estimates of the percentage of scientists who subscribe to the consensus position on human-caused climate change, but doesn’t significantly affect (statistically or practically) their beliefs in climate change or support for mitigation--the same thing that happened in the actual VLFM study.

The path analysis presented in the VLFM paper can’t tell which is which.

Can you?  If you want to try, you can download the simulated data sets here.

To get the right answer, one has to examine whether the experimental treatment affected the study outcome variable (“mitigation”) and the posited mediators (“belief” and  “gwrisk”) (Muller, Judd & Yzerbyt 2005). That’s what VLFM’s path analysis neglects to do.  It’s the defect in VLFM that my re-analysis remedies.

Or is it "Study No. 2"? Download the data & see; it's not hard to figure out if you don't use a misspecified SEMFor details, check out the “appendix” added to the VLFM data reanalysis

Have fun—and think critically when you read empirical studies.

References

Muller, D., Judd, C.M. & Yzerbyt, V.Y. When moderation is mediated and mediation is moderated. Journal of personality and social psychology 89, 852 (2005).

van der Linden SL, Leiserowitz A.A., Feinberg G.D., Maibach E.W. The Scientific Consensus on Climate Change as a Gateway Belief: Experimental Evidence. PLoS ONE (2015), 10(2): e0118489.doi:10.1371/journal.pone.0118489.

 

Thursday
May192016

Serious problems with "the strongest evidence to date" on consensus messaging ... 

So . . .

van der Linden, Leiserowitz, Feinberg & Maibach (2015) posted the data from their study purporting to show that subjects exposed to a scientific-consensus message “increased” their “key beliefs about climate change” and “in turn” their “support for public action” to mitigate it.

Christening this dynamic the "gateway belief" model, VLFM touted their results as  “the strongest evidence to date” that “consensus messaging”— social-marketing campaigns that communicate scientific consensus on human-caused global warming—“is consequential.”

At the time they published the paper, I was critical because of the opacity of the paper’s discussion of its methods and the sparseness of the reporting of its results, which in any case seemed underwhelming—not nearly strong enough to support the strength of the inferences the authors were drawing.

But it turns out the paper has problems much more fundamental than that.

I reanalyzed the data, which VLFM posted in March, a little over a year after publication,  in conformity with the “open data” policy of PLOS ONE, the journal in which the article appeared.

As I describe in my reanalysis, VLFM fail to report key study data necessary to evaluate their study hypotheses and announced conclusions. 

Their experiment involved measuring the "before-and-after" responses of subjects who received a “consensus message”—one that advised them that “97% of climate scientists have concluded that human-caused climate change is happening”—and those who read only “distractor” news stories on things like a forthcoming animated Star Wars cartoon series. 

In such a design, one compares the “before-after” response of the “treated” group to the “control,” to determine if the "treatment"—here the consensus message—had an effect that differed significantly from the control placebo. Indeed, VLFM explicitly state that their analyses “compared” the response of the consensus-message and control-group subjects

But it turns out that the only comparison VLFM made was between the groups' respective estimates of the percentage of climate-change scientists who subscribe to the consensus position. Subjects who read a statement that "97% of climate scientists have concluded that climate-change is happening" increased theirs more than did subjects who viewed only a distractor news story.

But remarkably VLFM nowhere report comparisons of the two groups' post-message responses to items measuring any of the beliefs and attitudes for which they conclude perceived scientific consensus as a critical "gateway" .

Readers including myself, initially, thought that such comparisons were being reported in a table of “differences” in “Pre-” and “Post-test Means” included in the article.

These aren't experimental effects after all...

But when I analyzed the VLFM data, I realized that, with the exception of the difference in "estimated scientific consensus," all the "pre-" and "post-test" means in the table had combined the responses of consensus-message and control-group subjects.

There was no comparison of the pre- and post-message responses of the two group of subjects; no analysis of whether their responses differed--the key information necessary to assess the impact of being exposed to a consensus message.

Part of what made this even harder to discern is that VLFM presented a complicated “path diagram” that can be read to imply that exposure to a consensus message initiated a "cascade" (their words) of differences in before-and-after responses, ultimately leading to “increased support for public action”—their announced conclusion.

The misspecified "gateway belief" SEM...

But this model also doesn't compare the responses of consensus-message and control-group subjects on any study measure except the one soliciting their estimates of the "percentage of scientists [who] have concluded that human-caused climate change is happening."

That variable is the only one connected by an arrow to the "treatment"--exposure to a consensus message.

As I explain in the paper, none of the other paths in the model distinguishes between the responses of subjects “treated” with a consensus message and those who got the "placebo" distractor news story. Accordingly, the "significant" coefficients in the path diagram reflect nothing more than correlations between variables one would expect to be highly correlated given the coherence of people’s beliefs and attitudes on climate change generally.

In the paper, I report the data necessary to genuinely compare the responses of the consensus-message and control-group subjects.

It turns out that, subjects exposed to a consensus message didn’t change their “belief in climate change” or their “support for public action to mitigate it” to an extent that significantly differed, statistically or practically, from the extent to which control subjects changed theirs.

Indeed, the modal and median effects of being exposed to the consensus message on the 101-point scales used by VLFM to measure "belief in climate change" and "support for action" to mitigate it were both zero--i.e., no difference in "after" or "before" responses to these  study measures. 

No one could have discerned that from the paper either, because VLFM didn't furnish any information on what the raw data looked like. In fact, both the consensus-message and placebo news-story subjects' '"before-message" responses were highly skewed in the direction of belief in climate change and support for action, suggesting something was seriously amiss with the sample, the measures, or both--all the more reason to give little weight to the the study results.

But if we do take the results at face value, the VLFM data turn out to be highly inconsistent with their announced conclusion that "belief in the scientific consensus functions as an initial ‘gateway’ to changes in key beliefs about climate change, which in turn, influence support for public action.”

The authors “experimentally manipulated” the expressed estimates of the percentage of scientists who subscribe to the consensus position on climate change. 

Yet the subjects whose perceptions of scientific consensus were increased in this way did not change their level of "belief" in climate change, or their support for public action to mitigate it, to an extent that differed significantly, in practical or statistical terms, from subjects who read a "placebo" story about a Star Wars cartoon series.

That information, critical to weighing the strength of the evidence in the data, was simply not reported.

VLFM have since conducted an N = 6000 "replication."  As I point out in the paper, "increasing sample" to "generate more statistically significant results" is recognized to be a bad research practice born of a bad convention--namely, null-hypothesis testing; when researchers resort to massive samples to invest minute effect sizes with statistical significance, "P values are not and should not be used to define moderators and mediators of treatment" (Kraemer, Wilson, & Fairburn 2002, p, 881). Bayes Factors or comparable statisics that measure the inferential weight of the data in relation to competing study hypotheses should be used instead (Kim & Je 2015; Raftery 1995). Reviewers will hopefully appreciate that. 

But needless to say, doing another study to try to address lack of statistical power doesn't justify claiming to have found significant results in data in which they don't exist. VLFM claim that their data show that being exposed to a consensus message generated a “a significant increase” in “key beliefs about climate change” and in "support for public action" when “experimental consensus-message interventions were collapsed into a single ‘treatment’ category and subsequently compared to [a] ‘control’ group” (VLFM p. 4).  The data -- which anyone can now inspect-- say otherwise.

Hopefully reviewers will pay more attention too to  how a misspecified SEM model can conceal the absence of an experimental effect in a study design like the one reflected here (and in other "gateway belief" papers, it turns out...). 

As any textbook will tell you, “it is the random assignment of the independent variable that validates the causal inferences such that X causes Y, not the simple drawing of an arrow going from X towards Y in the path diagram” (Wu & Zumbo 2007, p. 373).  In order to infer that an experimental treatment affects an outcome variable, “there must be an overall treatment effect on the outcome variable”; likewise. in order to infer that an experimental treatment affects an outcome variable through its effect on a “mediator” variable, “there must be a treatment effect on the mediator” (Muller, Judd & Yzerbyt 2005, p. 853). Typically, such effects are modeled with predictors that reflect the “main effect of treatment, main effect of M [the mediator], [and] the interactive effect of M and treatment” on the outcome variable (Kraemer, Wilson, & Fairburn 2002, p, 878).

Because the VLFM structural equation model lacks such variables, there is nothing in it that measures the impact of being “treated” with a consensus message on any of the study’s key climate change belief and attitude measures. The model is thus misspecified, pure and simple.

To illustrate this point and underscore the reporting defects in this aspect of VLFMI'll post "tomorrow" the results of a fun statistical simulation that helps to show how the misspecified VLFM model-- despite its fusillade of triple-asterisk-tipped arrows--is simply not capable of distinguishing the results a failed experiment from one that actually does support something like the “gateway model” they proposed.

BTW, I initiatlly brought all of these points to the attention of the PLOS One editorial office.  On their advice, I  posted a linke to my analyses in the comment section, after first soliciting a response from VLFM.

A lot of people are critical of PLOS ONE

I think they are being unduly critical, frankly.

The mission of the journal--to create an outlet for all valid work-- is a valuable and admirable one.

Does PLOS ONE publish bad studies? Sure. But all journals do! If they want to make a convincing case, the PLOS ONE critics should present some genuine evidence on the relative incidence of invalid studies in PLOS ONE and other journals.  I at least have no idea what such evidence would show.

But in any case, everyone knows that bad studies get published all the time-- including in the "premier" journals. 

What happens next-- after a study that isn't good is published --actually matters a lot more. 

In this regard, PLOS ONE is doing more than most social science journals, premier ones included, to assure the quality of the stock of knowledge that reserchers draw on. 

The journal's "open data" policy and its online fora for scholarly criticsm and discussion supply scholars with extremely valuable resources for figuring out that a bad study is bad and for helping other scholars see that too.

If what's "bad" about a study is that the inferences its data support are just much weaker than the author or authors claim, other scholars will know to give the article less weight.

If the study suffers from some a serious flaw (like unreported material data or demonstrably incorrect forms of analysis), then the study is much more likely to get corrected or retracted than it would be if it managed to worm its way into a "premier" journal that lacked an open-data policy and a forum for online comments and criticism.

Peer review doesn't end when a paper is published.  If anything, that's when it starts. PLOS ONE gets that. 

I do have the impression that in the social sciences, at least, a lot of authors think they can dump low quality studies on PLOS ONE.  But that's a reason to be mad at them, not the journal, which if treated appropriately by scholars can for sure help enlarge what we know about how the world works.

So don't complain about PLOS ONE. Use the procedures it has set up for post-publication peer review to make authors think twice before denigrating the journal's mission by polluting its pages with bull shit studies.

References

Kraemer, H.C., Wilson, G.T., Fairburn, C.G. & Agras, W.S. Mediators and moderators of treatment ef-fects in randomized clinical trials. Archives of general psychiatry 59, 877-883 (2002).

Muller, D., Judd, C.M. & Yzerbyt, V.Y. When moderation is mediated and mediation is moderated. Journal of personality and social psychology 89, 852 (2005).

van der Linden SL, Leiserowitz A.A., Feinberg G.D., Maibach E.W. The Scientific Consensus on Climate Change as a Gateway Belief: Experimental Evidence. PLoS ONE (2015), 10(2): e0118489. doi:10.1371/journal.pone.0118489.

Wu, A.D. & Zumbo, B.D. Understanding and Using Mediators and Moderators. Social Indicators Re-search 87, 367-392 (2007).

 

 

 

 

Tuesday
May172016

Job opening for social-science editor at Nature! 

Are you a trained social scientist now driving a cab (or uber-registered vehicle)?

Or a gainfully employed US social scientist looking for an exit strategy in case Donald Trump is elected president?

Well, here you go! A job at Nature!

The job itself would be lots of fun, I'm sure, but think of all the cool things you could learn from office scuttlebutt as the journal issues get put together every week!

 

Wednesday
May112016

WSMD? JA! "Confidence intervals" for "Political Polarization Literacy" test

Winning entry. Verified by "Virgin Mary on your French toast, you say?"™ miracle certifiersFormer Freud expert & current stats legend  Andrew Gelman and Josh " 'Hot Hand Fallay' Fallacy" Miller have announced publicly that they scored perfect 14.75's (higher, actually) on the CCP/APPC "Political Polarization Literacy" test.  

They have now demanded that they be awared the "Gelman Cup." That request actually made their "political polarization literacy" scores a bit more credibile, since obviously they are too busy measuring public opinion to stay current with CCP contests and their respective prizes (I've sent them an authentic "Worrship the Industrial Strength Risk Perception Measure!" Virgin Mary Frenchtoast slice" for their performances).

But speaking of CCP games ... you guessed it: Time for another installment of

 the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

Statistical Modeling, Causal Inference, and Social Science commentator@Rahul wondered what it would look like if the plots in the "Political Polarization Literacy" test figure had confidence intervals.

Here's the answer: 


(If you forgot what the policy items are, you can either go back to the original post or just click here.)

Actually, I'm not sure CIs add interesting information here.

Once one knows that the N = 1200 & the sample is representative, it's pretty easy to know what the CIs will look like (around 0.04 at pr = 0.50; smaller as one approaches pr = 0 & pr = 1.0).

The intersting information here is in the covariances of positions and left_right. The CIs don't make that any clearer; if anything, they make that a bit harder to see!  So I'd say for the purposes of the game, the lowess plots, sans CIs, were all the "statistics" & "modeling" needed for us to start learning something (about WEKS) from the data.

But that's my view. Others might disagree.

Who knows-- they might even disagree with me that "spike plots" rather than, say, colored confidence bands are a prettier way to denote 0.95 CI zones if one thinks there is something to be gained by fitting a model to data like these!

Tuesday
May102016

In awe of the Industrial Strength Risk Perception Measure. . . .

This is a little postscript on yesterday’s post on the CCP/APPC "Political Polarization Literacy" test.

A smart friend asked me whether responses to the items in the “policy preferences” battery from yesterday might be different if the adjective “government” were not modifying “policies” in the introduction to the battery.

I think, frankly, that 99% of the people doing public opinion research would find this question to be a real snoozer but in fact it it’s one that ought to keep them up all night (assuming they are the sort who don’t stay up all night as a matter of course; if they are, then up all day) w/ anxiety.

It goes to the issue of what items like these are really measuring—and how one could know what they are measuring.  If one doesn’t have a well-founded understanding of what responses to survey items are measuring—if anything—then the whole exercise is a recipe for mass confusion or even calculated misdirection. I’m not a history buff but I’m pretty sure the dark ages were ushered in by inattention to the basic dimension of survey item validity; or maybe we still are in the dark ages in public opinion research as a result of this (Bishop 2005)?

In effect, my colleague/friend/collaborator/fellow-perplexed-conversant was wondering if there was something about the word “government” that was coloring responses to all the items, or maybe a good many of them, in way that could confound the inferences we could draw from particular ones of them . . . .

I could think of a number of fairly reasonable interpretive sorts of arguments to try to address this question, all of which, it seems to me, suggest that that’s not likely so.

But the best thing to do is to try to find some other way of measuring what I think the policy items are measuring, one that doesn’t contain the word “government,” and see if there is agreement between responses to the two sets of items. If so, that supplies more reason to think, yeah, the policy items are measuring what I thought; either that or there is just a really weird correspondence between the responses to the items—and that’s less a likely possibility in my view.

What do I think the “policy” items are measuring?  I think the policy items are measuring, in a noisy fashion (any single item is noisy)  pro- or con- latent or unobserved attitudes toward particular issues that themselves are expressions of another latent attitude, measured (nosily but less so  because there are two “indicators” or indirect measures of it) by the aggregation of the “partisan self-identification” and “liberal-conservative” ideology items that “Left_right” comprise.

That’s what I think risk perception measures are too—observable indicators of a latent pro- or con-affective attitude, one that often is itself associated with some more remote measure of identity of the sort that could be measured variously with either cultural worldview items, religiosity, partisan political identity, and the like (see generally Peters & Slovic 1996; Peters Burraston & Mertz 2004; Kahan 2009).

The best single indicator I can think of for latent affective attitudes is . . . the Industrial Strength Risk Perception Measure!

As the 14 billion readers of this blog know, ISRPMs consist in 0-7 or 0-10 rankings of the “risk” posed by a putative risk source. I’m convinced it works best when each increment in the Likert scale has a descriptive label, which favors 0-7 (hard to come up w/ 10 meaningful labels).

As I’ve written about before, the ISRPM has a nice track record.  Basically, so long as the putative risks source is something people have a genuine attitude about (e.g., climate change, but not GM foods), it will correlate pretty strongly with pretty much anything more specific you ask (is climate change happening? are humans causing it? are wesa gonna die?) relating to that risk.  So that makes the ISRPM a really economical way to collect data, which can then be appropriately probed for sources of variance that can help explain who believes what & why about the putative risk source.

It also makes it a nice potential validator of particular items that one might think are measuring the same latent attitude.  If those items are measuring what you think, they ought to display the same covariance patterns  that corresponding ISRPMs do in relation to whatever latent identity one posits explains variance in the relevant ISRPM.

With me? Good!

Now the nice thing here is that the ISRPM measure, as I use it, doesn’t explicitly refer to “government." The intro goes like this ...

and then you have individual "risk sources," which, when I do a study at least, I always randomize the order of & put on separate "screens" or "pages" so as to minimize comparative effects:

Obviously, certain items on an ISRPM  battery will nevertheless imply government regulation of some sort.

But that’s true for the “policy item” batteries, the validity of which was being interrogated (appropriately!) by my curious friend.

So, my thinking went, if the ISRPM items had the same covariance pattern as the policy items in respect to “Left_right,” the latent identity attitude formed by aggregation of a 7-point political identity item and a 5-point liberal conservative measure, that would be a pretty good reason to think (a) the two are measuring the same “latent” attitude and (b) what they are measuring is not an artifact of the word “government” in the policy items—even if attitudes about government might be lurking in the background (I don’t think that in itself poses a validity problem; attitudes toward government might be integral to the sorts of relationships between identity and “risk perceptions” and related “policy attitudes” variance in which we are trying to explain).

So. . .

I found 5 “pairs” of policy-preference items an corresponding ISRPMs.

The policy-preferences weren’t all on yesterday’s list. But that’s because only some of those had paired ISRPMs.  Moreover, some ISRPMs had had corresponding policy items not on yesterday’s list.  But I just picked the paired ones on the theory that covariances among “paired items” would give us information about the performance of items on the policy list generally, and in particular whether the word “government” matters.

Here are the corresponding pairs:

I converted the responses to z-scores, so that they would be on the same scale. I also reverse coded certain of the risk items, so that they would have the same valence (more risk -> support policy regulation; less risk -> oppose).

Here are the corresponding covariances of the responses to the items—policy & ISRPM—in relation to Left_right, the political outlook scale

 

Spooky huh?!  It’s harder to imagine a tighter fit!

Note that the items were administered to two separate samples

That’s important! Otherwise, I’d attribute this level of agreement to a survey artifact: basically, I’d assume that respondents were conforming their answer to whichever item (ISRPM or policy) that came second so that it more or less cohere with the one they gave to the first.

But that’s not so; these are response from two separate groups of subjects, so the parallel covariances gives us really good reason to believe that the “policy” items are measuring the same thing as the ISRPMs—and that the world “government” as it appears in the former isn’t of particular consequence.

If, appropriately, you want to see the underlying correlation matrix in table form, click here (remember, the paired items were administered to two separate samples so we have no information about their correlation with each other--only their respective correlations with left_right.)

So two concluding thoughts:

1. The question "what the hell is this measuring??," and being able to answer it confidently, are vital to the project of doing good opinion research.  It is just ridiculous to assume that survey items think they are measuring what you think; you have to validate them.  Otherwise, the whole enterprise becomes a font of comic misunderstanding.

2. We should all be friggin’ worshiping ISRPM! 

I keep saying that it has this wonderful quality, as a single-item measure, to get at latent pro-/con- attitudes toward risk; that responses to it are highly likely to correlate with more concrete questions we can ask about risk perceptions, and even with behavior in many cases.  There’s additional good research to support this.

But to get such a vivid confirmation of its miraculous powers in a particular case! Praise God!

It’s like seeing Virgin Mary on one’s French Toast!

 References

Bishop, G.F. The illusion of public opinion : fact and artifact in American public opinion polls (Rowman & Littlefield, Lanham, MD, 2005).

Kahan, D.M. Nanotechnology and society: The evolution of risk perceptions. Nat Nano 4, 705-706 (2009).

Peters, E. & Slovic, P. The Role of Affect and Worldviews as Orienting Dispositions in the Perception and Acceptance of Nuclear Power. J Appl Soc Psychol 26, 1427-1453 (1996).

Peters, E.M., Burraston, B. & Mertz, C.K. An Emotion-Based Model of Risk Perception and Stigma Susceptibility: Cognitive Appraisals of Emotion, Affective Reactivity, Worldviews, and Risk Perceptions in the Generation of Technological Stigma. Risk Analysis 24, 1349-1367 (2004).

Sunday
May082016

Hey everybody -- take the cool CCP/APPC "Political Polarization Literacy" test!

Because we, unlike certain other sites that I won’t deign to identify, actually listen to our 14 billion regular readers, CCP Blog is adding yet another member to its stable of wildly popular games (which, of course, include MAPKIA!, WSMD? JA!, & HFC! CYPHIMU?): the CCP/APPC “Political Polarization Literacy” Test! 

Official game motto: “Because the one thing we all ought to agree on is what we disagree about!”

Ready . . . set . . . open your test booklet and begin!

Match the policies on this list . . .

 

 

to the plotted lines in this figure:

Take your time, no rush.

Keep going.  

Scroll down when the proctor declares that the exam is over.

If you finish early, feel free to click on the random ass pictures that I've inserted to prevent you from inadverently spotting the answers before completing the test!

On their way to Canada, possibly?

Time’s up!

Okay, here’s what I’m going to do.  First, I’m going to start by showing you the “answer key,” which consists of the original figure with labels.

Second, I’m going to tell you how to score your answers.

To do that, I’ll display separate figures for (a) policies that are strongly polarizing; (b) policies that are weakly polarizing; (c) policies that reflect bipartisan ambivalence; and (d) policies that reflect bipartisan support. In connection with each of these figures, I’ll supply scoring instructions.

So . . .

“Answer key”

 

Point-score figures

1. Strongly polarizing

 

Award yourself (or your child or pet, if you are scoring his or her test) 1 point for each policy that appears in this set and that you (or said child or pet) matched with any of these five plotted lines regardless of which of the lines you actually matched it with. 

Got that? No? Okay, well, e.g., if you matched “stricter carbon emission standards to reduce global warming” with the “magenta” colored line you get 1 point; but you also get 1 point if you matched it with red, blue, midblue, or cyan-colored lines. Same for every other friggin’ policy in this set—okay?

Good. Now give yourself (et al.) a 3-point bonus if you matched “Gay marriage (allowing couples of the same sex to marry each other)” with any of the plotted lines depicted in this figure.

2. Weakly polarizing

 

Award yourself (et al.) 1.5 points for each policy in this set that you (enough of this) matched with either of the two plotted lines in this figure.

3. Bipartisan ambivalence

 

Award yourself 0.75 points if you got this one.

4. Bipartisan support

Award yourself 3 points if you matched “Approval of an amendment to the U.S. constitution that would allow Congress and state legislatures to prohibit corporations from contributing money to candidates for elected office” with either of the plotted lines in this figure.

Subtract 5 points if you failed to match “Requiring children who are not exempt for medical reasons to be vaccinated against measles, mumps, and rubella” with one of the two lines in this figure.

Subtract 5 points if you matched “Gay marriage (allowing couples of the same sex to marry each other)” with either of the two lines in this figure.

Grade:

17: you are a cheater and are banned from this site until Pete Rose is inducted into the Baseball Hall of Fame, Hell Freezes Over, or Donald Trump is elected President, whichever happens first.

14.75: you are either a “political polarization genius,” pr = 0.25, or a liar, pr = 0.75.

10-14.74: Damn! You are one of the 14 billion regular readers of this blog!

5-10: Meh.

0-4: Not awful.

-10: You win! Obviously you have better things to do with your time than waste them viewing the sad spectacle of unreason that our democracy has become!  (But what the hell are you doing on this site?)

Now, some explanation on the scoring.

It was done by a Hal9001 series super computer,  which designed the “game” (obviously if you missed anything, that is due to human error).

But note that the Hal9001 put a lot emphasis on two policies in particular:

“Requiring children who are not exempt for medical reasons to be vaccinated against measles, mumps, and rubella”

“Gay marriage (allowing couples of the same sex to marry each other)”

The reason, I’m told by Hal9001, is that getting these ones wrong is a sign that your child or pet (obviously, we aren’t talking about you here!) is over-relying on heuristic, Political Polarization System 1 reasoning.  As a result, your child or pet is succumbing to the extremely common “what everyone knows syndrome,” or WEKS, a bias that consists in, well, treating “what everyone knows” as evidence. 

Or more specifically, treating as evidence the views of the biased sample (in a measurement sense, not necessarily a cognitive or moral one) of people who one happens to be exposed to disproportionately as a result of the natural, understandable tendency to self-select into “discourse communities” populated w/ people who basically have relatively uniform outlooks and motivations and experiences.

For sure WEKS biases people’s opinions on public opinion vaccines.

There is overwhelming empirical evidence of public support for universal immunization across all political, cultural, religious, etc. groups. Yet commentators, treating each other’s views as evidence,  keep insisting that either one group or another (“the conservative don’t-tread-on-me crowd that distrusts all government recommendations,” “limousine liberals,” blah blah) is hostile to vaccines or even  more patently false a “growing distrust of vaccinations” among  “a large and growing number” of “otherwise mainstream parents.”  And lots of people assume, gee, if “everyone knows that” it must be true!

Same on gay marriage.

In the sources that people on the “left” consult, “everyone knows that” there has been an been “an astounding transformation of public opinion.”  They constantly call for "replicating the success of marriage equality" on climate change, e.g.

Actually, the “transformation” on gay marriage was primarily just a bulging of public support among people on the “left.” Support among people who identify as “liberal” grew from 56% to 79%, and among those who identify as Democrat from 43% to 66%, in the period from 2000 to 2015; among self-identified “conservatives” and Republicans, the uptick was much more modest--from 18% to 30% and 21% to 32% respectively.

That’s a shift, sure. But 79%:30%/66%:32% is . . . political polarization.

The “how to replicate gay marriage” on climate change meme rests on a faulty WEKS premise or set of them. 

One is that Gay marriage isn’t as divisive than climate change. It is.  Or if it isn't, it's only because  there is still a higher probability that a “liberal Democrat” and a “conservative Republican” will agree that gay marriage shouldn’t be legalized than they will agree that the U.S. should or shouldn’t adopt “stricter carbon emission standards to reduce global warming.” 

Maybe climate change advocates should "replicate the success" of gun control advocates and affirmative action proponents, too?

Another faulty premise has to do with the instrument of legal change in gay marraiage.

Legalization of gay marriage occurred primarily by judicial action, not legislation: of the 37 states where gay marriage was already legally recognized before the U.S. Supreme Court decided Obergefell v. Hodges, 26 were in that category as a result of judicial decisions invalidating apparently popularly supported legal provisions (like California’s 2008 popular referendum “Prop. 8”) disallowing it.

Those judicial decisions, in my view, were 100% correct: the right to pursue one’s own conception of happiness in this way shouldn’t depend on popular concurrence.

But I don't think it's a good idea to propogate a false narrative about what really happened here or about what today’s reality is.   False narratives, underwritten by WEKS, lead people to make mistakes in their practical decisionmaking.

Indeed, WEKS-- the disposition of people to confuse the views of people who share one's outlooks, motivations, experience as “evidence” of how the world works—on public opinion and other topics too is one of the reasons we have polarization on facts that admit of being assessed with valid empirical evidence.

One of the reasons in other words that we are playing this stupid game.

Friday
May062016

Raw data: the best safeguard against empirical bull shit!

The last two posts were so shockingly well received that it seemed appropriate to follow them up w/ one that combined their best features: (a) a super smart guest blogger; (b) a bruising, smash-mouthed attack against those who are driving civility from our political discourse by casting their partisan adversaries as morons; and (c) some kick-ass graphics that are for sure even better than "meh" on the Gelman scale!  

The post helps drive home one of the foundational principles of critical engagement with empirics: if you don't want to be the victim of bull-shit, don't believe any statistical model before you've been shown the raw data!

Oh-- and I have to admit: This is actually a re-post from asheleylandrum.com. So if 5 or 6 billion of you want to terminate your subscription to this blog & switch over to that one after seeing this, well I won''t blame you --  now I really think Gelman was being kind when he described my Figure as merely "not wonderful...."


When studies studying bullshit are themselves bullshit...

We have a problem wth PLoS publishing bullshit studies.


Look, I really appreciate some aspects of PLoS. I like that they require people to share data. I like that they will publish null results. However, I really hope that someday the people who peer-review papers for them step up their game. 

This evening, I read a paper that purports to show a relationship between seeing bullshit statements as profound and support for Ted Cruz. 

The paper begins with an interesting premise: does people's bullshit receptivity--that is, their perception that vacuous statements contain profundity--predict their support for various political candidates? This is a particularly interesting question. I think we can all agree that politicians are basically bullshit artists

Specifically, though, the authors are not examining people's abilities to recognize when they are being lied to; they define bullshit statements as 

communicative expressions that appear to be sound and have deep meaning on first reading, but actually lack plausibility and truth from the perspective of natural science.


OK, they haven't lost me yet. 

The authors then reference some recent literature that has describes conservative ideology as what amounts to cognitive bias (at the least) and mental defect (at the worst).

I identify as liberal. However, I think that this is the worst kind of motivated reasoning on the part of liberal psychologists.  Some of this work has been challenged (see Kahan take on some of these issues). But let's ignore this for right now and pretend, that the research they are citing here is not flawed.

The authors have the following hypotheses:

  • Conservativism will predict judging bullshit statements as profound. (I can tell you right off that if this were mostly a conservative issue, websites like spirit science would not exist)
  • The more individuals have favorable views of Republican candidates, the more they will see profoundness in bullshit statements. (So here, basically using support for various candidates as another measure of conservativism).
  • Conservativism should not be significantly related to seeing profoundness in mundane statements. 
These hypotheses were clearly laid out, which I appreciate. I also appreciate that the authors followed the guidelines and posted their data. Makes post-publication peer-review possible.

Here is one of my first criticisms of the method of this paper. The authors chose to collect a sample of 196 participants off of Amazon's Mechanical Turk. Now, I understand why, MTurk is a really reasonably priced way of getting participants who are not psych 101 undergraduates. However, there are biases with MTurk samples. Mainly that they are disproportionately male, liberal, and educated. Particularly when researchers are interested in examining questions related to ideology, MTurk is not your best bet. But, let's take a look at the breakdown of their sample based on ideology, just to check--especially since we know that they want to make inferences about conservatives in particular.
I don't have the exact wording of their measure of conservativism, but they describe it as asking participants to place themselves on a 7-point Likert scale where 1 is liberal and 7 is conservative. The above table shows the frequency of participants at each level. You can see that the sample is disproportionately liberal/liberal leaning. In fact, if you total the participants in columns 5, 6, and 7 (the more conservative columns), you have a whopping total of 46 participants choosing a somewhat conservative designation versus the 108 participants in columns 1, 2, and 3.

Thus, it is unfair--in my opinion--to think that you can really make inferences about conservatives in general from this data. Many studies in political science and communications use nationally-representative data with over 1500 participants. At Annenberg Public Policy Center we get uncomfortable sometimes making inferences from our pre/post panel data (participants who we contact two times) because we end up with only around 600. I'm not saying that it is impossible to make inferences from less than 200 participants, but that the authors should be very hesitant, particularly when they have a very skewed sample.

I'm going to skip past analyzing the items that they use for their bullshit and mundane statements. It would be worth doing a more comprehensive item analysis on the bullshit receptivity scale--at least going beyond reporting Cronbach's alpha. But, that can be done another time.

The favorability ratings of the candidates are another demonstration of how the sample is skewed. The sample demonstrates the highest support for Bernie Sanders and the lowest support for Trump. 
This figure shows the density of participants rating four of the candidates (the authors also include Martin O'Malley and Marco Rubio) on the 5 point scale. You can see that a huge proportion of participants gave Trump and Cruz the lowest rating. In contrast, People tended to like Clinton and Sanders more. 

Moving onto their results.

The main claim that the authors make is that:
Favorable ratings of Ted Cruz, Marco Rubio, and Donald Trump were positively related to judging bullshit statements as profound, with the strongest correlation for Ted Cruz. No significant relations were observed for the three democratic candidates.
Now, the authors recognized that their samples were skewed, so they conducted non-parametric correlations. But, I'm not sure why the authors didn't simply graph the raw data and look at it.

Below, I graph the raw data with the bullshit receptivity scores on the x-axis and the support scores for each candidate on the y-axis. The colored line is the locally-weighted regression line and the black dashed line treats the model as linear. I put Ted Cruz first, since he's the one that the authors report the "strongest" finding for. 
So, does this look like a linear relationship to you? As the linear model goes up (which I imagine is what drives the significant correlation), the loess line goes down. I imagine that this is a result fo the outliers who score high on BSR. Note that of the ones scoring highest on BSR, the largest clump actually shows low support for Cruz, with one outlier rating him at 4. Indeed, the majority of the sample sits at 1 for supporting Ted cruz, and the samples' BSR scores are grouped in the middle.

You can see similar weirdness for the Trump and Rubio Ratings. The Trump line is almost completely flat--and if we were ever to think that support for a candidate predicted bullshit receptivity, it would be support for Trump---but I digress.... Note, too, how low support is.  Rubio, on the other hand, shows a light trend upwards when looking at the linear model (the black dashed line), but most people are really just hovering around the middle. Like with Cruz, the people with the highest bullshit receptivity (scores of around 5) rate Rubio low (1 or 2).


So, even if I don't agree that your significant correlations are meaningful for saying that support for conservatives is predicted by bullshit receptivity (or vice versa), you might still argue that there is a difference between support for liberals and support for conservatives. So, let's look at the democratic candidates.
I put Hillary Clinton on top, because I want to point out that her graph doesn't look much different from that of Cruz or Rubio. It still has a trend upwards, but not just for the linear model. In fact, where Cruz had one person that gave high ratings for him and had the highest level of bullshit receptivity, Hillary has three.  In fact, let's take a look at Clinton's loess Line and Cruz's loess line mapped on the same figure....
Kind of looks to me like there is a stronger argument for Clinton supporters having more bullshit receptivity. Now, I do support Hillary Clinton, so don't get me wrong. My point is only to demonstrate that significant p values for one type of correlation do not mean that the finding is real.

Here are the figures for Sanders and O'Malley. Again, pretty straight lines. But for the highest level of bullshit receptivity, Bernie indeed has a cluster of supporters--people rating him at the highest level. I do not think that this supports the opposite conclusion to what these authors found--that bullshit receptivity predicts democratic candidate support--but I don't think that the authors should have made the conclusions that they did. These results really just appear to be noise to me.
In conclusion...

The authors *do* list the limitations of their study. They state that their research is correlational and that their sample was not nationally representative. But they still make the claim that conservatism is related to seeing profoundness in bullshit statements.. Oh, which reminds me, we should have looked at that too...
That's a graphical representation of conservatism predicting bullshit receptivity. 

What concerns me, here, is two-fold.

First, despite what p values may or may not be below a .05 threshold, there is no reason to think that this data actually demonstrates that conservatives are more likely to see profundity in bullshit statements than liberals--But the media will love it.

Moreover, there is no reason to believe that such bullshit receptivity predicts support for conservative candidates--but the media will love it. This is exactly the type of fodder picked up  because it suggests that conservatism is a mental defect of some sort. It is exciting for liberals to be able to dismiss conservative worldviews as mental illness or as some sort of mental defect. However, rarely do I think these studies actually show what they purport to. Much like this one.

Second, it is this type of research that makes conservatives skeptical of social science. Given that these studies set out to prove hypotheses that conservatives are mentally defective, it is not surprising that conservatives become skeptical of social science or dismiss academia as a bunch of leftists. Check out this article on The Week about the problem of liberal bias in social science

If we actually have really solid evidence that conservatives are wrong on something, that is totally great and fine to publish. For instance, we can demonstrate a really clear liberal versus conservative bias in belief of climate change. But we have to stop trying to force data to fit the view that conservatives are bad. I'm not saying that this study should be retracted,  but it is indicative of a much larger problem with the trustworthiness of our research.