follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« "They already got the memo" part 2: More data on the *public consensus* on what "climate scientists think" about human-caused global warming | Main | Is the controversy over climate change a "science communication problem?" Jon Miller's views & mine too (postcard from NAS session on "science literacy & public attitudes toward science") »

As their science comprehension increases, do members of the public (a) become more likely to recognize scientific consensus exists on human-caused climate change; (b) become more politically polarized on whether human-caused climate change is happening; or (c) both?!

Holy mackeral! Pr(this)/Pr(virgin mary on french toast)=10^-3So CCP and the Annenberg Public Policy Center just conducted a humongous and humongously cool study on climate science literacy. There’s shitloads of cool stuff in the data!

The study is a follow up to an earlier CCP/APPC study, which investigated whether it is possible to disentangle what people know about climate science from who they are

“Beliefs” about human-caused global warming are an expression of the latter, and are in fact wholly unconnected to the former.  People who say they “don’t believe” in human-caused climate change are as likely (which is to say, extremely likely) to know that human-generated CO2 warms the earth’s atmosphere as are those who say they do “believe in” human-caused climate change.

They are also both as likely-- which is to say again, extremely likely--to harbor comically absurd misunderstandings of climate science: e.g.,  that human generated  CO2 emissions stifles photosynthesis in plants, and that human-caused global warming is expected to cause epidemics of skin cancer.

In other words, no matter what they say they “believe” about climate change, most Americans don’t really know anything about the rudiments of climate science.  They just know -- pretty much every last one of them--that climate scientists believe we are screwed.

Click me to see what Rs & Ds think climate scientists think!The small fraction of those who do know a lot—who can consistently identify what the best available evidence suggests about the causes and consequences of human-caused climate change—are also the most polarized in their professed “beliefs” about climate change.


The central goal of this study was to see what “belief in scientific consensus” measures—to see how it relates to both knowledge of climate science and cultural identity.

I’ll get to what we learned about that "tomorrow."

But today I want to show everybody something else that surprised the bejeebers out of me.

Usually when I & my collaborators do a study, we try to pit two plausible but mutually inconsistent hypotheses against each other. I might expect one to be more likely than the other, but I don’t expect anyone including myself to be really “surprised” by the study outcome, no matter what it is. 

Many more things are plausible than are true, and in my view, extricating the latter from the sea of the former—lest we drown in a sea of “just so” stories—is the primary mission of empirical studies.

But still, now and then I get whapped in the face by something I really didn’t see coming!

This finding is like that.

But to set it up, here's a related finding that's  interesting but not totally shocking.

It’s that the association between identity and perceptions of scientific consensus on climate change, while plenty strong, is not as strong as the association between identity and “beliefs” in human-caused climate change.

This means that  “left-leaning” individuals—the ones predisposed to believe in human-caused climate change—are more likely to believe in human caused climate change than to believe there is scientific consensus, while the right-leaning ones—the ones who are predisposed to be skeptical—are more likely to believe that there is scientific consensus that humans are causing climate change than to actually “believe in” it themselves.

Interesting, but still not mind-blowing.

Here’s the truly shocking part:

Got that?

First, as science comprehension goes up, people become more polarized on climate change.

Still not surprising; that’s old, old, old,  old news.

But second, as science comprehension goes up, so does the perception that there is scientific consensus on climate change—no matter what people’s political outlooks are!

Accordingly, as relatively “right-leaning” individuals become progressively more proficient in making sense of scientific information (a facility reflected in their scores on the Ordinary Science Intelligence assessment, which puts a heavy emphasis on critical reasoning skills), they become simultaneously more likely to believe there is “scientific consensus” on human-caused climate change but less likely to “believe” in it themselves! 

Whoa!!! What gives??

I dunno.

One thing that is clear from these data is that it’s ridiculous to claim that “unfamiliarity” with scientific consensus on climate change “causes” non-acceptance of human-caused global warming.

But that shouldn’t surprise anyone. The idea that public conflict over climate change persists because, even after years and years of “consensus messaging” (including a $300 million social-marketing campaign by Al Gore’s “Alliance for Climate Protection”), ordinary Americans still just “haven’t heard” yet that an overwhelming majority climate scientists believe in AGW  is patently absurd. 

(Are you under the impression that there are studies showing that telling someone who doesn't believe in climate change that “97% of scientists accept AGW” will cause him or her to change positions?  No study has ever found that, at least with a US general public sample.  All the studies in question show -- once the mystifying cloud of meaningless path models & 0-100 "certaintly level" measures has been dispelled-- is that immediately after being told that “97% of climate scientists believe in human-caused climate change,” study subjects will compliantly spit back a higher estimate of the percentage of climate scientists who accept AGW.  You wouldn't know it from reading the published papers, but the experiments actually didn’t find that the “message” changed the proportion of subjects who said they “believe in" human caused climate change....)

These new data, though, show that acceptance of “scientific consensus” in fact has a weaker relationship to beliefs in climate change in right-leaning members of the public than it does in left-leaning ones. 

That I just didn’t see coming.

I can come up w/ various “explanations,” but really, I don’t know what to make of this! 

Actually, in any good study the ratio of “weird new puzzles created” to “existing puzzles (provisionally) solved” is always about 5:1. 

That’s great, because it would be really boring to run out of things to puzzle over.

And it should go without saying that learning the truth and conveying it (all of it) accurately are the only way to enable free, reasoning people to use science to improve their lives.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (50)

I'm not surprised. I'm surprised you are..

Ie those sceptical, with high education background will have looked hard at the science, and also be aware of the 'Cook consensus' and reject the Cook consensus misrepresentations.
As Joe Duartes does (who is no climate sceptic|) they've just looked at the data and science.

most sceptics |I know are part of the real objject of the consensus, ie earth warmed, since 1750's, CO2 a GHG, man contributes, etc, etc

they just reject the do you believe in 'climate change - yes/|no - false representation
denier if no - the world view of Cook/Lewandowsky/|Oreskes/Klein

see here:



February 8, 2016 | Unregistered CommenterBarry Woods

Ie I accept the IPCC consensus as being perfectly reasonable.. the WG1 since 1950, more than half the warming, etc,etc.

but reject the media-acttivist soundbite representation of 97% Cook/Doran/Anderegg , etc consensus, as they misrepresent the object of the that consensus

so when people reject this "97% scientific consensus|", but accept the science are they are not really just consciously rejecting the hyped up propaganda consensus.

February 8, 2016 | Unregistered CommenterBarry Woods


1. Remember, it's hazardous to generalize from one's own understanding of issues to the meaning of public responses ot survey items on climate change.

2. Most right-leaning Americans reject both AGW and the propositon that there is scientific consensus on AGW.

3. It's interesting to note that the intensity with which (or probability that they will) reject the latter is is greater; rejecting AGW, I'd infer, is a more reliable indicator of the cultural identity that predisposes to skepticism than is rejection of the proposition that there is consensus.

4. Most Americans, regardless of identity, perceive that climate scientists have concluded we are screwed b/c of AGW. The answer they give to questions relating to that reflect neither identity nor knowledge but a general affective sense that science says we are screwed. It is implicit in that that those sorts of questions don't provoke the identity-protective reaction that either "do you believe in AGW" or "there is scientific consensus on AGW-- true or false?" does.

5. On how people react to "97% msg" ... stay tuned.

February 8, 2016 | Registered CommenterDan Kahan

Hi - Dan I put some links in my other comment, more than just my understanding - Andrew montford response (and others) to Cook's 97% consensus - He was part of it, isn't everyone. (lots of urls, probably caught in spam trap)

Has anyone actually asked people WHY, rather than speculating |(USA surveys)

February 8, 2016 | Unregistered CommenterBarry Woods

@Barry & @AndyWest: Coments w/ multiple URLs tend to get sent to spam Guantanamo. I think all comments have now been released.

@Barry-- ask people why what? And exactly what do you expect the 80% who believe CO2 is toxic to plants to tell you when you ask them why anything about climate change?

February 8, 2016 | Registered CommenterDan Kahan

I don't think this is so surprising.

More scientifically knowledgeable people will know that most climate scientists say that most of the warming of the last century was human caused. That's a "fact" that you can learn for example by reading the IPCC report.

But scientifically knowledgeable right-wing people (or even left-wing people) may have good reasons to doubt the verdict of "most expert climate scientists", to think that they might be biased or misleading or exaggerating the problem.

There could be many reasons for them thinking this.

a) Scientists are susceptible to groupthink.

b) University academics have a strong tendency to left-wing views. Is it just a coincidence that the climate change story (industry bad, capitalism bad, poor people suffering, all our fault) fits so well with the worldview of the middle-class lefty, the worldview that dominates in academia?

c) Science is competitive. If one climate scientist says we're heading for 1 degree of warming, another says 3, and another says 5, who is going to get the most attention?

d) Scientists have to exaggerate the importance of their own work. For example by getting a not very surprising result and describing it as Whoa!!! Sometimes this is in order to get funding. To get a research grant you have to convince the funders that your work is vital, and more important than the other guy's. Sad but true.

e) As Feynman said "Science is the belief in the ignorance of experts".

But this is all speculation, as you say, they are "explanations". And as you rightly say, good research raises as many questions as it answers, or more.

The next stage would be (and I think this is what Barry is suggesting) to find these people with high science intelligence who agree with "most climate scientists agree..." but disagree with "There is solid evidence that..." and ask them why they disagree.

I think we may have discussed this before, but no harm doing it again!

February 8, 2016 | Unregistered CommenterPaul Matthews


what you say makes sense of course but it would make more if there weren't still such a humongous political effect. If it's a straightforward "what do you know" fact question I wouldn't expect those at 90th percentile or above who are on right to be struggling to hit 50%.

On the other hand, if it's a straightforward "who are you" identity question, then I also expect those at 90th percentile plus not be so far north of 0%.

What sort of quesetion is this at this point?

For sample as whole, correlation (spearman's rho) for belief in human caused climate change & agreement that there is scientific consensus = 0.48. That's what surprises me. I can go back & check, but I'm pretty sure the last time I collected data on this issue (probably 2009 or so), the connection between "scientific consnesus" & "belief in climate change was much much stronger. That the correlation gets weaker as sciencve comprehension goes up floors me -- that's not what happens for any set of items that measure identity.

So "scientific consesus" seems to be losing *its* sense of identity. It's becoming noisy. That's interesting.

And of course interesting too that that's happening w/o the climate change item budging a centimeter

February 8, 2016 | Registered CommenterDan Kahan

Dan, I am wondering what it is you find surprising. The surprise is about the result expressed in the last figure -- the one which compares belief in the “scientific consensus” question and the “most expert climate scientists agree that human activity is warming planet” question. Yes?

Aside: It’s easier for the reader/commentator Dan if you label the illustrations & figures.

I wonder what you find surprising because the result expresses what “Luke-Warmers” have been saying for years. Most expert climate scientists, including so-called skeptics and Luke-Warmers agree that human activity is warming the planet. The Luke-warmers have been patiently repeating over and over again that the difference between a luke-warmer and what you identify as “the scientific consensus” is a matter of degree. (pun intended). That is, in case anyone misses the point,that:
- human activity is warming the planet
- and it is quite possible (or even likely to listen to some) that the warming is NOT due “mostly to human activity”.

So if this result is new then it is good news to the luke-warming project of science communication. That is, this might be an example of ‘effective’ science communication!

But I thought that you, Dan, would already understand that about the luke-warmer/skeptic position and therefore your surprise must be about something else.

February 8, 2016 | Unregistered CommenterCortlandt


Same response as to @PaulMathews

February 8, 2016 | Registered CommenterDan Kahan

@BarryWoods-- same as @PaulMathews!

February 8, 2016 | Registered CommenterDan Kahan

@Dan. More excellent data :)

>>...That I just didn’t see coming.

From your data, this graph shouldn't be a surpise. But your logic train appears to have slipped away from what the data says. So...

>>“Beliefs” about human-caused global warming are an expression of the latter [i.e. identity]

So far so good, this is fine.

>>They are also both as likely... harbor comically absurd misunderstandings of climate science.

This conclusion is an artefact of your questioning style (which nevertheless worked in distancing from identity issues). You initially recognized what the data pointed to, but then rejected the idea because it didn't fit one of your priors: "Someone could say, reasonably, that asking people what they think 'climate scientists believe' is different from measuring whether those people themselves believe what they [sic] climate scientists have concluded."

Yes. Most folks in both camps are telling us only what they think the scientists believe, but this is *not* what they believe themselves. And it is mostly the Dem / Libs who let their guard down when identity challenge is lessened, not the Rep / Cons. For the bulk of Rep/Cons, this assumption of 'scare stories' is more or less their normal public position regarding the state of climate science. Their response is consistent. Yet for many of the Dem/Libs this represents a significant shift from the party line of complete belief in the output of that favored discipline; in other words a weakening of their belief and a loss of resolve, because they are not directly challenged here to defend their party and associated identity. A smaller group of Dem / Libs genuinely believe *themselves*, but the two groups cannot be distinguished via those questions alone. You need other data for that.

See for more detail, especially Appendix 2.

>>They just know -- pretty much every last one of them--that climate scientists believe we are screwed.

No. The apparent bi-partisan consensus is not driven by a widespread apprehension of climate danger (latent or otherwise), but in large part by a common lack of faith in climate science and in lesser part by the question design, which can’t distinguish between the two possible main motivations for affirmative responses as mentioned above (mainly within the Dem/Libs).

>>The small fraction of those who do know a lot... ...are also the most polarized in their professed “beliefs” about climate change.

Yes. This is a sign of strong culture. Polarization of this kind occurs because as folks become more domain knowlegable, they discover *different* knowledge bases (in this case 'orthodox', and 'skeptical'). Because your first polarization graph is symmetrical, one can't tell which side is the cultural position and which side is the evidential position (and indeed there is the 3rd option that they are both cultural, as would be the case for a religious schism, say). You need further data for that, see the same link above, or for simpler analysis the 'who is who' file here:

You essentially force-fit the situation by assigning 100% correctness to the Dem/Libs (via the underlined answers), which is also equivalent to saying therefore that 100% of the cultural bias is in the Rep / Con camp. But we should let the data *tell us* which side is the cultural one and which the evidential, which is allowed to happen at the above links. This prevents prior bias from spiking the analysis.

>>Here’s the truly shocking part

So we no longer have any need for shock. If you travel the above path, it shows that in the first polarization graph the cultural bias is in the Dem / Lib camp. Self affirmation of 'the 97%' is itself a central narrative of the culturally enforced consensus the Dem / Libs ascribe to, hence we'd expect to see a similar shape for them in the second graph. There is more affirmation as their (orthodox) knowledge increases.

For the Rep / Cons we would expect some rise too, because with increased (skeptical) knowledge they become more aware that they are facing a consensus, which they certainly acknowledge. Yet this is only in the sense that it's a culturally enforced consensus and not one based on an evidential position. Hence the rise is attenuated; the notion of a 'scientific consensus' is undermined by the true nature of the phenomenon, and of course as their skeptical knowledge deepens they also become more aware of the scientists who certainly aren't in the consensus (regarding calamity), plus the fact that the majority of themselves would be inside it if the definition remained limited to the properties of CO2. Yet certainly you'd expect that more of them would agree with the statement; even if it happens to be far from 97%, it is clear to all that a majority of domain experts hold this position.

>>weird new puzzles created

So there doesn't seem to be any puzzle, and as I said above, most excellent data.

February 8, 2016 | Unregistered CommenterAndy West


hmmm. will need to think about your points for a spell.

But in meantime, can you help me with this?--

You essentially force-fit the situation by assigning 100% correctness to the Dem/Libs (via the underlined answers), which is also equivalent to saying therefore that 100% of the cultural bias is in the Rep / Con camp. But we should let the data *tell us* which side is the cultural one and which the evidential, which is allowed to happen at the above links. This prevents prior bias from spiking the analysis.

In item response profiles for OCSI 1.0,, I do underscore the answer scored as "correct." But why do you say the "Dem/libs" position is 100% correct? Do Republicans believe that climate scientists expect human C02 emissions to prevent photosynthesis in plants? Or that climate scienists believe that global warming will cause skin cancer?

Actually, individuals of opposing political outlooks were equally likely to select the 'wrong' answer to those.

As I've stated before (in connection with the major govt scandal that erupted over how I scored the "Northpole"' item), the 'wrong' answers to particular items could no doubt reasonably be defended by very smart people. But the likelihood that anyone who picked a "wrong" answer did so for the reason that a very smart person would is very very low; the answers scored 'correct' were the ones that in fact were ones the likelihood of selecting increased in proportion to how many other correct answers a test taker got.

In that sense the data was allowed to "score" the test.

If there's an item that is "biased" toward either poltical group--that is, is more likely to be answered "correctly" by a member of one group than one of another with the same estimate "climiate science literacy" capacity-- then the question is one that I would want to discard. That is exactly the reason I used Item Response Theory to score and analyze responses -- b/c it is the right tool for detecting "cultural bias" of that sort in standardized tests.

February 8, 2016 | Registered CommenterDan Kahan

@Dan - I suggest you take a strong look at the various "meta" elements in the "AGW is a Hoax" narrative, such as "consensus is anathema to science" which is huge and constantly kept in fighting trim. It cleverly makes the "97% consensus" into a witness against itself (no 5th amendment there). I tried to convey what scientific consensus really means (mentioned above; reference if needed) with an analogy to the Plate Tectonics study of social epistemologist Miriam Solomon. One problem with the 97% consensus is that any real scientific consensus and hence the AGW one (which preceded and lead to some climate scientists becoming maybe too activist) looks nothing like word-mining scoring and counting academic papers and if we could get people to understand what it really is like that might make a difference. So in the present atmosphere especially (and in general really), science communication ought to frequently make efforts to convey that.

A few years ago, when I was first launched into becoming the amateur investigator of what's up with whatsupwiththat, and the flood of really well crafted (certainly not done by ignorant people) anonymous emails conveying little known proof of Obama's secret Islamitude, and other lies that would damage Rush Limbaugh's reputation if he were to personally deliver them ... Ah Say, Ah Say (Foghorn Leghorn accent) when I was first launched into all that, from reading prodigious comment-storms in many places, including, but also invading more liberal venues, I concluded what we have here is less a movement for anything, than a massively stroked and stoked "Great Liberal Hating and Baiting Cult", with a very big self-organizing component, but definitely nourished in all sorts of ways by the folks you can read about in Dark Money: The Hidden History of the Billionaires Behind the Rise of the Radical Right by Jane Meyer (best book yet of its class and I've read many).

February 8, 2016 | Unregistered CommenterHal Morris

Perhaps it is just time to stop the statistical analysis of survey answers.. and to ASK people why they gave those survey answers, then all might become clearer

an example, if someone tick climate science a hoax, does that mean

a) they think it is a hoax
b) they don't think it is a hoax, put tick it because they are luke warm, and a negative will give the sceintist a false opinion of their views
c) they fully accept climate science, but such a blatant attempt to paint sceptics as nutter it is irritating.
d) reaction to the science is settled political rhetoric on policy
e) bored, ticked it for a laugh, only a 15 minute survey, not really interested/committed to it.
f) dozens of other reasons

drop the statistics for a bit, survey the people again, and if tick a certain box, add a comments box, and ask WHY?

also do people can put the 'right' answer' for the wrong reasons.
Also ask the people who are very concerned about why, they are concerned, could be because they have read IPCC AR 5 summary for policymakers, or, they've heard the great swathes of the earth will be uninhabitable in a few years (celeb Emma Thompson, speaking for Greenpeace)

I've found a survey paper, where dozens of these answers go begging for further questions,but instead statistical analysis is used to find something new (albeit nobody in the data agreed with it.

I'll give some examples, the people surveyed/involved may surprise you!

February 9, 2016 | Unregistered Commenter|Barry Woods

This discussion is getting very interesting.

Dan can you clarify one thing: In your reply to Andy you talk about the OCSI test. But the graphs in the post are labelled OSI, which is rather different. I just wanted to check, are the graphs labelled correctly, was it OSI?

Andy I think you are being a bit unfair to Dan.
"You essentially force-fit the situation by assigning 100% correctness to the Dem/Libs".
I don't think he does this. From the 14 billion hours I've spent studying Dan's blog I get the impression that he takes a balanced view of this, saying that both sides skew the science towards their own political viewpoint.
This is one of the things that sets him apart from 97% of the academic researchers in this area who only want to look at one side of the coin.

February 9, 2016 | Unregistered CommenterPaul Matthews

Great discussion all. Thanks. Please keep it up.

February 9, 2016 | Unregistered CommenterEric Fairfield


"OSI" = Ordinary Science Intelligence" assessment.

"OCSI" = Ordinary Climate Science Intelligence" assessment.

I assumed @AndyWest was referring to latter, since he was makiing a point about the relative comprehension of climate science among members of the public who "believe in" & "don't believe in" human-caused climate change (also, I'm pretty sure I underscored "correct" answers only in a graphic of item response profiles of latter). The most recent study is part of ongoing effort to create an OCSI 2.0 -- an instrument that like 1.0 "disentangles" knoweldge & identity but that can distinguish levels of knowledge in a more fine grained way across larger portion of popuolation & w/ regard to more interesting elements of climate science.

February 9, 2016 | Registered CommenterDan Kahan


I find what you are referring to interesting, certainly, but as you can see, it is different from the sort of thing I tend to study.

Actually, I think people often *don't* see that! E.g., one way to undertand the comments of @BarryWoods, @PaulMathews & @Andywest is that they think the survey respondents who generate the patterns reported in the post are people *like them*. Very few people, even in the 90th percentil & above in OSI, spend more than 2 seconds a day on avg thinking about climate change or the climate change debate. Asking them to "elaborate" would be inviting them to engage in confabulation. They don't have "reasons" for their answers; they just *are* their answers.

Or that is my impression of the relative population frequency of caring & having arguments at the ready on climate change & other public policy issues (actually, climate change is one where people are 10^5 more likely to have an opinoin & an argument than critical policy issue picked at random).

But I do agree w/ @BarryWoods that people should *talk* to people & not just survey them. They should talk to him & to others picked at random from population so he & I can have information that helps us assess our competing understandings of how elaborated the views are for the sorts of responses measured in the survey questions. They should also talk b/c in the end, we should be confident only in explanations supported convergently by multiple valid methods (ethnographic can be valid; but lots of interview methods can be invalid-- just as lots of survey ones are; no escape from judgment here).

That said, I'm trying to take @AndyWest's & everyone else's "your missing the point about the Red guys!" seriously. I think it woudl be a useful thing for me to identify competing hypotheses & then I & others can think of observations that can be made that would give us more or less reason to believe one or the other of those conjectures is right.

But back to your point: I think the style & logic of the "skeptical" rhetoric -- & the "non-skeptical" or whatever we would call it -- are interesting matters for investigation. But they can't explain what I'm intersted in -- which is why ordinary people, who would rather watch Honey Boo Boo goes to Las Vegas than either O'Reily or Madosow "news" hrs, manage to end identifying themselves so strongly w/ these poisitions & using them to figure out whom to hang out w/ & whom to hate...

Oh-- & this is neither here nor there, but on "scientific consensus." Surely it (the weight of expert judgment on an issue that admits of empirical inquiry) is a something a reflective person trying to make a decision would want to know. But the only reason it is of any value to such a person is that "scientific consensus" is not something scientists themselves treat as relevant evidence when they exercise professional judgment on matters w/i the domain of their expertise. I think Galileio is a comic book illustration of this point. Popper's "Logic of Scientific Discovery"-- for all its imperfections -- gets this point right: the truth is whoever the current reigning champion is in a perpetual match of conjecture & refutation open, w/o restriction, to all comers

February 9, 2016 | Registered CommenterDan Kahan

This line of questioning brings to mind the Delphi method of discerning a best guess among a group of experts:
- In step one, the members of the group are asked to give their personal best guess, anonymously.
- In step two, they are informed of the statistics of the result of step one; and asked for their opinion again.

Typically, the breadth of the distribution is much narrower on the second round: There tends to be a convergence effect.

(A description of the Delphi method can be found in Wikipedia.)

The comparison to the climate discussion is useful, partly because climate phenomena involves both a very broad range of scientific disciplines and a great deal of complexity. I don't believe any one person can be considered a real expert in ALL the various arenas of science that apply to the climate. It is a field where you need a large group of individuals with deep and partially overlapping expertise.

(The overlap ensures not only that many eyes are looking at every aspect of the problem, but also allows individual scientists to validly influence each other. For example: Suppose I am an expert in subfields A and B of climate studies, and another individual is an expert in subfields B and C. If an issue comes up on a topic in subfield C, I myself would not have a very well-informed opinion. But it makes sense for me to be interested in what my colleague says about it, because:
a) I know that he is an expert in C; and
b) From our common discussions in subfield B, I have experience with how (and how well) he thinks and forms opinions. If I know him to be a careul and knowledgable thinker in subfield B, it is not unreasonable to expect that he would also be careful and knowledgable in subfield C. Conversely, if I regard his contributions to subfield B as being more confusing than helpful, I would be much less willing to accept his opinions on subfield C.)

So we might think of the result of this study as analogous to an instant Delphi test for the participants: They can more easily sniff out the "sense of the committee" on the topic of climate change than they can give a coherent opinion and rationale of their own.

The last graph shown indicates that conservatives are more likely to go against what they perceive as scientific opinion than liberals. That doesn't seem unreasonable: After all, as Stephen Colbert remarked, "Reality has a well-known liberal bias."

February 9, 2016 | Unregistered CommenterNeal J. King

I've discussed climate change, GMOs, and nuclear power with many who disagree with scientific consensus. I find the GMO and climate change discussions most similar, in that those who oppose scientific consensus have the most trouble articulating any argument to support their position. Both love the argument, "More tests!" Anti-nuclear people can at least say "Chernobyl", although most are stuck if forced to say more. Eg, when people ask, "What about nuclear waste?", I ask, "What about nuclear waste?", and few can produce a response.

What I've seen:
• In my limited experience (hundreds who oppose GMOs, maybe 3 who were scientifically literate), those who oppose GMOs and are at all scientifically literate do so because of the patent issue. In my very limited experience on this topic, asking why the ag patent issue differs for non-GMOs and GMOs leads them to lose interest over time in opposing GMOs.
• Those who are scientifically literate and oppose nuclear power, do not, for the most part, articulate arguments against nuclear power. They explain that we can do it all with renewables. Or say something to that effect, eg, that Germany and CA have replaced nuclear with renewables (including biopower, which they dislike), which isn't quite the same as saying we can do it with renewables.
• I've seen more of the most scientifically literate opposing climate science consensus. This is in part because I have argued with them on energy and skeptic pages. Like the anti-GMO people, they become stuck very early. Ask them to identify flaws in the physics and provide an alternative explanation of the observations, and only one in my experience produced any argument—one example of climate change not primarily CO2 related. The consensus discussion comes up a lot, and I ask them to define consensus. They usually ignore the question or, occasionally, say something to the effect that it's about data, not voting in the absence of data, sometimes quoting sci-fi writer Crichton. I often push on those who disagree with scientific consensus, and almost never get a definition of any kind. (Among scientists, there is consensus when a topic has been approached from so many ways over such a period of time, that no one is interested in studying it unless new information appears, except for the OJ and the glove people. Go to American Geophysical Union, and there are oodles of lectures and posters on the effects of particulates on weather and climate. No discussion as to whether GHG are warming Earth.)

I've seen the discussion about climate change morph over the years. People who used to find unspecified flaws in the arguments about GHG warming Earth now find unspecified flaws in the predictions of what will happen this century and beyond, all without identifying flaws in the physics or providing alternative explanations of the observations. Their arguments are few:
• I haven't seen real change in Iowa, just the usual variability (from an Iowa farmer)
• there is disagreement among respected scientists, so I can take whatever position I want and run with it, loudly and forcefully (a popular argument in all discussions, although scientists in the same position often become less assertive about their position).
• Parts of Earth have been this warm before (in the last few thousands of years), and Earth as a whole was even warmer in the long past, therefore.... This argument is not fleshed out in any way, eg, discussions about the rate of change and human settlements mostly lead to a repeating of the original argument.

I would be interested in how many of those who are scientifically literate and oppose climate consensus spend some or a lot of their time arguing their position, and the flaws they find. Also, can they explain themselves?

February 9, 2016 | Unregistered CommenterKaren Street

as relatively “right-leaning” individuals become progressively more proficient in making sense of scientific information [...] they become simultaneously more likely to believe there is “scientific consensus” on human-caused climate change but less likely to “believe” in it themselves!

Belief that scientists often are dishonest, or conspiratorial, is the missing link. Distrust of scientists rises with education (& other information markers) among Tea Party supporters, but declines with education among Democrats or Independents. I know education is not the same as your indicator, but in this case it seems to capture the same dynamic.

February 9, 2016 | Unregistered CommenterL Hamilton


>>But why do you say the "Dem/libs" position is 100% correct?

To be fully clear, I said you (inappropriately) assign 100% correctness to the Lib / Dems, which we can see you've done via your underlines. For example regarding the test that detects the polarization of the more knowledgable. So in analyzing your climate related responses in such tests, you've applied a hard prior assumption that may not be true.

This assumption automatically forces your various analyses to conclude that the main cultural bias *must* be in the Rep / Con camp, because it clearly cannot be in the camp you've just *defined* as correct. Yet there are ways to let the data *tell us* which camp (e.g. on your [original] symmetrical polarization graph) the cultural bias lies within. This is far better to do because it avoids the bias of belief, or anti-belief, in any particular scientific assumption (science can easily be derailed by strong culture, e.g. the early twentieth century eugenics episode). For sure we know there is a strong ideological bias and corresponding identity association, or you wouldn't get that polarization. But we can't assume we know where this is coming from (it could be from *both* camps, for instance).

>>Do Republicans believe... ...Or that climate scientists believe that global warming will cause skin cancer?

Moving on to your specific test that includes faked climate issues (rather than those focusing exclusively on the issues core climate scientists actually do raise, per above) this is a good cross-check and indeed a great way to try and let the data speak to us sans bias. But the apparent bi-partisan consensus on the 'wrong' answers to these questions is *not* due to a universal belief that we're screwed, as you put it. In this specific test you did try to let the data speak for itself, but then you ignored what it said! The data actually tells us what your initial gut feel was heading for, per my quote of you up-thread, yet you later rejected this because that direction contradicted your prior!

So yes... the Rep / Cons (largely) believe that climate scientists do indeed put out scare stories. This is more or less their public attitude regarding the unfaked issues, it is hardly a stretch that they would think the same regarding your faked issues too. Although it's anecdotal the media is full of such scare stories, some of which are far more outlandish than your faked ones. Via Occam's razor this is the simplest explanation, the Rep / Cons (largely) treat all output and claimed output from climate scientists the same, i.e. they largely disbelieve it. This raises no puzzles and no contradictions; the Rep / Cons act (largely) the same way *whether or not* identity challenge is present, so this is a genuine insight regarding their thoughts.

The much more interesting question is why do the Lib / Dems (largely) appear to react the same way for this specific test? And incidentally the North-pole kerfuffle is a storm in a teacup that apart from injecting some unfortunate confusion, doesn't change the basics of what you are trying to do here and the main nature of the returned answers. Because you're trying to get a response to red herrings, I agree it doesn't happen to matter too much that one of your questions is rather off kilter; some of them are very deliberately off kilter anyhow. The great majority of both camps had a similar response to all questions, and this is the result that counts.

There are two obvious main candidates for why the Lib / Dems respond in the same fashion. a) Their emotive commitment to the imminent calamity that consensus climate science forecasts is so deep, they have sacrificed all skepticism in this domain and *themselves* truly believe the faked stories simply because they purport to come from climate scientists (authority) and hence are yet more indicators of general anticipated calamity. b) When identity challenge is removed and they no longer have to defend their team line, just like for the Rep / Cons, the Lib / Dems actually have little faith in the pronouncements of climate science, and likewise assume the scientists are simply putting out scare stories. i.e. they believe that *the climate scientists believe* this stuff, but they don't believe it themselves.

Your form of questioning cannot distinguish between these 2 possibilities, and further data is needed to do this. The first link in my post upthread demonstrates one means of resolving the possibilities. It turns out that groups representing BOTH possibilities are present within the Dem / Libs. The b) group is much the most interesting, because these are the ones who are changing their responses depending on whether there is or isn't identity challenge in the questions! The a) group are ideologically committed to the concept of climate calamity, and hence happen to be Dem / Libs as well as adherents to climate culture. They do not change their responses depending on identity challenge, because their core belief is so strong they adhere to it both personally and publicly. Public survey data shows that the b) group is at least as large as the a) group, and likely larger (possibly much larger), meaning that even within the Dem / Libs, only a *minority* truly believe in the concept of imminent calamity from AGW.

Your own logic train misses all this simply because you are glued to a hard prior that blindsides your analysis of what is excellent (and excellently collected) data. This is why you end up with 'weird puzzles' and apparent paradoxes, when in truth they don't exist. They are artifacts of a flawed analysis. Yet in the end, you cannot live in isolation. There is other great data out there, and while much of it does not have the same quality or resolution as your own, there is masses of the stuff and it does speak to us. For instance your conclusion of a 'universal latent fear that we're all screwed', completely contradicts every# single survey in the US that ranks the climate change issue with other issues for the president or the government to consider. Depending on the number of issues (typically about 6 to 20), CC is always dead last or very low. Revealingly, even where such surveys have a political breakdown, CC still comes low or very low *for Lib / Dems only*. (Priority lists are another good way of diminishing identity issues, which is why the Dem / Lib numbers crash from their isolated 'belief in dangerous AGW' responses). There is similar support from US surveys emphasizing the costs of action. Global surveys give similar results too, CC is dead last in the 8 million people UN pole, for instance. Given that important links in your logic chain like this are completely undermined by such data, surely reappraisal with a view to consistency between data sets, at the very least, is necessary. This example is not a marginal inconsistency, it's a head-on collision.

# or at least every one of the many I've checked out spanning the last 5 or 6 years. None buck this trend.

February 9, 2016 | Unregistered CommenterAndy West

@Paul Matthews

I think Dan's motives are good. I think he tries hard to be objective. I think he has collected excellent data that is way above and beyond what I've seen elsewhere, and is most certainly trying his damndest to bottom out the 'weird' problems this process seems to have uncovered for him. But those problems / paradoxes are nevertheless an artifact of a flawed analysis. This is turn is due to a hard assumption injected into the analysis, rather than giving the data full reign and letting it show us where the cultural biases lay. The links above to Climate Etc (I think you've read), explain in full detail how this occurs. I am not impugning motive or intelligence or indeed effort (the efforts here are huge and impressive!)

February 9, 2016 | Unregistered CommenterAndy West

most of their public get their climate science via the media.. in the UK we had climate change will cause dogs to get depressed - just this week.

February 9, 2016 | Unregistered CommenterBarry Woods

@Dan, @Matthew,

I see from above there may be confusion. Don't get wrapped around the axle by the 'underlines'. This is just mere mechanics. They happen to be used in diagrams like this: represent that the 'correct' answer is the 'consensus' one. But YOU CAN'T DO THAT when you are seeking within a domain of uncertainty to understand where the cultural bias that caused the polarization to occur, actually came from. You have to ask further questions of the data to help IT TELL YOU this. Is it from the blue side, the red side, or both?

However, Dan's assumption that the Consensus answer = 'correct', forces the only possible solution that the bias is on the Rep / Con side of the graph. So when going into later stages of asking the data for more answers, this assumption is already made, and so in turn forces peculiar interpretations of the later results, such as 'a universal latent fear that we are all screwed'. And per my quote of Dan above, a rejection of the more obvious conclusions because these contradict this same hard assumption.

I don't have any beef with the deliberate red herring questions or their underlines e.g. per the July 23rd 2014 OCSI post.

What I'm really saying is that you CANNOT assume the climate consensus answers are correct at the start of the investigation, and then expect the analysis to show you where and what the bias is all about. You have to assume no answers are 'correct' or 'incorrect', and let the data SHOW YOU what is happening.

Doing this produces a very different result.

So 'underline' is just a proxy for defining 'climate Consensus answer' as 'correct'.

Please read above armed with this clarification :)

February 9, 2016 | Unregistered CommenterAndy West

The "consensus" question appears to be respondent's agreement with a statement that "most expert climate scientists agree that human activity is warming the planet." The statement about most expert climate scientists is unproblematic and true, as just about anyone who reads the literature or goes to the meetings should know -- even if they are unaware of the convergent findings from research measuring the degree of agreement through widely different methods by Cook, Anderegg, Doran, Oreskes and others; or of the statements by every major organization of scientists.

There *is* broad agreement on that basic fact, even while disagreements exist and research continues on many finer details like how much warming, how soon, with what impacts. So, if respondents think such agreement exists, they are correct, whether from knowledge, prejudice or just a lucky guess.

February 9, 2016 | Unregistered CommenterL Hamilton


Darn. You guys were having such a nice conversation I was going to reward you with a big, juicy chunk of data, something @LarryHamilton woudl have feasted on.

But I accidentally hit the "delete" instead of "post" button.

Oh well. I'll reconstitute it.

but it defnitely wil complicate some of the positions being worked out here!

February 9, 2016 | Registered CommenterDan Kahan

Dan -

Not sure what to think about the finding...but as to whether I find it surprising....I think that it isn't particularly surprising because in the end, the questions themselves aren't informative enough to draw truly meaningful conclusions.

And in the end, I think that the discussion would be better informed by data on somewhat different questions:


1) There is solid evidence that continued and increasing aC02 emissions pose a potential risk for harmful climate change.

2) Most expert climate scientists agree that continued and increasing aC02 emissions pose a potential risk for harmful climate change.

After all, (IMO, at least) those are the more important questions.

Of course, there's always the important question as to whether actively engaged online climate combatants are representative of any larger cohort beyond the cohort of actively engaged online climate combatants. And of course, we know that it isn't (even though "skeptics" often make the mistake of thinking that it is. But that said...

Climate combatants like to focus on the sort of inadequate questions being answered in your survey in order to advance their agendas. For example, actively engaged online "skeptics" like to focus on whether climate scientists are convinced that "CAGW" is "settled science," even though relatively few climate scientists have ever made an argument that "CAGW is settled science" (or that even "CAGW" is a meaning concept, scientifically).

W/o more precise information, this kind of data presented in your survey just seems to me to be more fodder for sameosameo, where combatants try to utilize incomplete or partial data to support fully fleshed out opinions that in reality, aren't supported by the available data. I'm not sure how much the counter-intuitive or paradoxical juxtaposition you highlight really informs about the dynamics of how motivated reasoning and cultural cognition play out in the climate change/political proxy food fight.

What I know is that actively engaged online "skeptics" want to argue that there is no strong prevalence of agreement among climate science experts that continued and increasing aC02 emissions pose a potential threat...but they know that there is such a in the process of making that argument that there is "no consensus" they transform the meaning of "consensus" to be equivalent to "CAGW is settled science" and as such can find validation in arguing that there is no "consensus." Why do they do that? I suspect because probably they know that there is a strong prevalence of agreement among experts that continued and increasing aCO2 emissions pose a potential threat, but they find that "consensus" to be politically inconvenient because they don't want to accept the political associations with accepting that threat, so instead they focus on a red herring of a more politically convenient target of whether expert climate scientists agree that "CAGW" is "settled science"

On the other side of the coin, some actively engaged online "realists" exploit the ambiguity of the term "consensus" to translate a strong prevalence of shared opinions among climate science experts that continued and increasing aC02 emissions pose a potential risk, to give the public an impression that "CAGW" is "settled science." (IMO, that distortion among "realists" is probably not as uniform or pervasive as is the previous distortion I described coming from "skeptics").

February 9, 2016 | Unregistered CommenterJoshua

Dan -

I know that you don't find much value in speculating about plausible explanations w/o relevant and informative data, but...

If I can force you to guess, do you think that you'd find a similar (what you consider to be paradoxical) pattern play out with the associations between political views and beliefs about evolution/scientific expert opinion on evolution...or nuclear energy....or other issues that display a similar pattern of association between political orientation and interpretations of scientific evidence/how experts interpret that evidence?

February 9, 2016 | Unregistered CommenterJoshua

"I think it woudl be a useful thing for me to identify competing hypotheses & then I & others can think of observations that can be made that would give us more or less reason to believe one or the other of those conjectures is right"

Great! There is one described here, as a contrast against your own:

February 9, 2016 | Unregistered CommenterAndy West

...with more support from data for higher educated US Independent voters, here:

February 9, 2016 | Unregistered CommenterAndy West

...and followed up by a very simple 3 step social analysis showing that Creationism is a cultural phenomenon, and using *the same* 3 steps, that the belief in imminent calamity from AGW is also a cultural phenomenon. If one doesn't like the 3 steps for the latter, one has to say why they're also wrong for the former.

February 9, 2016 | Unregistered CommenterAndy West


See Kahan, D.M., Jenkins-Smith, H. & Braman, D. Cultural Cognition of Scientific Consensus. J. Risk Res. 14, 147-174 (2011).

Precisely b/c beliefs about contested risks and perceptiions of scientific consensus are so tightly correlated -- are best understood as simply alternative indicators of a single latent pro- or con- affective orientation-- I find the loosening of the "happening"-"consensus" link on climate change pretty amazing. Particular the loosing among the highest in OCI.

See also the latest AAAS-Pew survey for public perceptions fo what scientists believe on various issues, incliuding evolution

I love conjecture -- so long as it doesn't impersonate evidence-based inferences.

February 9, 2016 | Registered CommenterDan Kahan

@AndyWest-- I think your explanation rests on a false premise: that the correct answers are correlated with being liberal or with belief in climate change. Items that have that quality are going to be confounded with "identity"; the point of the scale development was to avoid that

February 9, 2016 | Registered CommenterDan Kahan

@Dan, your assumption is wrong.

My notion is to let the data TELL US what the correlation is, and not have any preset stance as to what this might be. Follow the logic train I note above to see this in action. Your great data, along with other data from public surveys, tells us that there is an independent cultural entity based on belief in calamitous climate change, and that this entity has alliances with both Dem / Libs (greater) and Rep / Cons (lesser).

Far from being a confounding factor, identity protective behavior is a key flag which helps us to navigate to the origin of the dominant cultural narrative in play. Your invaluable work in separating issue from identity helps us to see specifically that the group b) Dem / Libs noted above are the ones who alter their responses when identity challenge is switched on and off, whereas the Rep / Cons (largely) maintain their stance when identity challenge is switched on and off. For a visual representation of the cultural map, see Figure 6 in here:

Your own investigation arbitrarily assigns a correlation at the beginning; in your graph on the polarization of the more domain knowledgeable, you define a 'correct' answer to your questions, which is equivalent to assigning 100% of the cultural bias (and hence associated identity / identity protective behavior) to the Rep / Cons. This is an exercise of omnipotent knowledge that is simply not viable is a situation where everything is uncertain and you have to let the data speak. This act confounds the rest of your investigation, waterfalling down into the next stages where you are attempting to find Rep / Con behaviors (identity protective or otherwise) that fit the hard constraint you set at the start. This blinds you not only to the fact that Dem / Libs might display such behaviors, but also that there is cultural identity not even on the left-right political axis at all, but on a different, right-angled axis. You end up only with paradoxes and 'weird problems', unsurprisingly.

February 10, 2016 | Unregistered CommenterAndy West

@Andy West--

I still don't get it.

How is treating "false" as the correct answer to the statement "climate scientists believe the increase of atmospheric carbon dioxide associated with the burning of fossil fuels will reduce photosynthesis by plants" biased against conservatives?

Conservatives are likely to believe that. But no more so than are liberals. That's what the "data tell us."

February 10, 2016 | Registered CommenterDan Kahan


I make very explicit what the issue is with your hard prior assumption, and it is not within the example you give. As noted multiple times above, your red herring questions are fine, and the results from them perfectly valid and very interesting. It is the interpretation of those results (and others), i.e. your proposed explanations for the apparent bi-partisan consensus, that go wrong, because of the hard prior you already baked in from an earlier stage. You have essentially ruled out a lot of what the data tells us before you even got to the red herring stage. The explanations you put forward are hence not the best fit for the data, and are indeed a very awkward fit (as evidenced by your resultant 'weird problems').

Also as noted multiple times above, the baked-in prior comes from assuming the climate Consensus answer is the 'correct' one at the earlier stage where you discover the polarization of the more knowledgeable. This polarization is a classic sign of ideological / cultural bias, yet your assignment of 'correct' to one side automatically colors the rest of your investigation to only look for that bias on the opposite, Rep / Con side, and this is what drives the whole scope of your explanations above. Without omnipotent knowledge, one must assume that the bias could be on either side, or indeed both, and you haven't done that.

February 10, 2016 | Unregistered CommenterAndy West

@AndyWEst-- Thanks for helping me to try to understand. But I still think that there's somethign off in your object.

Lots of quetsions are ones where wrong answer "looks right" to someone who doesn't know the answer; they aren't red herrings but items that try to discriminate between people answering on basis of a general affective "we're screwed" outlook & those answering on basis of knowledge. Without those, the test would for sure be biased toward whichever group's "affective orientation" lined up with the right answers. the nice thing about the instrument is that it turns out no one's does.

In any case, I don't think anything in the covariance matrixes, which are what the data use to "tell" us whether the scale is measuring anything and what it might be, care what the "right" answers are. You can treat as "incorrect" every question I score as "correct" & get the same result, save that the peopole who are lowest in science comprehension (regardless of cultural identity) will (strangely!) have the highest scores.

*Which* questions in OCSI_1.0 or in the candidate 2.0 set make any sort of assumption that anyone is biased? Is it biased to score as "true" the statement "climate scientists see CO2 as a gas that causes the temperature of atmosphere to increase"? that "climate scientists believe rising global temperatures from CO2 emissions will cause flooding in certain regions"? Skeptics and nonskeptics were just as likely to get those questions right as they were to get the harder, "risk-concern affect" questions *wrong*.

If you want to say that the expositional or interpretive or normative framework I use has a normative orientation built in, that's a different story. I might demur; I have normative commitments that I try not to disguise. The principal one is that science should be used by democratic societies to enable free & reasoning people to recognize and give proper effect to the best available evidence in deciding *for themselves* what sorts of policies, reflecting whatever balance of interests best fits their own values, will achieve the ends they prefer. I'm confident that in the current science communication environment no one can confidently sort that out, no matter what their "cultural identity."

Beyond that, I might well have commitments that figure in my analsysis, either consciously or unselfconsciously b/c of insufficient self-awareness.

Sure tell me about that (I guess you are; you are saying that I am committed to seeing one cultural group's view stigamized as biased-- I don't accept that, though).

I don't think, though, that to be a scholar -- to conform to the norms of the craft & to do what I do in a way that respects the reason of others -- that I have to eschew normative commitments. I on ly have do work that is valuable to other reflective people *regardless* of *their values.*

I won't do a perfect job, I'm sure. But if *you* are getting value (as I take it you are; I appreciate, in fact, that you are telling me so at the same time that you very appropriately tell me where you think I'm going wrong) & *so are* readers that have very different commitments from yours, then I feel I'm getting somewhere.

February 10, 2016 | Registered CommenterDan Kahan

Andy -

==> " Your invaluable work in separating issue from identity helps us to see specifically that the group b) Dem / Libs noted above are the ones who alter their responses when identity challenge is switched on and off, whereas the Rep / Cons (largely) maintain their stance when identity challenge is switched on and off.""

Seems to me that the basic principles of motivated reasoning, confirmation bias, cultural cognition, group-think, etc.,illustrate how these patterns in how people reason apply w/o differentiation by ideological orientation. They are rooted in fundamental aspects of human cognition (pattern recognition) and psychology (identity protection).

These principles also suggest that it is entirely normal for people to think that these phenomena are more likely to be manifest among "others" and not within one's own identity group. Thus, your determination that Dan's interpretation of the evidence is biased by his priors (when he says that the manifestation of these phenomena isn't disproportional in one ideology group as opposed to another) whereas your interpretation of the evidence to show that Republicans, conservatives, and "skeptics" are less prone to these phenomena (in an issue where they are highly identified, no less) seems more than just a tad ironic.

Of course, we can't be highly confident if we assume bias by ideology, and it would be a mistake for me to do so in your case (just as it would be a mistake to do so in Dan's case - even though I can point you to many cases where "skeptics" have assumed bias in Dan's case by labeling him as a "liberal" or "AGW zealot" or whatever)..

But maybe at some point you will produce evidence that has been scientifically collected and analytically tested that can prove your theories about the asymmetry in manifestation of motivated reasoning within the ideological matrix of opinions on climate change.

February 10, 2016 | Unregistered CommenterJoshua

A request for guidance.
I am not trying to make sense of the behavior of groups of 1,000 or 1,000,000 people at a time.
I am trying to make sense of individual people and individual brains, neuron by neuron and synapse by synapse.
If you want to help me go faster, contact me. I would love the help.

February 10, 2016 | Unregistered CommenterEric Fairfield

>> Thanks for helping me to try to understand.

Apparently I’m not trying hard enough ): Am I inadvertantly talking in Turkish or Ukranian? I don’t get why you keep returning to issues that I’ve repeatedly said are not the ones I have a main beef with, and repeatedly missing the one I keep raising as the most salient problem. Maybe this is because I’m explaining the stages of your investigation (from my perspective) in the same time order and with specific post details as I saw them come up at your website, and it now occurs to me that this may not be the time order you actually progressed in (especially as I didn’t track all posts). I’ll lose the time order references and make my explanation more generic. And just in case, again, I agree that disentangling people’s cultural identities from their comprehension of climate change is key; in fact it is completely fundamental to discovering what I think “is going on”, as you might say.

Re the last part of your post: yes you are getting somewhere, yes I think your efforts here (on climate and other topics) are extremely impressive, yes we are all trying our best :) But no I don't think I've gotten through what it is I mean yet, regarding the way bias has crept in. No doubt this is my transmission as much as your reception. (Capitals below don’t mean I’m shouting, just can’t be bothered with html / italic emphasis).

For about the 3rd time, I don’t have an issue with your red herring questions (OCSI either version) and (notwithstanding some minor confusion / issues) the way they are designed. Yet the bulk of your last answer seems to focus yet again on this particular area. And okay I was using ‘red herrings’ as slang, but I know your intent regarding the OCSI test and that intent seems perfectly fine afaics. I think I first saw the OCSI as the sixer chart at the following link, where you also discuss that 'the overwhelming majorities of *both* Republicans and Democrats think that “climate scientists believe” that human-caused climate change poses all manner of danger to people and the environment':

So armed with those clarifications, let’s start with this image here, which shows the increasing polarization of the more domain knowledgeable:

This is great place to start, because polarization of this kind is a classic sign of the presence of strong cultural bias. For instance you get a similar picture with your studies on creationism (the gradients are a bit different, because creationism is an old and ‘received’ culture). However, YOU are the investigator here, within a domain where everything is uncertain (or must be assumed so). The very point of your exercise to to assess bias in an environment where the consensus is challenged. You CANNOT assume omnipotent knowledge. Hence you cannot assume you know where the detected bias comes from. It may come wholly from the blue side [A], it may come wholly from the red side [B], it may come in some relative proportions from BOTH sides [C]. (The same chart for a religious schism, for instance, would be a case of C).

BUT YOU DO make this omnipotent assumption. In your diagram, one of the answers is underlined as ‘correct’. This forces the implicit assumption that ALL the bias is coming from the red side. How can it be otherwise, when you have DEFINED the blue side to be ‘correct’, hence necessarily unbiased!

No doubt it is unconscious, but this assumption waterfalls into everywhere else in your investigation. You have just seriously limited your pool of available solutions to "what is going on" from A, B, C, to B only, and this *does* pervade your thinking and does derail your process.

So for instance lets return to the sixer chart above and the results it pulled from the general public, which somewhere you describe as a 'bi-partisan consensus'. The data is what it is; given I'm happy (enough) with the question set, I'm also happy with the data. Yet when *interpreting* that data there is a wide pool of potential solutions which may explain it, some of which are a much better fit than others. A pool that should *all* be available to you. But it isn't all available to you. You only have B, not all of A, B and C. I.e. any solutions that may need cultural bias from both red and blue together, or blue alone, in order to work, are excluded. You have pre-filtered the possibilities with this prior, gagged the data.

Fortunately, because you are studious in expressing your thought process, we can actually see this happening. At one point you say: "Someone could say, reasonably, that asking people what they think 'climate scientists believe' is different from measuring whether those people themselves believe what they [sic] climate scientists have concluded." This happens to be the right path to the best fit solution; some of the respondents are indeed saying what they believe climate scientists believe, but *not* what they believe themselves. HOWEVER you then abandon that path, essentially because it is a native of the A or C part of the pool that you already rejected. That path contradicts your strong prior already baked in, this is why you reject it.

Partially related is a preference to see solutions that fit a simple model of calamity from AGW, which drives towards a *single* reason for the 'bi-partisan consensus', i.e. a "universal latent fear that we're screwed". But even the wording 'bi-partisan consensus' reveals an invalid assumption. The data says that most Reps and Dems poll in the same direction; but this does not imply that they all do that *for the same reasons*, i.e. it may not be a consensus as such, but a coincidence of *different* motives. There may be more than one group even within the same political camp. In practice you need further data to distinguish whether and what the groupings are. You can see this identification at the links I posted above.

So the right approach to interpretation of your fantastic data, is first to unbake your fixed prior and not assume you know where the cultural bias is coming from in the chart where the high OCSI scores are more polarized, taking care that you remove this assumption from all other parts of your investigation. This allows the wider range of solutions, that as it happens *do* contain a much better fit to the data, with no left-over 'weird problems'. You will find there are THREE different groups that all poll in the same direction; 1 rep / con and 2 dem / lib groups. The rep / con and one of the dem / lib groups have a common behavior, the other dem / lib group has a different behavior that nevertheless polls the same way.

Along the way you need further data to figure out where the cultural bias actually does originate from, and a key way to do this is to see who changes their position when identity challenge is or isn't applied. Those who change their responses with this switch are in alliance (for team identity reasons) with a core culture, but don't truly believe in it, yet their behavior helps identify what *is* the core culture. Turns out the origin of bias is not actually from the left-right axis anyhow, but expresses there through alliance and overlap.

Did I get through the Ether?

February 10, 2016 | Unregistered CommenterAndy West


The characteristics you cite are indeed universal. They do not apply differently to any group. No group, whoever they are, is 'less prone', and I make no such claim that they would be.

Dan is the investigator in this endeavor. He must rise above all related groups. Likewise for me in the same endeavor. No, it's not at all easy for anyone to do.

An ultimate outcome of this trail for Dan, is the concept of 'knowing disbelief'. It has to be a mass phenomenon to do what Dan claims. I say it cannot be a mass phenomenon, and further, it isn't needed to explain the data. Dan's claim of 'knowing disbelief' is also made regarding his investigation into Creationism. My claim that it can't be a mass phenomenon, and further isn't needed, also remains the same for the Creationism domain. Yet in this domain, Dan and I are nominally in the same group. The investigator should in any case attempt all possible methods that rise above groups, yet clearly it is not group affiliation that can be making a difference here.

February 10, 2016 | Unregistered CommenterAndy West

P.S. as to evidence, though I use many public surveys too, I find Dan's data to be the best available evidence, which provides strong support for my case :) Follow the links.

February 10, 2016 | Unregistered CommenterAndy West

@Dan writes

as relatively “right-leaning” individuals become progressively more proficient in making sense of scientific information (a facility reflected in their scores on the Ordinary Science Intelligence assessment, which puts a heavy emphasis on critical reasoning skills), they become simultaneously more likely to believe there is “scientific consensus” on human-caused climate change but less likely to “believe” in it themselves!

As I wrote earlier, it has long been talking point of those attacking the idea of AGW that consensus is anathema to science, so yes, they constantly talk about this consensus and that it represents "groupthink" or the mindset of the persecutors of Galileo, and so of course they -- that is the hardcore cognoscenti of climate change denial -- all talk about the consensus and all virtually treat it as more evidence that the science is corrupt. Those in conservative and libertarian circles who had only heard the occasion denunciation of climate science may never have heard of a 97% or any kind of consensus.

For some idea of how this group constantly reinforce each other, you might look at the recent "(Mis) communicating science in public controversies" at and the 328 comments that follow.

The main article concerns a study of German scientists who were found to to some degree tend to worry about communication to the public of studies that publishing some results in the media '“could be exploited by interest groups” and “misinterpreted in public” – though the latter difference is not significant.'

In the following 328 comments, discussants have a huge tut-tutting party over this, and along the way through a process like the game of telephone, transform the partly slightly significant results into proof of the perfidy of climate scientists.

Or you might turn to one of the more technical articles in "Climate, Etc." like with its only 137 comments to get the picture of a bunch of high scorers on scientific comprehension excoriating the scientific consensus. You get references to Descartes, Kant, Feynman, Stephen Toulmin’s Cosmopolis, Leibniz’s universal language, the application of mysticism to science, under the cloak of empiricism, the "whole positivist debacle…." by people who clearly know a little bit about what they are talking about, which would naturally make them look proficient in science comprehension based on some slight survey.

@Andy West, I have read and listened and relistened to your "Who is Who" technical appendix to your "Climate, Etc." article (in

Your central argument seems to boil down to "There are (a) consistent religious creations, (b) religious people (Christian or nearly all Christian) who believe in evolution, and there are (c) nonreligious believer in evolution. Since (b)s simultaneously affirm their scriptures, and hence, if they were consistent, by your reasoning should believe in creationism, we have an inconsistent or self-contradictory group which proves that the doctrine under question is what you call CD or Culturally Determined.

Now there are (A) Consistent CAGW believers, (B) those who say they believe in CAGW, but don't put climate at the top of their list of concerns, above terrorism, etc., and (C) those who don't believe in CAGW. (B)s "belief" should lead them to put climate mitigation at the top of their list of concerns, and the presence of these "hollow" believers proves that the doctrine of CAGW is Culturally Determined, since there is no similarly contradictory group of nonbelievers in CAGW (presumably they would be nonbelievers in CAGW who demand that trillions should be spent to mitigate global warming). Anybody who has listened over the years to surveys of the American people should know that things almost never get to the top of our list of concerns until something is seriously in their face. If it bleeds, it leads, otherwise it takes a back seat. Economists might be saying we are in the middle of a dangerous bubble, and a "middle band" of Americans would say they ought to know, but the economy is going well so they forget it a minute later. In 1940 the typical American view was "Hitler is really, really bad, but we can't do all that much about it without excessive sacrifice".

On the face of it, the very idea that the data should tell us all we need to know "more or less independently of domain knowledge" (see sec. 2 or your "Who is Who" -- i.e. we should set aside our assumptions and the data would tell us which position was "CD") seems very far-fetched, and an attempt at closely reading your paper hasn't improved my opinion of that.

It seems to me the guys I used to interact with at, well, the brighter ones appreciate a clever argument, and some of them even have their own elaborate theories, though they don't compare or criticize each other's theories - it is enough that any one of them "disproves" CAGW - but these theories don't have to stand up to the next group of junior scientists who, to get beyond their current low paid adjunct position must prove something unexpected. They seem to think a single "knockdown argument", to quote Humpty-Dumpty, ought to do the trick. If the next group of junior scientists really wanted to break out of their trap, they should go to Heritage or Heartland or Cato, where they would probably easily get a 6 figure salary, but these organizations can only assemble a few mediocre scientists. That's a scenario. It's not very conclusive, but I put it up against other scenarios that purport to explain how they all become part of an assembly line manufacturing elaborate lies, in many distinct fields of study, from climatology to paleo-biology to physics to chemistry to computer modeling to geology, etc.

February 10, 2016 | Unregistered CommenterHal Morris

Andy -

==> An ultimate outcome of this trail for Dan, is the concept of 'knowing disbelief'. It has to be a mass phenomenon to do what Dan claims. I say it cannot be a mass phenomenon, and further, it isn't needed to explain the data."

My take on Dan's concept of "knowing disbelief" boils down to something that seems quite obvious and common to me: People hold beliefs that are contradictory, without a hitch in their step, because of how their identity orientation influences their reasoning. Not only does that pattern seem ubiquitous to me, it also seems to me to be near universal and certainly not distributed disproportionately in line with views on climate change. Take, for example, the widespread statements by many "skeptics" that they don't doubt the basic physics of the GHE and that aCO2 warms the climat, yet that there has been a "pause" in "global warming" despite continued emissions (and atmospheric concentration) of aCO2.

==> Yet in this domain, Dan and I are nominally in the same group. The investigator should in any case attempt all possible methods that rise above groups, yet clearly it is not group affiliation that can be making a difference here.

As is not infrequently the case for me when reading your comments, I'm not able to follow your logic there (not because I assume that what you write isn't logical or clearly argued, btw)

==> I find Dan's data to be the best available evidence, which provides strong support for my case :) Follow the links.

I'm afraid that doesn't work for me. I don't see how Dan's evidence supports your arguments, and I'm afraid that trying my best to follow your arguments (and links) hasn't solved that problem. As an example,as near as I can tell you argue that Dan's evidence helps you to construct a foolproof argument explaining how a disconnect between (1) "concern" on the part of some about "CAGW" and (2) the ranking they give climate change within the overall hierarchy, relatively, of social problems, or (3) with how they go about arranging their everyday life activities in ways that don't always prioritize that concern, makes it clear that realists" views on climate change are more "culturally" or "emotionally" determined than the views of "skeptics." Yet I see no disconnect between expressing a belief that aCO2 emissions pose a risk and not knowing how to solve that problem/not raising that problem to one that prioritizes everyday activities because the risk exists on a long term time horizon and is hard to frame in tangible ways. Further, from what I can tell, Dan's evidence makes it quite clear, that there are many ways that "skeptics" are very much prone to reasoning that is heavily influence by ideological orientation (as opposed to non-ideologically and objectively dispassionate analysis of the evidence.

It would seem to me that if you want to argue that "skeptics" are, as a group, less influenced by identity and emotion in their reasoning about climate change than "realists," you should be able to design some kind of mechanistic hypothesis for why that is the case, come up with some experiment methodology for collecting and analyzing data that would support your theory, and then collect the data and write it up. FWIW, that is what I would find more convincing than for you to claim that Dan's evidence supports your hypothesis. Obviously, you don't need to be particularly concerned with convincing me, and perhaps you really have already done as I have described and I just can't follow what you've written well enough to understand that you've already done it. Apparently your arguments work quite well for convincing a number of "skeptics," so maybe that's a good enough outcome.

February 11, 2016 | Unregistered CommenterJoshua

what you say makes sense of course but it would make more if there weren't still such a humongous political effect. If it's a straightforward "what do you know" fact question I wouldn't expect those at 90th percentile or above who are on right to be struggling to hit 50%.

On the other hand, if it's a straightforward "who are you" identity question, then I also expect those at 90th percentile plus not be so far north of 0%.

@Dan I'm having a hard time understanding what you are trying to communicate in the passage above. I'm having to make guesses as the what the references to "90th percentile", "hit 50%" are referring to.
90 percentile on what? (OSI, OCSI, or ?)
"Hit %50" on what?

What sort of question is this at this point?

For sample as whole, correlation (spearman's rho) for belief in human caused climate change & agreement that there is scientific consensus = 0.48. That's what surprises me. I can go back & check, but I'm pretty sure the last time I collected data on this issue (probably 2009 or so), the connection between "scientific consensus" & "belief in climate change was much much stronger. That the correlation gets weaker as science comprehension goes up floors me -- that's not what happens for any set of items that measure identity.

If I understand what you describe as the correlation getting weaker the same as you do
then I also see that result as consistent with the opinion of a well informed climate skeptics/luke-warmers.

You seem to be saying that your are surprised that the relatively small group of people with well above average science comprehension answer more like members of the "Liberal Republic of Science" and are less influenced by their other cultural identities?

These two questions get to a key difference between luke-warmers and warmers. Do you think it's a mistake to assume that people with high science comprehension scores also tend to answer such questions with care?

My theory predicted that result. A shift in the sample as a whole towards the answer of those with the high science comprehension scores is also predicted by my theory and cultural cognition. Again, it indicates to me that the science communication efforts of skeptics is starting to have a effect at least on those of the political right. If it' is mostly a cultural cognition influence then perhaps informed luke-warmers are having more of a cultural impact then they did previously.

A "how well do you understand the thinking of the 'other side'" type survey question might ask why the "surprising" result makes sense to a skeptic/luke-warmer.

February 12, 2016 | Unregistered CommenterCortlandt

I'm an awkward fit on your graphs, being a confirmed climate sceptic who's off the left-leaning edge of your graphs, so you won't be surprised that I don't share your surprise with the results on consensus.

Ask me whether I agree that most expert climate scientists agree that human activity is warming the climate, and I may reply differently at different times, depending on my mood. Do I interpret “expert” as meaning someone who's demonstrated the basic scientific qualities of scepticism, honesty and openness? Then the answer is No. Do I interpret “expert climate scientists” to mean what I think the researcher wants me to think it means? Then the answer is Yes.

The trouble with second order questions of the form “Do you agree that most Xs agree..?” is that the duller respondents won't understand them and may give responses which are the opposite of what they intend, while the brighter ones may take the process further in their minds, turning it into a third or fourth order question by asking themselves: “Do I agree with the way this question is framed? How do I express my doubts..?” etc. This may add a lot of noise to the data, disproportionately so for the right-leaning scientifically savvy, who will understand and take into account the purpose of the research.

It's a familiar enough situation in aesthetics, where it's easy enough to get people to agree that say – Michelangelo is the greatest painter in history, though they may prefer a Degas ballet dancer on the wall. Are they confused? I don't think so. There are areas where acceptance of consensus opinion is rational, but science isn't one of them. Especially for those who've actually read Doran & Zimmerman and Cook et al. - 0.00001% of the population, but probably most of those commenting here.

February 12, 2016 | Unregistered Commentergeoff chambers

Have you published this results?

I see the link to your 2014 paper.


Not published anywhere other than here. And this is only an appetizer

Am guessing the "2014 paper" is the working paper version of Kahan, D.M. Climate-Science Communication and the Measurement Problem. Advances in Political Psychology 36, 1-43 (2015), which features OCSI_1.0. Or possibly, Kahan, D.M. "Ordinary Science Intelligence": A Science Comprehension Measure for Study of Risk and Science Communication, with Notes on Evolution and Climate Change. J. Risk Res. (in press), which examines the non-relationship between belief in climate change & belief in evolution, respectively, to science comprehension.

February 13, 2016 | Registered CommenterDan Kahan


You are noise, then, as are the sorts of rspts you imagine. If there are covariances & correlations like this, the question isn't whether the items are measuring something but only what

February 13, 2016 | Registered CommenterDan Kahan

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>