follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Polarization on policy-relevant science is not the norm (the "silent denominator" problem) | Main | Now, getting back to disgust: we've done guns & drones; what about *vaccines*? »
Wednesday
May292013

The impact of "science consensus" surveys -- a graphic presentation

I am really really tired of this topic & am guessing everyone else is too. And for reasons stated in last couple of posts, I think a "market consensus" measure of belief in global warming would be a much more helpful way to measure and communicate the weight & practical importance of scientific evidence on climate change than any number of social science surveys of scientists or of scientific papers (I think we are up to 7 now).

But since I had occasion to construct this graphic to help a group of professional science communicators assess whether the failure to communicate scientific consensus can plausibly be viewed as the source of persistent cultural polarization over climate change in the US, I thought I'd post it.  I've included some "stills," but watch it in slide show mode if you want to get the nature of the empirical proof it embodies.

And here are the answers to the predictable questions:

1. Does that mean "scientific consensus" is irrelevant?

No.

People of all cultural outlooks support policies they believe are consistent with scientific consensus.

But they have to figure out what scientific consensus is, which means they have to assess any evidence that is presented to them on that.

In the current climate of polarization, members of opposing cultural groups predictably credit and discredit such evidence in patterns that reinforce their belief that the scientific consensus is in fact consistent with the position that predominates in their cultural group.

Until the atagonisitic cultural meanings that motivate this selective crediting and discrediting of evidence are dispelled, just flooding the information market with more and more studies of "scientific consensus" won't do any good.

Indeed, it will only amplify the signal of cultural contestation that sustains polarization. 

Meanings first, then facts.

2. Does this mean we should ignore people who are misinforming the public?

No.

But it means that just "correcting" misinformation won't work unless you convey affirming meanings.  

Indeed, in a state of polarized meanings, rapid-response "truth squads" also amplify polarization because they reliably convey the meaning "this is what your side believes -- and we think you are stupid!"

Meanings first, then facts!

3. Does this mean we should just give up?

No.

The only thing anyone should give up is a style of communicating "facts" or anything else that amplifies the message that positions on climate are part of an "us-them" cultural struggle.   

The reason the US and many other liberal democracies are polarized on climate change is not that people are science illiterate or over-rely on heuristic-driven reasoning processes. It isn't that they haven't been told that human CO2 emissions increase global temperatures.  It isn't that they are being exposed to biased news reports or misled by misinformation campaigns. And it certainly isn't that no one has advised them yet about the numerous studies finding "97% of scientists ..." agree that that human activity is causing climate change.

The reason is that we inhabit a science communication environment polluted with toxic partisan meanings on climate change.

Conveying to people -- a large segment of the population in the US & in other countries too-- that accepting evidence on climate change means accepting that members of their cultural community are stupid or corrupt is itself a form of science-communication pollution.  

If you don't think that many ways of communicating "facts" (including the extent of scientific consensus on climate change) convey that meaning, then you just aren't paying attention.

If you think there's no way to communicate facts that avoids conveying this meaning, and in fact affirms the identity of culturally diverse people, you aren't thinking hard enough.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (56)

Dan, I think if their "meaning" was to get more people inside the tent, and more options on the table without making either trivial, we would be more likely to get facts on the table. But one "fact" needs to be dispelled "If you disagree with me, you are VILE." This is from a person trying to address the denier issue over at Lucia's.

One of the fundamental issues, IMO, is that people still approach the situation as though it is not a world wide failure of the commons. I mean fundamental as in the basic framework to consider a solution. Essentially all of mankind is emitting or benefitting from CO2 emissions. Everybody has a right to sit at the table. Everybody has a stake and a share in this.

This is one of the reasons that I think your hypothesis of using something different than louder, more often, more antagonistic should be better. With the range of climate sensitivity in IPCC WG1, I wonder if we shouldn't round file everything else until after we get more people agreeing on talking, with other persons listening.

May 29, 2013 | Unregistered CommenterJohn F. Pittman

In my opinion, one of the science communication failures is talking in terms of what scientists "believe". And not just because this puts the discussion into an us vs them struggle. For example, in discussions of evolution. This should not be a discussion of which do you believe?, your Bible or your biology textbook? Talking about beliefs gets in the way of talking about evidence. The science of evolutionary biology is not a faith based system.

As noted above, scientific evidence can be discussed in terms that build on things that everyday people already know and in terms of things that have the potential of impacting their lives.

May 29, 2013 | Unregistered CommenterGaythia Weis

What's interesting to me is that Dana's defense of their paper seem to represent takiing the opposite tack you recommend. He has gone full "recursive fury".
http://www.guardian.co.uk/environment/climate-consensus-97-per-cent/2013/may/28/global-warming-consensus-climate-denialism-characteristics

The defense of his paper seems to be little more than "them against us" circling against wagons.

For example: It's not clear to me that given the absense of information in the paper, Tol looked at his own paper ratings and finding the one case he looked at riddled with errors. After all: cherrypicking is hunting through evidenc provided and then finding the instances that make your point. So Tol's case seems to represent "anecdotal" rather than "cherry picking". But setting quibbling over definition aside: If Poptech and Tol's critiisms are cherry picking or anecdotal, why not show that by providing readers with a correlation coefficient between the SkS ratings and author self ratings. If that's positive, then the few cases looked at are outliers. For someone with access to the data, the calculation should take 15 minutes to code. With R you could create the scatter plot quickly too.

Why not respond substantively rather than by labeling your critics "cherry pickers"? The later can only increase any cultural divide.

But I guess they are convinced that merely publicizing their paper will somehow convert people to "their side". (And it's quite clear they are communicating here are sides in this. Dana called Tol a denier when he criticized the paper's methodology!)

May 29, 2013 | Unregistered Commenterlucia

One thing that I think might be helpful (if you've got free time to burn- doesn't everyone?!) would be for you to occasionally critique articles/documentaries/outreach efforts. Point (explicitly) to what you think are examples of good ideas and bad ideas. "It would have been nice to make point x without bringing in this cultural association." or "Point y is beautifully communicated by putting it in terms of z." Here I'm whining about theory/praxis again. I personally get "the idea", but I often struggle working out how to implement it successfully. Seeing it in concrete might knock some ideas into place.

But maybe I always feel this way... I've gotten to the point where discussions of pedagogy (especially in conference or workshop settings) frequently flash-boil my blood, because I'm sick of being told "Hey, listen to my revolutionary idea, don't be a lecturing chump- active learning is the 21st century!!!" My inner response (which I usually manage to keep to myself) is often "Yeah- duh. Now help me design an effective 'active learning' tool for soil profiles or atmospheric circulation that isn't just a waste of 15 minutes to make me feel like I'm hip to the sound."

In a somewhat similar way (not that I'm irritated with this), it would be instructive to examine communication examples. We may not have the data on how successful each example really was at communicating but we can at least elucidate how it was consistent/inconsistent with the two-channel framework you're outlining.

May 29, 2013 | Unregistered CommenterScott Johnson

"Conveying to people -- a large segment of the population in the US & in other countries too-- that accepting evidence of climate change means accepting that members of their cultural community are stupid or corrupt is itself a form of science-communication pollution."

And vice versa, I assume.

May 29, 2013 | Unregistered CommenterNiV

@NiV:

I understand "evidence of" to cover everything relevant, including any that implies "not" happening. If so, there's no vice versa; indeed, we are back to question I asked you before-- why do you think 1/2 population (or whatever it might be) overestimates; what's the communciation problem from your point of view? Is it that "skeptics" are insulting ECs?

But will change "of" to "on"-- would that help?

May 29, 2013 | Unregistered Commenterdmk38

@Scott:

As you know, I do frequently review, call attention to & analyze/critique social science papers on communication strategies, including ones that present evidence of things that seem to help offset defenseive biases.

But that's not what you have in mind, right? You have in mind examples of real-world communciation efforts.

I do point out instances of communication that I think are manifestly ill-advised given what we know about polarization on climate change. I put Cook et al itself in that category; it was desgined not to generate knowledget but to generate attention -- & the attention was of a sort that would quite foreseably deepen polarization. That made me angry, as you know, and for reasons I spelled out in my contribution to the excellent exchange Cook et al, you & I had.

I have pointed out real-world communication that I think reflects & tries to implement what we know based on empirical study. The most recent example is in What is to be done?

I am eager to do more of this. I don't think there are as many examples of people trying to do sensible climate communication, informed by evidence, as there should be. But I will make an effort now to highlight as many examples of people trying to use what we know to do what they reasonably hypothesize will work. This is a very good suggestion.

But one thing I wouldn't do is engage in a sort of movie-critic style of analysis -- "this was done well; ouch-- awful" -- w/r/t those sorts of efforts.

I wouldn't do it b/c I think the right way to analyze them is by collecting evidence. Someone like me should go out to where people are communicating & help them form strategies that reflect evidence-based hypotheses, & then measure the effects of those strategies & retport the results. (No one should agree to do this, either, unless the field communicator agrees the results can be shared; this is a corollary of the ethical obligation of scientists of science communication to help anyone seeking to promote public engagement with science and to refrain from helping anyone whose goals are contrary to that.) I *am* doing that & will tell you something about it when I actually have something to report.

But what people who study science communication empirically shouldn't do is act as if one could deduce from empirical lab studies of the mechanisms of science communication determinate communication strategies for particular situations. That's bogus; it's not evidence based & is in fact is story telling abuse of decision science. I've written about this many times too.

May 29, 2013 | Unregistered Commenterdmk38

is it your position that 97% consensus among experts proves that increases in the earth's temperature are due to human changes in the environment?

May 29, 2013 | Unregistered Commenterjim luther

@Jim:

You--talking to me? You talking to me? You talking to me?

You must be confusing me w/ NiV or Lucia. They are the ones who've been arguing that.

May 29, 2013 | Registered CommenterDan Kahan

"If so, there's no vice versa; indeed, we are back to question I asked you before-- why do you think 1/2 population (or whatever it might be) overestimates; what's the communciation problem from your point of view? Is it that "skeptics" are insulting ECs?"

I don't know, and I don't think so.

I've come up with several hypotheses, but I only have anecdotal evidence for them. I think it's largely due to the way events can be easily fitted into popular narratives - and being fitted to certain of the characters in those narratives precludes further communication.

"But will change "of" to "on"-- would that help?"

My comment wasn't intended as criticism or an admonishment. I thought it was interesting that the extract seemed so astonishingly appropriate when seen from the other side - an admirable insight - and yes, to remind you that there is another side, from which some of us look at these same events and seek the same sort of answers. How can we best communicate to you why we think what we do?

But yes, it helps.

May 29, 2013 | Unregistered CommenterNiV

@NiV:
Well, now w/ this business about assuring me you meant no offnese -- you are obviously in fear after seeing how I handled @Jim's query, no doubt.

May 29, 2013 | Registered CommenterDan Kahan

Jim,

I think it was Dan's position that 97% consensus is evidence of there being evidence that increases in the temperature are human-caused. Or something like that. Evidence is not proof.

Lucia and I (if I may presume to speak for her) think that it's the wrong question to ask - we all agree that some of the increase is human caused, the argument is about how much, and whether it's dangerous.

And just incidentally, we think the survey was badly done and gives the wrong answer, and that consensus is a poor argument anyway. I don't have any particular problem with doing surveys on the topic, nor with the idea that my view is in the minority - it's an interesting question for sociological if not for climatological reasons - but these particular surveys don't help. (I tried asking Dan about a rather better survey a few posts ago, but the discussion had moved on by then so I got no reply.) As usual, communication is prevented by both sides assuming the other side's objections are down to having an agenda, so I don't think this line of discussion is going to get anywhere.

Dan,

I confess that I play both games at different times - sometimes I aim to offend people and wind them up, and other times I try diligently to avoid it. The latter sort of conversation I find more enjoyable, and I usually only resort to the former when the other option has been ruled out. On this occasion, it truly wasn't meant as a criticism. I can see how it came across that way.

I'm not sure what to make of your response to Jim. It's far politer than it might be, but it doesn't answer the question. Deliberately so, I assume?

May 29, 2013 | Unregistered CommenterNiV

@NiV & @Jim:

I believe that only the opinion of people who are willing to put cold hard cash on the line can prove things.

Who could possibly believe that a nose count of scienitsts "proves" anything? No scientist would; such a position would be contrary to the logic of scientific discovery, which compels us to treat all conclusions arrived at by empirical testing as permanently provisional-- "or something of that sort"-- even when one "proves" them by one's own empirical testing.

Do you know what one calls a scientist who challenges "scientific consensus"?

A scientist.

A nose count of scientists can't even "prove" what % of scientitist believe something insofar as how to measure such a thing will be subject to competing theories & to fallibility in measurement & hence always open to being challenged.

Of course, if I have to make a practical decision (or am simply curious about something) I might well find it very useful to know what scientists w/ expertise on a matter know, particularly about a matter on which I myself don't have expert knowledge. Certainly the weight of scientific opinion also will be relevant to me or any other reasonable person in that situation. Accordingly, I'll avail myself of reliable evidence of what experts believe.

Is it even interesting to report this? These are banalities, I take it.

I thought @Jim was trying to make me out to be some sort of sissy, so I felt obliged to puff my self up & scare him away, that's all.

May 29, 2013 | Registered CommenterDan Kahan

Here are two important blog posts on science communication:
Simon Donner on how climate activists misinform, leaving an open goal to the sceptics.
Steve Goreham on "Is it a failure to communicate, or faulty climate science?". It's the same with the Cook paper of course, the results of which you have accepted without question.

May 30, 2013 | Unregistered CommenterPaul Matthews

http://www.medscape.com/viewarticle/472896_5

Scott, this is a good resource, but is somewhat specialized.

May 30, 2013 | Unregistered CommenterJohn F. Pittman

Who could possibly believe that a nose count of scienitsts "proves" anything? No scientist would; such a position would be contrary to the logic of scientific discovery, which compels us to treat all conclusions arrived at by empirical testing as permanently provisional-- "or something of that sort"-- even when one "proves" them by one's own empirical testing.

Do you know what one calls a scientist who challenges "scientific consensus"?

A scientist.

A nose count of scientists can't even "prove" what % of scientitist believe something insofar as how to measure such a thing will be subject to competing theories & to fallibility in measurement & hence always open to being challenged."Of course, if I have to make a practical decision (or am simply curious about something) I might well find it very useful to know what scientists w/ expertise on a matter know, particularly about a matter on which I myself don't have expert knowledge. Certainly the weight of scientific opinion also will be relevant to me or any other reasonable person in that situation. Accordingly, I'll avail myself of reliable evidence of what experts believe.

IMO, we could use those paragraphs as a simple test of motivated reasoning in the climate debate: Just evaluate whether an individual argue with what you say there.

IMO, there is nothing arguable about what you wrote, but yet we have: (A) realists on one side arguing that prevalence of expert opinion is (or at least should or could be) dispositive, and we have (B) "skeptics" arguing that prevalence of scientific opinion is irrelevant (amusingly, even as they focus much energy on proving that realists quantify it improperly) or even that evaluating the prevalence of scientific opinion is at cross purposes with a non-expert evaluating the debate.

As for two follow-on follow-on arguments:

(1) I think it should be obvious that while information about the prevalence of expert opinion might sway the opinions of whoever isn't already convinced one way or the other, we know from evaluating how humans reason in the face of controversies that most folks will only use information about the prevalence of expert opinion to solidify their pre-existing viewpoint. On the other hand, while it seems to me that in the end hammering people about the prevalence of expert opinion might wind up having an effect opposite to that desired by "realists" - i.e., convince some folks that they're being sold a bill of goods - the number of non-previously convinced folks likely to be so-affected is likely to be small.

(2) On the flip side, it seems to me that "skeptics" arguing about the prevalence of expert opinion, or further, exaggerating the amount of disagreement among experts (as to whether anthropogenically-influenced climate change presents enough of a potential threat to merit careful consideration of policies aimed at some combination of mitigation and adaptation) also has a similarly limited impact on public opinion (although, I would guess, a larger impact than the impact of "realists" hammering away about "consensus"). In balance, such efforts will also primarily serve to solidify pre-existing opinions.


So what I see is fallacious arguments (A and B) being employed to little effect (1 and 2).

Same ol' same ol' in the climate wars - same as it ever was.

May 30, 2013 | Unregistered CommenterJoshua

sorry, I left the scare quotes off my first use of "realist." My motivated reasoning made me do it.

May 30, 2013 | Unregistered CommenterJoshua

You must be confusing me w/ NiV or Lucia. They are the ones who've been arguing that.
You think I've been arguing that the existence of a consensus would prove the consensus position is true?! Nope.

I also don't think Cook's paper is useful for quantifying the number of people who believe "the consensus position", nor even figuring out what the position is.

May 30, 2013 | Unregistered Commenterlucia

Joshua
(B) "skeptics" arguing that prevalence of scientific opinion is irrelevant
Could you clarify? You think some "scare quote skeptics" are arguing it is irrelevant to what issue?

I know when criticizing people's views, some people like to omit details describing what those being criticized actually claim. Or possibly, some people's cognitive filters make them blind to the notion that a claim of irrelevance must apply to some specific issue. But I think many, though not all, "scare-quote skeptics" either say the issue they think it's irrelevant to, or that issue is communicated by context. (Exception: Criticism on Twitter often lack detail. )

Depending on the issue the existence or size of the consensus could matter or it could be utterly irrelevant. Consequently there is absolutely nothing inconsistent with thinking the existence of a consensus is irrelevant to some one issue, relevant to some other issue and pointing out that someones claim to have measured the degree of consensus is flawed. The only way you can make it sound inconsistent is to make it appear that a claim of "irrelevance" is a universal feature applied to anything and every conceivable issue.

May 30, 2013 | Unregistered Commenterlucia

Lucia -

Could you clarify? You think some "scare quote skeptics" are arguing it is irrelevant to what issue?

A great question. No doubt, I am subject to biasing influences on my interpretations.

I have seen a lot of posts and comments from "skeptics" that list or highlight specific occasions when the "consensus" was wrong, or where the "consensus" was biased by self-interest, political influences, corrupted powers, self-protective instincts, etc., without a concerted effort to place those examples in the larger context (i.e., the many times that "consensus" was right or where skeptics wrong or where "skeptics" make a basic assumption about the correctness if consensus - where in fact they their lives largely based on such an assumption).

I interpret that sort of argumentation as a selective analysis - with an implication that "consensus" is, at it's heart, a flawed criterion to use for measuring probabilities.

Of course, "realists" set themselves up for such argumentation by not recognizing, up front, that the prevalence of expert opinion is only useful for weighing probabilities, but not in itself dispositive. Similarly, "realists" set themselves up by not being more careful in how they define the "consensus" opinion of "experts," or who qualifies as an "expert." And of course, in a related way, "skeptics" set themselves up by exaggerating disagreement among "experts," using fallacious arguments vis a vis the relevance of "consensus," etc.

Please note - sometimes I forget to use "some," as in some "skeptics" or some "realists." I don't want to take the time to go back and put that in for what I've already written - but I recognize my tendency to fallaciously generalize or imply "guilt by association,." Same thing with using "IMO" to distinguish fact from opinion.

May 30, 2013 | Unregistered CommenterJoshua

Joshua,

First: I think you may be over interpreting the argument-- though in some cases that may be the argument. The consensus can be wrong is a correct statement: It can be. Unless they actually advance the argument you interpret, your interpretation may amount to nothing more than an attempt to read their minds while wearing your cognitive bias blinders.

It is true that a small fraction of people likely think the consensus even doesn't even us what is probably true, but I suspect if you asked, they number few. One hopes the number who think we must believe the consensus is always correct is equally few. But the fact that the consensus is sometimes wrong does not prevent some people arriving at my blog from posting "persuasive" arguments that appear to amount to nothing more than. "This is the consensus position. You are required to pledge 100% belief in the consensus position". Obviously, because that statement is simply wrong, people will often respond: The consensus can be wrong.

Whether the argument develops more nuance is a matter of chance, particularly at blogs where people pop in at out under the best of circumstances.

I see you've set on "realists" as the descriptive label for your position? Heh. ;-)

May 30, 2013 | Unregistered Commenterlucia

Lucia -

Some more thoughts:

I know when criticizing people's views, some people like to omit details describing what those being criticized actually claim.

This is certainly true. In fact, I'd say that your "some" is a pretty weak qualifier - I would say most, and most of the time (when heavily engaged in contentious debate). Including myself.

Accordingly, I will say that I don't know that I've ever read a "skeptic" say, precisely, that a prevalence of "expert" opinion is always completely irrelevant. In fact, my point is that while I would imagine most "skeptics" would not make such an argument per se, many do present arguments that seem (subjectively speaking, of course) to rest exactly that on such a logical implication. Again, I am referring to rhetoric such as compiling long lists of when the "consensus"" was wrong on issues without attempting to add the wider context of when it has been right. Or pointing out, over and over, specific examples of when it was wrong (plate tectonics, anyone?), as if that suffices for an argument (implying that being wrong sometimes but being write perhaps far more often are somehow mutually exclusive).

Of course, that knife cuts both ways - as I don't know that I've often seen "realists" argue that the prevalence of expert opinion is always, or even in any specific instance, 100'% determinative - although the logical implication of the 97% argument is just that (and as such, appropriately identified as fallacious by many "skeptics" although they often over-egg the pudding when doing so).

May 30, 2013 | Unregistered CommenterJoshua

@Lucia:

I think it was as reasonable for @Jim to attribute that position to you as me! (Did you see how effectively I scared him off? Impressive, no?)

I am curious: does anyone disagree w/ my answer to @Jim? I'm assuming no reasonable person would -- and that neither the "typical" climate change "believer" or "skeptic" would disagree -- w/ any of it.

Including my point about the relevance of the weight of scientific consensus to a reasonable person.

Specifiically, shouldn't a reasonable person treat a qualified scientist's position as relevant evidence on a question of fact that turns on scientific expertise? And shouldn't he or she treat the existence of widespread agreement among like experts as relevant?

Only "relevant evidence" -- evidence that makes it more reasonable to believe a proposition is true than one otherwise would have (in Bayesian terms, likelihood ratio ≠ 1)-- is admissible in US legal proceedings.

In such proceedings, a party in litigation can solicit testimony from a single expert on a matter that depends on scientific knowledge not in possession of ordinary citizens. And that party can also put into evidence that a particular proposition is widely accepted within a relevant scientific field. E.g., that the likelihood of a "random match" between traces of DNA from a piece of evidence and a suspect's DNA is 1 in 10^6 etc.

So my banal report on "what I believe, Jim," is foundational, really, in the US litigation system, which we know is infused to the core with the logic of scientific proof.

@Joshua thinks agreement or disagreement w/ my statement is an acid test of motivated reasoning.

Surely no one here could fail it!

I'm sure even NiV & Larry agree w/ me -- & that sort of consensus would be truly extraordinary for this site.

May 30, 2013 | Registered CommenterDan Kahan

"In such proceedings, a party in litigation can solicit testimony from a single expert on a matter that depends on scientific knowledge not in possession of ordinary citizens."

OK, let's say that they do. And lets say that the defence exhibits an email from the prosecution expert witness discussing a submission contradicting their evidence in which they say "It is also an ugly paper to review because it is rather mathematical, with a lot of Box-Jenkins stuff in it. It won't be easy to dismiss out of hand as the math appears to be correct theoretically". A second email discussing further falsifying data is exhibited with the postscript: “p.s. I know I probably don’t need to mention this, but just to insure absolutely clarify on this, I’m providing these for your own personal use, since you’re a trusted colleague. So please don’t pass this along to others without checking w/ me first. This is the sort of “dirty laundry” one doesn’t want to fall into the hands of those who might potentially try to distort things…” An independent reviewer examines the work and reports "For example, authors reported high confidence instatements for which there is little evidence, such as [...]. Moreover, the guidance was often applied to statements that are so vague they cannot be falsified. In these cases the impression was often left, quite incorrectly, that a substantive finding was being presented."

The results are based on a database of DNA records - the person responsible for maintaining that database is found to have privately said of a measure in which they corrupted the database by making up false IDs when inconsistent records were detected: "In other words, what [we] usually do. It will allow bad databases to pass unnoticed, and good databases to become bad, but I really don't think people care enough to fix 'em, and it's the main reason the project is nearly a year late."

It is reported that there is widespread agreement amongst experts that "the errors don't matter."

In these circumstances, should a reasonable person treat a qualified scientist's position as relevant evidence on a question of fact that turns on scientific expertise?

Should they accept the expert's assurance that there is nothing to these quotes, they're taken out of context, you're not qualified to assess their significance, and it's all a fuss about nothing generated by a conspiracy of shadowy oil company executives who secretly control the global media?

What criteria and heuristics do you think they should apply in coming to that decision?

If this was a real court case, tried by a jury of a dozen rational but scientifically untrained citizens, what do you think they would decide? Do you agree that they have no choice but to trust the experts, and dismiss the defence's so-called evidence?

May 30, 2013 | Unregistered CommenterNiV

Specifiically, shouldn't a reasonable person treat a qualified scientist's position as relevant evidence on a question of fact that turns on scientific expertise? And shouldn't he or she treat the existence of widespread agreement among like experts as relevant?

Very often yes. Particularly situations where it can be shown that most experts would agree on that specific fact. Like, for example, comparison of two DNA tests. Things become less clear on other things, like, for example: What is the best thing to do if you find you have the breast cancer gene Angelina Jolie had.

Also, things become dicier if two experts do disagree on the fact. My impression is this has been known to happen in court. I knew two guys in multiphase flow both of whom testified as expert witnesses for opposites sides in a court case once. The fact is, they weren't really testifying on facts but interpretation of facts. They didn't agree.

@Joshua thinks agreement or disagreement w/ my statement is an acid test of motivated reasoning.

So, did I pass or fail the test?
FWIW: I don't think there is an acid test of motivated reasoning. Certainly, agreeing or disagreeing with a statement can't be one.

May 30, 2013 | Unregistered Commenterlucia

Lucia -

Things become less clear on other things, like, for example: What is the best thing to do if you find you have the breast cancer gene Angelina Jolie had..

I don't think I understand your point there. Of course, in situations like that the question of personal choices about trade-offs comes into play - but aren't "things" less clear in situations like that more or less in direct inverse proportion to the level of widespread agreement among "experts?"

Also, things become dicier if two experts do disagree on the fact.

Of course two experts might disagree. But I don't see how that is directly relevant to when there is widespread agreement among "experts." Is that not relevant, although not conclusive, information? Particularly for someone who cannot evaluate the evidence at a technical level, but has a compelling reason to consider any evidence available to evaluate probabilities?

So, did I pass or fail the test?

Did you agree or disagree with the paragraphs I excerpted?

May 31, 2013 | Unregistered CommenterJoshua

Joshua
I don't think I understand your point there.
There are multiple points:
1) Often choices represented as facts are actually mixed matters.
2) Even the fact aspect of Angelina's choice is debated among experts. The probability of getting breast cancer when one has the gene is not known with pinpoint accuracy.
3) Even given a known probability, different doctors have different advice about whether one should merely be very vigilant about monitoring or having surgery. This is not entirely a 'personal' matter, but a judgement about the probability that monitoring vs. surgery might work and also a judgement about when new treatments might be available.

but aren't "things" less clear in situations like that more or less in direct inverse proportion to the level of widespread agreement among "experts?"
Sure. Out of curiosity, what did you understand this to mean when I wrote it?

particularly situations where it can be shown that most experts would agree on that specific fact.

But the fact is: generally, very strong consensus tends to exist on rather narrow facts like "What is the standard procedure to read a DNA scan? Do the Navier-Stokes equations apply to this problem? Stuff like that.
If you start to move onto more complicated issues, experts will begin to give 'nuanced' descriptions and that often strongly correlates with lack of strong consensus on a number of details.

But I don't see how that is directly relevant to when there is widespread agreement among "experts.
Uhhmmm... It's relevant because Dan asked a question. I gave an answer and I'm giving a caveat to the answer. The alternative is that my answer will not properly reflect what I consider to be the full and complete answer. That is: I think the answer to Dan's question requires including the issue of the level of agreement.

If the degree of agreement is a factor that matters, and I think it does, the person relying on the evidence of experts needs to judge whether the degree of agreement among experts. There are different ways to do so.

Is that not relevant, although not conclusive, information? Particularly for someone who cannot evaluate the evidence at a technical level, but has a compelling reason to consider any evidence available to evaluate probabilities?
Ok..but we've got a "turtles all the way down" problem here. How do you determine the existence of widespread agreement? Especially in an area outside your subject matter competence?

Anyway: Why are you asking some of a question like "Is that not relevant,". I said the experts answer on a statement of fact should generally be taken as providing evidence in the sense Dan defined. So, I should think you'd already read that my answer is yes -- with caveats that apply in many cases where experts are asked to report on things that are either represented as fact based and/or where there could be a differences of opinion as to facts. These cases are very, very common.

May 31, 2013 | Unregistered Commenterlucia

@Lucia:

There's no acid test, agreed.

I don't think the methods by which one can discern the signature of motivated reasoning ("& its cognates") are capable of discerning the processes by which a single individual makes sense of evidence. Social psychology and psychoanalysis are very different (in the way, I think, that astronomy and astrology are).

I suppose once we understand a process like motivated reasoning, we can make surmises about particular individuals; indeed, I'm sure we'll do that, because making sense of other individuals is essential to interacting with them.

But it's good, if we do that, to remember one of the "cognates": "naive realism." Naive realism describes an asymmetry in individuals' perception of the influence of motivated reasoning. People are pretty good at discerning it in others, but pretty bad in discerning it in themselves. What to make of this if you are a reflective person who is trying to interact with others -- & in settings in which there is likely to be some "social surplus" in the exchange of information (enlightenment, identification of solutions to a common problem, etc)?

To me this means that when I see "motivated reasoning" in someone I want to engage, I am as likely to be experiencing it myself. So the diagnosis, in that sort of transaction, is not reliable; it is as likely to mislead as guide me to where I want to go. The cost of being misled is very high -- a missed, likely preempted, opportunity to gain insight; the cost of forgoing a correct "diagnosis" of motivated reasoning in an other person, in contrast, is close to zero, since I will learn soon enough whether a prospective interlocutor is in fact not someone with whom I can profitably transact, even if I can't ever be sure exactly why. So generally speaking, I avoid imputing motivated reasoning & the like to individuals with whom I am trying to have a discussion.

But I think @Joshua's acid test is actually extremely valuable. I don't think it was offered -- or at least used by him -- as a diagnosis of others. What I saw him doing was offering up a sort of heuristic for engaging in exactly the sort of critical checking & self-interrogation that I was just trying to describe.

A reasonable person who values being genuinely reflective should worry about having his or her own reflection dulled by cultural cognition (a cognate; the most relevant one, I think, in the context of our discussion) & the like. To ward this off, such a person should be monitoring, checking, testing his or her own reasoning. That's hard to do.

But one way one might do it is to try to formulate positions or arguments that one believes are correct and indeed of fundamental importance but that one recognizes would likely pose a challenge to people on both sides of an issued that polarizes people along cultural lines. If one can't do that, then one has good cause to be very very worried about his or her quality of reasoning.

My answer to @Jim was indeed offered in that spirit. I think both of the propositions I stated -- that science's way of knowing demands that we treat everything we know (everything it knows) as provisional, and hence legitimately open to challenge; and that a reasonable person will treat as probative evidence of truth the knowledge of those he or she recognizes as possessing the sort of insight scientific inquiry yields -- are of fundamental importance. And I recognize, too, that in the context of a discussion of climate change (something I actually didn't expressly mention in my "answer to @Jim"), each one is likely to make one side uncomfortable.

I am curious what others think of my "answer." But in fact I won't view their acceptance or rejection of it as diagnostic of their using motivated reasoning or their being someone who it is worth engaging in conversation.

Instead, I'll just see what they say, and see what I can learn from them -- so long as I can discern that they are trying their best too to protect their opportunity to learn from me and from others from being preempted by the distorting influence of "motivated reasoning & its cognates.'

I think you (@Lucia, @NiV, @Larry, @Joshua, @et al.) all "pass" this test.

Or else I wouldn't be engaged in conversation with you.

May 31, 2013 | Registered CommenterDan Kahan

Lucia -

I see that I missed your comment where you started with this:

First: I think you may be over interpreting the argument-- though in some cases that may be the argument. .... It is true that a small fraction of people likely think the consensus even doesn't even us what is probably true, but I suspect if you asked, they number few.

I think I addressed your points in my comment immediately following - perhaps anticipating the logic of your response to my previous comment. Still, a response here: I don't think I'm over interpreting the argument - I am critiquing the rhetoric of the arguments that I have seen many make. My point is that the rhetorical arguments are fallacious - and so they are "tells," if you will, for "motivated reasoning" -- whereby smart and knowledgeable people use those rhetorical devices.

I have certainly often seen "skeptics" say something on the order of "...but plate tectonics..." as if that proves anything except as a response to an equally fallacious rhetoric on the other side: "...but 97% of scientists agree..." But then again, I have certainly often seen the "...but 97%....": argument made by "skeptics," as if that proves anything except as a response to an equally fallacious rhetoric on the other side: "...but plate tectonics..."

Each of those rhetorical arguments are, in a sense, valid when viewed in isolation (well, to some extent -- I doubt the veracity of the 97% number. I don't think that you could get 97% of any particular group of people to actually agree on much of anything), but fairly useless rhetoric when placed in context.

Context demands the scrutiny, or skepticism if you will, of asking:

(A) "Yes, but how much does one example of the "consensus" being wrong (or a long list of only those times when "consensus" was wrong) tell us (and in particularly, a non-expert) about how to evaluate what prevalence of expert opinion tells us about probabilities?"

and

(B) "How much does prevalence of expert opinion, in and of itself, tell us about how to evaluate the probabilities?"

But the fact that the consensus is sometimes wrong does not prevent some people arriving at my blog from posting "persuasive" arguments that appear to amount to nothing more than. "This is the consensus position. You are required to pledge 100% belief in the consensus position".

No doubt. If that argument is not made explicitly, without the context and caveats that skepticism demands, that is the implication of an argument frequently found from one side of the debate.

But agreeing that is true more generally in the debate even if not being terribly conversant in what happens at your blog, I find your following response lacking:

Obviously, because that statement is simply wrong, people will often respond: The consensus can be wrong.

First, I don't see the cause-and-effect as being unidirectional as you describe there. Who, exactly, is responding to whom? Who takes the fist step in this debate, at the macro (Fred Singer or the IPCC?) or at the micro level (blog commenter X (a "realist"), or blog commenter B: (a "skeptic)?

Second, answering one argument that implies fallacious reasoning (even if not stated explicitly) with a second argument that implies fallacious reasoning (even if not stated explicitly) may well describe what we often see in the climate wars, but it doesn't make the second argument valid.

May 31, 2013 | Unregistered CommenterJoshua

@all:

as a result of motivated reasoning, some of @Lucia's posts are being detained in the spam folder. I am trying to teach the filter that this is a mistake

May 31, 2013 | Unregistered Commenterdmk38

Joshua

I am critiquing the rhetoric of the arguments that I have seen many make. My point is that the rhetorical arguments are fallacious - and so they are "tells," if you will, for "motivated reasoning" -- whereby smart and knowledgeable people use those rhetorical devices.

And yet, when I told you I often see people coming in seeming to make fallacious arguments about how one must</I> accept a position if it is the consensus, you seemed to suggest I was over interpreting. But sure.

Fallacious arguments are tells. On both sides. :)

I have certainly often seen "skeptics" say something on the order of "...but plate tectonics..."

And I've seen the "people who call themselves realists" say something like "You'd go to a doctor..." (And yet, this is almost a worse example than 'plate techtonics'. Many people self diagnose and self medicate pretty often. Most MD's would even recommend it to a certain extent. Many get 2nd opinions and so on. And medical consensus has changed-- and quite a bit over time. Heck, when I was a young girl, menstrual cramps without evidence of fibroids or such were *officially* decreed to be mental in origin. It was a consensus in text books. That fell by the wayside. Advice on diet has shifted. All sorts of things have evolved. People are expected to consider medical advice judiciously. )

Each of those rhetorical arguments are, in a sense, valid when viewed in isolation

Sure. And the same holds for the "I call my self a realist" rhetorical arguments.

(A) "Yes, but how much does one example of the "consensus" being wrong (or a long list of only those times when "consensus" was wrong) tell us (and in particularly, a non-expert) about how to evaluate what prevalence of expert opinion tells us about probabilities?"

If there was only one example, your question would be valid. But you should at least expand it to 'N' examples. And regardless, the answer is "more than nothing" but "less than everything". So, yes: That is the question. But asking it doesn't magically provide it's own answer: Different people give different answers it.

First, I don't see the cause-and-effect as being unidirectional as you describe there. Who, exactly, is responding to whom? Who takes the fist step in this debate, at the macro (Fred Singer or the IPCC?) or at the micro level (blog commenter X (a "realist"), or blog commenter B: (a "skeptic)?

I haven't claimed it's always unidirectional. I'm saying that does happen. It does.

As for first step: different people take the first step at different times. And why pick Fred Singer or the IPCC? We are so far from the 'beginning' that who can possibly identify it. It's not fair to somehow decree that "your" side is always reacting while "the other" side is initiating these fallacious arguments. It's pretty balanced.

At my blog, I do sometimes see people-- often first time commenters-- arrive and suddenly, out of the blue try to 'rebut' arguments by making sudden declarations of consensus. I also see some first time commenters arrive out of the blue and suddenly say "This is bogus. Climate change is a FRAUD". Whatever argument you want, if I wanted to do a hunt, I'd find it and it would just pop up out of the blue.

Heck: See May 29, 2013 jim luther comment? I see junk like that too. There are flyby whatever... As a lukewarner I see that from both sides.


Second, answering one argument that implies fallacious reasoning (even if not stated explicitly) with a second argument that implies fallacious reasoning (even if not stated explicitly) may well describe what we often see in the climate wars, but it doesn't make the second argument valid.

I haven't said it does.

But I would point out that if I report I see "those who Joshua calls realist" make the sort of argument you have just described and providing a "tell" to motivated reasoning, you are rather quick to explain that I am over-interpreting. Yet, you yourself are pretty quick to see it as a "tell" if the person presenting the argument is "someone Joshua calls a skeptic".

Either these things are tells, or they are not. They happen on both sides. I've seen it on both sides-- and I'd say I've seen it pretty equally on both sides.

May 31, 2013 | Unregistered Commenterlucia

Lucia -

You are interpreting a unilateral quality in my comments that are not consistent with my views. I don't know why that is the case, but it is the case:

And yet, when I told you I often see people coming in seeming to make fallacious arguments about how one must</I> accept a position if it is the consensus, you seemed to suggest I was over interpreting.

That is not what I am suggesting. I don't doubt that you do see that. I have seen that. We have all seen that.

Fallacious arguments are tells. On both sides. :)

No doubt. I am usually pretty careful to make it clear that is the starting point for me. I think that what we know about basic functionality in how humans reason leads us all towards fallacious arguments - particularly in contentious debates that overlap with social, political, cultural, personal, and psychological identifications.

I see no asymmetry, and I am highly skeptical of arguments based on asymmetry - no matter from which side they emanate.

And I've seen the "people who call themselves realists" say something like "You'd go to a doctor..."

Indeed. As have I. And I have argued against that logic - explaining that I find if fallacious.

http://blogs.discovermagazine.com/collideascape/2012/02/01/summing-up/#.UajRjJwXlco

So, it seems we are in agreement. Then why are we arguing?

I haven't claimed it's always unidirectional. I'm saying that does happen. It does.

I didn't think that you were claiming that. I wasn't suggesting that it doesn't happen. I was critiquing your comment that only pointed to a unidirectional phenomenon, without stating explicitly that it happens bilaterally.

At my blog, I do sometimes see people-- often first time commenters-- arrive and suddenly, out of the blue try to 'rebut' arguments by making sudden declarations of consensus.

Again, I don't question that it does happen. My point is that even when it does happen in an unambiguous way, locating the origin of such a discussion of very problematic. A comment at your blog is not spontaneously generated - it comes in context, a context replete with fallacious arguments on both sides.

But I would point out that if I report I see "those who Joshua calls realist" make the sort of argument you have just described and providing a "tell" to motivated reasoning, you are rather quick to explain that I am over-interpreting. Yet, you yourself are pretty quick to see it as a "tell" if the person presenting the argument is "someone Joshua calls a skeptic".

You left out the quotes. I use scare quote around "realist," to suggest a putative connotation. Someone who makes the "97% of scientists" argument, devoid of contextualization, is a realist in quotes - IMO.

you are rather quick to explain that I am over-interpreting.

Again, I that was not my intent. I am not explaining that you are over-interpreting. I am saying that the phenomenon is complex and bilateral. I see those arguments as "tells" on both sides. It seems that we are in agreement.

I tried to make that clear with each comment. The failure to communicate here is fascinating.

(1) Either these things are tells, or they are not. (2) They happen on both sides. (3) I've seen it on both sides-- (4) and I'd say I've seen it pretty equally on both sides.

(1) I agree, and further, I think they are tells. (2) I agree. (3) I have also. (4) I'd say so also.

I fear that at this point we may have turned a corner in this exchange - to a direction that entails more misunderstanding than understanding; so I'm hoping those numbered comments will suffice as a reversal point from which to move forward. I think those four points are the most important elements.

May 31, 2013 | Unregistered CommenterJoshua

Joshua
You are interpreting a unilateral quality in my comments that are not consistent with my views. I don't know why that is the case, but it is the case:
Possibly because your examples of people indulging in fallacies are always or nearly always the skeptics with the examples of people who might merely be taking short cuts because they are responding to skeptics are the group you have called "realists". (A word choice that itself has implications). I don't think you necessarily think that but that's the net effect of your posts.

You are interpreting a unilateral quality in my comments that are not consistent with my views. I don't know why that is the case, but it is the case:

Yes. But the thing in scarequotes is nevertheless realist which has implications. It's not your fault there are no good words. I like lukewarmer, warmer, statist, cooler, hell-fire-and-brimstone warmer (e.g. Joe Romm) but those really haven't caught on..

Then why are we arguing?
Because it's a blog? There is an element to blogs that does tend to cause issue. Besides my comments going to the spam bin.... I am also currently trying to do something for a post. That is an Analysis in R that is motivated by the "Keenan Kerfuffle" followed by a discussion with Paul M who pointed me to a draft paper, and we are arguing about a two sentence claim. Then I come back and read.

Anyway... back to R. :)

May 31, 2013 | Unregistered Commenterlucia

@Joshua & @Lucia:

How about do an experiment that disconfirms the hypothesis -- that one or the other of you is engaged in motivated reasoning?

Come up w/ an argument that you know is right & in fact very important but that has elements that will make "both sides" squirm. Articulate & defend it.

If you do this, I have no idea whether it will prove anything to the "other person."

But given how vulnerable even reasonable and reflective people are -- indeed, more reflective, critcal people appear more vulnerable, sadly, particularly on climate change--won't you feel better if you can prove to yourself that you are able to resist?

If this isn't the way-- or the only way -- then please tell me & others what else to do. What else to try to maintain sovereign authority over all the elements of our reason so that none betrays our intention to be open to learning something we didn't know before.

May 31, 2013 | Registered CommenterDan Kahan

Dan -

Come up w/ an argument that you know is right & in fact very important but that has elements that will make "both sides" squirm. Articulate & defend it.

Well - I didn't exactly come up with the argument, but you have articulated an argument that I have long felt has elements that will make both "sides" squirm. I have seen both sides squirm against that argument. And it is that argument that I am defending.

If you this, I have no idea whether it will prove anything to the "other person."

My experience is that no one proves anything to anyone else (who isn't already predisposed to agree with them, anyway) in these debates (the climate wars as being one example of many similar debates).

But given how vulnerable even reasonable and reflective people are -- indeed, more reflective, critcal people appear more vulnerable, sadly, particularly on climate change--won't you feel better if you can prove to yourself that you are able to resist?

I am satisfied with proving (to myself, anyway) that I can resist sometimes. I don't think I can expect more than that (in fact, I think that anyone expecting more than that is kidding him'herself), and I have done that already, a number of times, in this exchange. One example - here's what I wound up with:

(2) On the flip side, it seems to me that "skeptics" arguing about the prevalence of expert opinion, or further, exaggerating the amount of disagreement among experts (as to whether anthropogenically-influenced climate change presents enough of a potential threat to merit careful consideration of policies aimed at some combination of mitigation and adaptation) also has a similarly limited impact on public opinion (although, I would guess, a larger impact than the impact of "realists" hammering away about "consensus"). In balance, such efforts will also primarily serve to solidify pre-existing opinions.

I originally wanted to say that fallacious arguments from "skeptics" - e.g., exaggerating disagreement among "experts" - have a significantly larger impact on public opinion in those not already convinced. But then I realized that such an argument would be inconsistent with the operating assumption that by far, few people use information in this debate to do anything other than confirm biases. So I tweaked my argument a bit in the name of correcting for my own motivated reasoning. Perhaps not enough?

I tend to reflect on these types of things somewhat obsessively. Another example: I am thinking of making an offer on one of two investment properties in the next couple of days. One property I like aesthetically. It is in a location I like. It would appeal to renters who have similar tastes as mine - who really care about where they live and have enough money to divert some of it purely on the basis of aesthetics. The rate of return on investment is decent, but things have to fall right in line to get it to the rate of return I have pegged as a reasonable goal. The other property is one that doesn't appeal to me aesthetically. It would likely be rented by people who are less focused on aesthetic concerns, or who have less money to focus on less practical considerations. The rate of return on the investment is solid. Things could work out not quite as expected and it still would bring a rate of return that matches my pegged rate of desired return.

I find myself twisting my analysis to make the return on the one house that I like aesthetically look relatively better than the other house. It's funny because some people just roll with something like that - and choose options like that house because it is just simply what they want to do. I, rather obsessively, think that I have to control for those kinds of biases in order to make some decision that could be considered the "right" decision based on some supposedly "objective" analysis. It reminds me of the literature on decision-making, where they find that people who make decisions more impulsively and on a "feeling" basis turn out to make better decisions, long term. Are (more overt than I am) climate combatants more pleased with their own position in these debates?

My goal is to improve my ability to filter out my "motivations" in these debates, and I welcome help in doing so.

If this isn't the way-- or the only way -- then please tell me & others what else to do.

I think you've nailed it. I look forward to critiques of your paragraphs that I quoted, do see if anyone can explain why what you say there wouldn't suffice as an example of an argument that would make both sides squirm, and/or as an argument that everyone agrees is fundamentally sound. Of course, I will need to work hard to evaluate any critiques offered in a non-"motivated" manner.

May 31, 2013 | Unregistered CommenterJoshua

@Joshua:

Fine. But do you not disagree w/ me that there is little to be gained -- & posibly something importnat lost -- by trying to detect individuals who are experienceing motivated reasoning or one of its cognates? & do you disagree @Lucia? Of course, I don't think there's much risk either of you will stop talking to the other -- & why isn't that sufficient proof that neither of you is constrained by MR in a way that unduly threatens the possibility of exchanging ideas w/ one another? But still, as evidence of the cost-- & I hope I don't insult you;- my motive is actually a selfish one, since I learn from the conversation of others -- but what do you imagine you might have talked about instead if you hadn't spent as much time interrogating the other's reasoning? Might it have been something more likely to reveal a surprise of any sort?


May 31, 2013 | Registered CommenterDan Kahan

But do you not disagree w/ me that there is little to be gained -- & posibly something importnat lost --by trying to detect individuals who are experienceing motivated reasoning or one of its cognates?

Hmmm. I don't think that there is much to be gained or lost by trying to detect MR in individuals, per se. I can certainly say that I do not have much luck in convincing others of the MR I observe in their reasoning - so there is clearly little to gain by that measure. But ultimately, the value gained is in identifying it in myself and using the reasoning of others is a vehicle for doing so (although certainly not the only one).

I will say, however, that by presenting to others my observations about their potential MR, I sometimes get useful feedback (almost always in a hostile form) for evaluating my own MR. So then I guess the question might be of opportunity cost. Perhaps I could get that feedback in a less hostile form - which might then have less nose and more signal?

I hope I don't insult you;- my motive is actually a selfish one, since I learn from the conversation of others -- but what do you imagine you might have talked about instead if you hadn't spent as much time interrogating the other's reasoning? Might it have been something more likely to reveal a surprise of any sort?

No insult taken, Dan. First, I think it is silly to feel insulted by people presenting ideas on a blog, even if they are intended as insults (although I sometimes do feel that way, I know it is silly). Second, I have no question that you are engaged in good faith debate - one of the few (the only?) I've encountered in blogs I feel that way about- so to feel insulted would be double-silly in this case.

But to your point...

When I listen to jazz or other improvisational music, and classical music also, I often find myself trying to anticipate the next note. Much of the pleasure I get from listening to that sort of music is the surprise and/or confirmation of wrong and/or correct predictions. I think it is a rich domain for exploring the cross-over between the intellect and the intuitive and the emotional.

These debates are not entirely dissimilar for me. By stimulating response, I get to test and explore my intellectual, emotional, and intuitive predictions of how someone might see things differently than me. I certainly don't think that process is neutralized by testing the influence of MR on the reasoning of others. Sometimes there is surprise, and when it happens, it is the most enjoyable of these interactions, IMO. I'm not sure if surprise always leads to the greatest insight (sometimes my failure to be cognizant of what should have been easily predicable is where I gain the most insight - for it is a tell of my own MR)

Maybe I'm missing the point of your link? Is your point that examining for MR in others results in a loss opportunity for surprise or some other opportunity cost? How does not talking about --- something else instead --- lessen the chances of surprise or lead me to be overly-certain (be oblivious to uncertainty)?

May 31, 2013 | Unregistered CommenterJoshua

Heh. Less nose and more signal and also less noise and more signal.

May 31, 2013 | Unregistered CommenterJoshua

The link for the statement that providing evidence for consensus will "only amplify the signal of cultural contestation that sustains polarization" doesn't support the statement, IMHO.

The link says that Fox isn't reporting the 97% consensus. I suppose Dan could argue that non-Fox watchers will learn more of the truth, Fox watchers stay ignorant, and people are more polarized. I think the contrary argument is that to the extent people have overlapping information sources, the fence sitters and even those who lean against reality will hear about the study, and to the extent that the consensus gets reinforced in the public arena, this information will leak over to people who are hesitant to accept it.

A better proof of Dan's opinion would be Fox covering the 97% news by dismissing and denying it. That's not how things turned out in reality, though.

June 1, 2013 | Unregistered CommenterBrian Schmidt

@Brian:

In fact the point in the 2d to last sentence of your message is the one I was tryign to make. The link reports on Fox trashihng the study. Or at least that's what I thought the story was reporting -- & in way that itself expressed venom toward Fox, so there was in effect, a "twofer": a polarization-signal reinforcing denunication of Fox for amplifing the signal of polarization by denouncing the Cook study as junk. Was I wrong in my understanding of the report?

If I misinerpreted that particular news story, I take it you'd still agree that it would count as support of the "amplificatoin" claim if one could adduce evidence that there was a torrent of denunciatory partisan back-and-forths after the study was released. Have you looked to see if there was such a response?

another queston: What do you think would happen if we showed the Cook et al study -- or more realistically the press release that their university press office issued -- to a large, diverse sample of people in US? Would it decrease cultural polarization? Incrase it? Have no effect?

If you said, decrease, would you be willing to bet on that answer? How much? :)

June 1, 2013 | Unregistered Commenterdmk38

I disagree that other liberal democracies are polarized in the way the US is. In Canada, none of the leaders of any major party deny or even downplay the consensus on climate change. Even in Alberta, the leader of the right-wing Wildrose party blew a big lead in the provincial elections last year because she said that there was still significant doubt about the science on climate change.

In British Columbia, a revenue-neutral carbon tax was introduced by a centre-right party that went on to to win the next two provincial elections.

In Europe, conservative leaders have adopted progressive policies on climate change, for example, Merkel, Cameron, Sarkozy. French climate skeptics Claude Allegre and Vincent Courtillot are both socialists. Allegre was Minister of Education in the French government under socialist prime minister Lionel Jospin.

Even in the United States, only a few years ago, it was possible for Newt to share a couch with Nancy and talk constructively about climate change. The current political polarization on climate change in the US is real enough but it is by no means a universal nor (we hope) an eternal phenomenon. Polarization has occurred in many areas of policy and, presumably, these were not all caused by liberals bludgeoning conservatives with facts. Surely there are other factors at play here.

Science communicators should, of course, pay attention to to recent trends in the United States and the excellent work done to elucidate and ameliorate these phenomena by people like Dan Kahan. But the target of climate science communication efforts need by no means be confined to American conservatives. Not everybody needs to feel constrained to use methods tailored to one subset of contemporary American society.

I do not even try to communicate to convinced climate contrarians, they are beyond my reach; I do not speak the language that can get through to their mental System 1 (Kahneman) and I happily leave that job to people like Katharine Hayhoe .

My personal opinion (and this may not be shared by my colleagues at Skeptical Science) is that the main problem with the lack of progress on climate policy is that people who pay lip service to accepting the scientific consensus do not grasp how strong that consensus is, nor do they fully appreciate the gravity and urgency of the problem. The issue is not so much that there are powerful, dark forces opposed to climate policy, but rather that the people who are in favour of it are in favour in a half-hearted way, with other policies higher on their agenda. For example, we have Justin Trudeau, the new leader of the Canadian Liberal Party and no climate skeptic, criticising PM Stephen Harper for not pushing the Keystone XL pipeline hard enough.

I speculate that even those people concerned about climate change have social and personal motivations to exaggerate the scientific uncertainty and to minimize the consequences and the urgency of the problem. In this, I am influenced by Kari Norgaard and Stephen Gardiner and I may elaborate that argument someday on my own blog, because, to paraphrase Austen's Mr Bennet, I have probably by now delighted you long enough.

June 1, 2013 | Unregistered CommenterAndy Skuce

@Andy-- I agree that there is meaningful variance among "liberal democracies." I'm not so familiar w/ Canada, weirdly. But I know that the UK & Australia are both culturally polarized; we've done studies in both, but not to establish whether there is polarization -- that's actually obvious to see -- but to work on cross-culturally valid versions of our measures.

I was in Norway last week to talk w/ European science communicators. I was told that Norway is itself quite divided. Apparently Sweden is less so -- which is interesting given that their media coverage of climate apparently is indistinguishable from US (a finding that casts doubt on the proposition that balanced" media coverage is meaningful source of political controversy in US).

But clearly not all liberal democracies are polarized.

What's quite intersting, though, is that there apparently is no non-liberal, non-democratic society that is polarized! I'm advised that there's no controversy in China, e.g. No one really pays much attention.

Maybe one can only expect people to pay attention & care in democratic socieities? That's one hypothesis.

But another is that cultural polarization is more likely in liberal democracies for the same reason that they are congenial to science: their political culture nourishes habits of mind that feature independence, challenge to authority, etc., all of which contribute to the engine of conjecture & refutation" that drives science but all of which also complicate authoritative certification of truth... An interesting dilemma.

When you do post something on your blog, please advise me-- you haven't by any means exhausted my interest in the argument you are presenting. (Indeed, if you think it would be useful to have an "exchange" on this topic, I'd be happy to be a party to it!)

June 1, 2013 | Unregistered Commenterdmk38

Dan, you say the link "reports on Fox trashing the study."

Here's the second sentence of the link: "Fox News has yet to report on these findings, after repeatedly insisting that there is no scientific consensus on the issue." Fox, so far at least, isn't trashing the study, they're trying to avoid letting their viewers know about it.

In other words, they're hiding from evidence, and they're doing it IMO because the evidence seems to them to be persuasive.

June 2, 2013 | Unregistered CommenterBrian Schmidt

Phenomenological reality, whatever it happens to be, is completely impervious to how human beings interpret it, how much they argue about it, or the reasons for their arguments. At any given time, including the present, we have competing interpretations of reality, and if there's one lesson of history, it's that we are never wholly correct, not even if 100% agree we are.

Expert or inexpert, were we to be given the opportunity to come back in a century to review our current understandings and opinions, I think we'd all discover a large portion of what we feel we know beyond doubt/believe/accept to be tosh. Even elementary school children will be taught stuff is wrong that today is considered not only right, but has an assembly of bearded professors shaking their heads wondering why people won't take their word for it.

I admire scepticism, in the purest sense of that word: viz, knowing what one doesn't know, and that there may be unknown unknowns. One might have leanings this way or that, but if one does, one isn't totally invested in them. FWIW, my leaning is that the radiative physics underlying AGW theory is correct, but that AGW is not problematic and has been greatly over-hyped. I'll drop that view if and when empirical evidence convinces me otherwise.

Here's something from Dr. Judith Curry's recent testimony to congress (http://curryja.files.wordpress.com/2013/04/curry-testimony-2013-il.pdf):

“The politicization of climate change presents daunting challenges to climate science and scientists. In my assessment, the single most important actions that are needed with regards to climate science – particularly in context of assessments for policymakers – is explicit reflection on uncertainties, ambiguities and areas of ignorance (both known and unknown unknowns) and more openness for dissent. Natural internal variability is a topic of particular importance over which there is considerable disagreement. Disagreement and debate is the soul of the scientific frontier, which is where much of climate science lies. Greater openness about scientific uncertainties and ignorance, and more transparency about dissent and disagreement, would provide policymakers with a more complete picture of climate science and its limitations. When working with policy makers and communicators, scientists should not fall into the trap of acceding to inappropriate demands for certainty; the intrinsic limitations of the knowledge base need to be properly assessed and presented to decision makers. The role of scientists should not be to develop political will to act by hiding or simplifying the uncertainties, either explicitly or implicitly, behind a negotiated consensus.”

IMO, Judith Curry possesses possibly the purest sceptical voice in this whole sorry debate. She has gradually become a major influencing force because she is an expert who isn't afraid to acknowledge the many uncertainties. I, and I'm sure many other people, respond to that kind of humility, and have developed respect for her expert opinions (even when I might disagree with them) precisely because of it. She is actually achieving what a number of talking shops seem only to be discussing. Discussing, it might be said, with what often appears to be an uncritical acceptance of--not so much consensus--as scientific political correctness (and what a travesty it is that such a thing could actually exist).

Self-styled cognoscenti who seek to communicate what they think is written on tablets of stone, and who can't imagine how anyone could have legitimate reservations, come across as patronising: as questioning everyone's opinions (perhaps even the right to have them) except their own. When those seeking impartial expert opinion encounter someone as well-accredited yet humble as Dr. Curry--well, as in so many areas of life, they are quite likely to give them due respect and attention.

June 2, 2013 | Unregistered CommenterMichael Larkin

"another queston: What do you think would happen if we showed the Cook et al study -- or more realistically the press release that their university press office issued -- to a large, diverse sample of people in US? Would it decrease cultural polarization? Incrase it? Have no effect?"

Most of them would be bored rigid at being lectured at yet again. Most people are interested in football or soap operas and celebrity gossip.

But assuming you got an audience who engaged with the material, it would depend on whether they had seen discussion on these surveys previously or knew about Cook's activism, it would depend on whether you only showed them the abstract or you showed them the claims made both for and against, and it would depend on whether you allowed those interested to do a search for more information. I don't know how much they know about the debate.

It would also depend on whether they understood what question the paper was asking and what questions are actually controversial.

What do you suppose would happen if the better survey (the Bray one) I linked to previously was shown to a large diverse sample? Would it polarise people the other way?

June 2, 2013 | Unregistered CommenterNiV

The link says Fox didn't cover the story about Cook paper. Did NBC, CBS, etc?

June 2, 2013 | Unregistered Commenterlucia

Lucia raises an interesting tangential question, one that I'm sure she's capable of researching if she care to do so. Regardless of the answer to that question, I don't see either the specific statement in the original post or Dan's followup statement in the comment to be supported by the link (sounds like Lucia might agree with me on this one issue).

I'm not claiming this is a big deal, but the reason why I clicked on the link to begin with was because I thought his statement, if verified, would provide substantial support to his general position as I possibly understand it that publicity about the evident truth of the consensus is a bad idea. I didn't find it when I clicked on that link.

June 2, 2013 | Unregistered CommenterBrian Schmidt

Brian--
You personally and dmk38 have been discussing Fox news and speculating what their not covering it means. You went so far as to suggest Fox's motive for not covering the story. Specifically

they're trying to avoid letting their viewers know about it.

I asked you whether other groups covered the story because trying to figure out what to make of your speculation about Fox's motive given that your discussion of Fox covering or not covering a story exist in a data vacuum where the question of whether other news groups have discussed isn't mentioned. Some storied aren't generally covered by news agencies. Others are.

I have to admit to not being all that motivated to do research to figure out whether your speculation that "they're trying to avoid letting their viewers know about it." is remotely plausible. But to be convinced, I'd need to know whether other NBC, CBS, ABC covered the story. But I thought if you suggested Fox's motives you might know and in which case, you would tell me.

The fact that you -- who speculated-- seem to have reacted by suggesting that I do the research suggests to me that you don't know whether NBC, CBC or ABC covered the story. Given lack of knowledge on this, I'd suggest that likely as not Fox didn't cover the story about a paper being published for the very simple reason: Zillions of papers are covered every day and they rarely cover any of them. The story is a non-story. Likely the other channels, large news papers and all sorts of publications saw it as a non-story and failed to cover the story.

That said: If you</I> are speculating about Fox's motives in not covering the story, I think you may have provided evidence that the publication of the paper will polarize. After all: Some people will speculate about the motives of newsagencies whose reaction to the story is "yawn.".

June 2, 2013 | Unregistered Commenterlucia

"I'd suggest that likely as not Fox didn't cover the story about a paper being published for the very simple reason: Zillions of papers are covered every day and they rarely cover any of them. The story is a non-story."

I'd think another reason would be that it's simply not newsworthy. It's a claim that's been made many times before, so it's not novel, and is very probably wrong (as even a brief search for discussion on it would show). A Bayesian would see no reason to update their beliefs - if you are inclined to believe then you probably already think that anyway, and if you're not you'd not be interested in being told something that was wrong. In a Bayesian sense, it is information-free, LR = 1 stuff. It's not news.

Sceptics as a rule are not shy about talking about papers that they disagree with. If they think it's important, they see it as more useful to get the information answering it out there. Trying to hide an issue by simply not mentioning it doesn't work in the internet age.

June 3, 2013 | Unregistered CommenterNiV

Trying to hide an issue by simply not mentioning it doesn't work in the internet age.
But I think my point is merely not discussing a paper is not necessarily hiding. Every newspaper, television program, and even blog can only cover a finite number of stories. And even smaller number are shown prominently. If-- in this case-- none of the other big media covered it either, then Fox's lack of coverage could simply be that it's not a big story for anyone.

On the other hand, if Richard Tol gets his paper criticizing it published and Fox immediately jumps on that rebuttal while no one else covers it that would mean something. But that hasn't happened-- though who knows. Maybe it will.

June 3, 2013 | Unregistered Commenterlucia

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>