follow CCP

Recent blog entries
« WSMD? JA! How confident should we be that what one "believes" about global warming, on 1 hand, and political outlooks, on other, measure the same *one* thing? | Main | "Messaging" scientific consensus: ruminations on the external validity of climate-science-communication studies, part 2 »
Wednesday
Jun182014

What is the *message* of real-world "scientific consensus" messaging? Ruminations on the external validity of climate-science-communication studies, part 3

This is part 3 of a series on external validity problems with climate-science-communication studies.The problem, in sum, is that far too many researchers are modeling dynamics different from the ones that occur in the real world, and far too many communicators are being induced to rely on these bad models.

In my first post, I described the confusion that occurs when pollsters assert that responses to survey item that don't reliably or validly measure anything show there's "overwhelming bipartisan support" for something having to do with climate change.

In the second, I described the mistake of treating a laboratory "messaging" experiment as better evidence than 10 yrs of real-world evidence on what happens when communicators expend huge amounts of resources on a "scientific consensus" messaging campaign.

This post extends the last by showing how much different real-world scientific-consensus "messaging" campaigns are from anything that is being tested in lab experments.

All of these are exercpts from a paper I'll post soon -- one that has original empirical data relating to what measures what in the study of climate-change science communication.

* * *

5. “Messaging” scientific consensus


a. The “external validity” question.
 * * *

b.  What is the “message” of “97%”?  “External invalidity” is not an incorrect explanation of why “scientific consensus” lab experiments produce results divorced from the observable impact of real-world scientific-consensus “messaging” campaigns. But it is incomplete. 

We can learn more by treating the lab experiments and the real-world campaigns as studies of how people react to entirely different types of messages.  If we do, there is no conflict in their results.  They both show individuals rationally extracting from “messages” the information that is being communicated.

Consider what the “97% scientific consensus” message looks like outside the lab.  There people are likely to "receive" it in the form it takes in videos produced by the advocacy group Organizing for Action.  Entitled “X is a climate change denier,” the videos consist of a common template with a variable montage of images and quotes from “X,” one of two dozen Republican members of Congress (“Speaker Boehner,” “Senator Marco Rubio,” “Senator Ted Cruz”). Communicators are expected to select “X” based on the location in which they plan to disseminate the video. 

The video begins with an angry, perspiring, shirt-sleeved President Obama delivering a speech: “Ninety-seven percent of scientists,” he intones, shaking his fist.  After he completes his sentence, a narrator continues, “There’s not a lot of debate left in this debate: NASA and 97% of the nation’s scientists agree . . .,” a message reinforced by a  cartoon image of a laboratory beaker and the printed message “97% OF SCIENTISTS AGREE.” 

After additional cartoon footage (e.g., a snowman climbing into a refrigerator) and a bar graph  (“Events with Damages Totaling $1 billion or More,” the tallest column of which is labeled “Tornadoes . . .”) , the video reveals that X is a “CLIMATE CHANGE DENIER.”  X is then labeled “RADICAL & DANGEROUS” because he or she disputes what “NASA” and the “NATIONAL ACADEMY OF SCIENCES” and “ 97% of SCIENTISTS” (bloc letters against a background of cartoon beakers) all “AGREE” is true.

What’s the lesson?  Unless the viewer is a genuine idiot, the one thing she already knows is what “belief” or “disbelief in” global warming means. The position someone adopts on that question conveys who he is—whose side he’s on, in a hate-filled, anxiety-stoked competition for status between opposing cultural groups.  

If the viewer of “X is a climate denier” had not yet been informed that the message “97% of scientists agree” is one of the stock phrases used to signal one cultural group’s contempt for the other, she has now been put on notice. It is really pretty intuitive : who wouldn’t be insulted by someone screaming in her face that she and everyone she identifies with “rejects science”?

 The viewer can now incorporate the “97% consensus” trope into her own “arguments” if she finds it useful or enjoyable to demonstrate convincingly that she belongs to the tribe that “believes in” global warming.  Or if she is part of the other one, she can now more readily discern who isn’t by their use of this tagline to heap ridicule on the people she respects.

The video’s relentless use of cartoons and out-of-proportion, all-cap messages invests it with a “do you get it yet, moron?!” motif. That theme reaches its climax near the end of the video when a multiple choice “Pop Quiz!” is superimposed on the (cartoon) background of a piece of student-notebook paper.  “CLIMATE CHANGE IS,” the item reads, “A) REAL,” “B) MANMADE,” “C) DANGEROUS,” or as indicated instantly by a red check mark, “D) ALL OF THE ABOVE.”

The viewer of “X is a climate denier" is almost certainly an expert—not in any particular form of science but in recognizing what is known by science. As parent, health-care consumer, workplace decisionmaker, and usually as citizen, too, she adroitly discerns and uses to her advantage all manner of scientific insight, the validity and significance of which she can comprehend fully without the need to understand it in the way a scientist would.  If one administers a “what do scientists believe?” test after making visible to her the signs and cues that ordinary members of the public use to recognize what science knows, she will get an “A.” 

Similarly, if one performs an experiment that models that sort of reasoning, the hypothesis that this recognition faculty is pervasive and reliably steers the members of culturally diverse groups into convergence on the best available evidence will be confirmed.

But the viewer’s response to the “97% consensus” video is measuring something else.

The video has in fact forced her to be become another version of herself. After watching it, she will now deploy her formidable reason and associated powers of recognition to correctly identify the stance to adopt toward the “97% consensus” message that accurately expresses who she is in a world in which the answer to “whose side you are on?” has a much bigger impact on her life than her answer to the question “what do you know?”

 

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (7)

==> "The video has in fact forced her to be become another version of herself. After watching it, she will now deploy her formidable reason and associated powers of recognition to correctly identify the stance to adopt toward the “97% consensus” message that accurately expresses who she is in a world in which the answer to “whose side you are on?”"

I don't think that she is "forced" to become anything. Maybe I misunderstand you, but it seems to me that you are assigning a questionable causality here. It seems that you are determining that it is, at least in part, the style of the 97% messaging that makes a big difference - which is suggested to me by the extent to which you focus your post on describing stylistic elements.

I don't agree. The viewer would reach the same conclusion regardless of the style or the precise content of the messaging - because she will, necessarily, fit any information about climate change into a preexisting taxonomy.

That isn't saying that the stylistic elements you described are effective. I don't think that they are. I think that they make little meaningful difference.

June 18, 2014 | Unregistered CommenterJoshua

What I really love about this story is that the claim "our side has the science" is being backed by a misstated statistic. The 97% number refers to a percentage of papers; it would only imply 97% of scientists only if all scientists published the same number of papers.

It would be a rotten trope even if accurate, but to claim the scientific high ground with a misstated statistic... I wonder what percent of English teachers would call that an example of "irony"?

June 18, 2014 | Unregistered CommenterAn igyt

@Joshua:

I think you are assuming the "style" or other elements independent of "content" have no impact on the *meaning* of the communication in relation to the message recipient's cultural identity.

Or maybe you are simply offering this hypothesis: there is no way to modify the cultural meaning -- to remove identity threat -- w/o changing the content.

But consider.... No way to reproduce *that* lab effect in the world?

June 19, 2014 | Unregistered Commenterdmk38

Dan -

Yeah - I did think of that study as evidence that refutes my point.

If it is reproducible, and has external validity, then it would suggest that altering content can affect differential outcomes (or how "meaning" is interpreted").

And I'm certainly not convinced that there is no way to reproduce the results of your study in the real world, but I do think that it would be tricky. In the real world, how would content, such as that disseminated in your study, be delivered absent any identity aggressive/protective stimuli? What is the vehicle? Obviously not Fox News or MSNBC, or the NYT, or any of those librul/socialist/eco-Nazi/fake consensus-fabricating academics.

Besides, if that study proved to be reproducible have external validity, I might have to completely restructure my view of the climate wars, and we can't have that, now can we?

June 19, 2014 | Unregistered CommenterJoshua

Some people accept or reject the IPCC findings based on unsupported beliefs (i.e., faith or ideology). For example, it is unlikely that those who have signed the Evangelical Declaration on Global Warming would be willing to change their minds, regardless of any facts (similar to the reaction to the theory of evolution). I know of no contrasting group on the other side that would not change their mind given enough evidence (e.g., National Academy of Sciences statements).
Whether there is a significant number of people that are willing to change their minds given valid proof, might be a question, but assuming there are, attempting to portray climate change skeptics as being irrational will not make everyone that has doubts react defensively (i.e., dismiss the evidence based on ideology).

June 23, 2014 | Unregistered CommenterEdSilha

While I'm intrigued by this analysis, the science literacy of non-scientists angle would be stronger if the "beakers" weren't clearly Erlenmeyer flasks.

June 29, 2014 | Unregistered CommenterDuncan

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>