follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Authors: to assure no one can read your articles, publish in a Taylor & Francis journal! | Main | WSMD? JA! How confident should we be that what one "believes" about global warming, on 1 hand, and political outlooks, on other, measure the same *one* thing? »
Friday
Jun202014

Response: An “externally-valid” approach to consensus messaging

John Cook, science communication scholar and co-author of Quantifying the consensus on anthropogenic global warming in the scientific literature, Environmental Research Letters 8, 024024 (2013), has supplied this thoughtful response to the first of my posts on "messaging consensus." --dmk38

Over the last decade, public opinion about human-caused global warming has shown little change. Why? Dan Kahan suggests cultural cognition is the answer: 

When people are shown evidence relating to what scientists believe about a culturally disputed policy-relevant fact ... they selectively credit or dismiss that evidence depending on whether it is consistent with or inconsistent with their cultural group’s position. 

It’s certainly the case that cultural values influence attitudes towards climate. In fact, not only do cultural values play a large part in our existing beliefs, they also influence how we process new evidence about climate change. But this view is based on lab experiments. Does Kahan’s view that cultural cognition is the whole story work out in the real world? Is that view “externally valid”?

The evidence says no. A 2012 Pew surveys of the general public found that even among liberals, there is low perception of the scientific consensus on human-caused global warming. When Democrats are asked “Do scientists agree earth is getting warmer because of human activity?”, only 58% said yes. There’s a significant "consensus gap” even for those whose cultural values predispose them towards accepting the scientific consensus. A “liberal consensus gap”.

My own data, measuring climate perceptions amongst US representative samples, confirms the liberal consensus gap. The figure below shows what people said in 2013 when asked how many climate scientists agree that humans are causing global warming. The x-axis is a measure of political ideology (specifically, support for free markets). For people on the political right (e.g., more politically conservative), perception of scientific consensus decreases, just as cultural cognition predicts. However, the most relevant feature for this discussion is the perceived consensus on the left.

At the left of the political spectrum, perceived consensus is below 70%. Even those at the far left are not close to correctly perceiving the 97% consensus. Obviously cultural cognition cannot explain the liberal consensus gap. So what can? There are two prime suspects. Information deficit and/or misinformation surplus. 

Kahan suggests that misinformation casting doubt on the consensus is ineffective on liberals. I tend to agree. Data I’ve collected in randomized experiments supports this view. If this is the case, then it would seem information deficit is the driving force behind the liberal consensus gap. It further follows that providing information about the consensus is necessary to close this gap. 

So cultural values and information deficit both contribute to the consensus gap. Kahan himself suggests that science communicators should consider two channels: information content and cultural meaning. Arguing that one must choose between the information deficit model or cultural cognition is a false dichotomy. Both are factors. Ignoring one or the other neglects the full picture. 

But how can there be an information deficit about the consensus? We’ve been communicating the consensus message for years! Experimental research by Stephan Lewandowsky, a recent study by George Mason University and my own research have found that presenting consensus information has a strong effect on perceived consensus. If you bring a participant into the lab, show them the 97% consensus then have them fill out a survey asking what the scientific consensus is, then lo and behold, perception of consensus shoots up dramatically. 

How does this “internally valid” lab research gel with the real-world observation that perceived consensus hasn’t shifted much over the last decade? A clue to the answer lies with a seasoned communicator whose focus is solely on “externally valid” approaches to messaging. To put past efforts at consensus messaging into perspective, reflect on these words of wisdom from Republican strategist and messaging expert Frank Luntz on how to successfully communicate a message: 

“You say it again, and you say it again, and you say it again, and you say it again, and you say it again, and then again and again and again and again, and about the time that you're absolutely sick of saying it is about the time that your target audience has heard it for the first time. And it is so hard, but you've just got to keep repeating, because we hear so many different things -- the noises from outside, the sounds, all the things that are coming into our head, the 200 cable channels and the satellite versus cable, and what we hear from our friends.” 

When it comes to disciplined, persistent messaging, scientists aren’t in the same league as strategists like Frank Luntz. And when it comes to consensus, this is a problem. Frank Luntz is also the guy who said: 

“Voters believe that there is no consensus about global warming in the scientific community.  Should the public come to believe that the scientific issues are settled, their views about global warming will change accordingly.  Therefore, you need to continue to make the lack of scientific certainty a primary issue in the debate, and defer to scientists and other experts in the field.” 

Luntz advocated casting doubt on the consensus for one simple reason. When people understand that scientists agree that humans are causing global warming, then they’re more likely to support policies to mitigate climate change. Confuse people about consensus, and you delay climate action. 

This finding has subsequently been confirmed by studies in 2011 and 2013. But a decade before social scientists figured it out, Luntz was already putting into place strategies to drum home the “no consensus” myth, with the purpose of reducing public support for climate action. 

Reflecting on the disinformation campaign and the social science research into consensus messaging, Ed Maibach at George Mason University incorporates both the “internally valid” social science research and the “externally valid” approach of Frank Luntz:

We urge scientific organizations to patiently, yet assertively inform the public that, based on the evidence, more than 97% of climate experts are convinced that human-caused climate change is happening. Some scientific organizations may argue that they have already done this through official statements. We applaud them for their efforts to date, yet survey data clearly demonstrate that the message has not yet reached or engaged most Americans. Occasional statements and press releases about the reality of human-caused climate change are unfortunately not enough to cut through the fog—it will take a concerted, ongoing effort to inform Americans about the scientific consensus regarding the realities of climate change.

How do we achieve this? Maibach suggests climate scientists should team up with social scientists and communication professionals. What should scientists be telling the public? Maibach advises:

In media interviews, public presentations, and even neighborhood and family gatherings, climate scientists should remember that many people do not currently understand that there is an overwhelming scientific consensus about human-caused climate change. Tell them, and give them the numbers.

The book Made To Stick looks at “sticky” messages that have caught the attention in the public’s eyes. It runs through many real-world case studies (e.g., externally valid examples) to demonstrate that sticky ideas are simple, concrete, unexpected and tell a story. For a general public who think there is a 50:50 debate among climate scientists, learning that 97% of climate scientists agree that humans are causing global warming ticks many of the sticky boxes.

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (3)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (83)

Ed - "but I will give you a steady 3mm year even though the 3mm / year is likely the top end of a 60 yr cycle.

Current sea level rise is anomalous within the context of the last the last 10,000 years. Numerous paleo sea level markers indicate that for the last 4-5000 years (prior to the Industrial Revolution) the volume of water in the ocean has been static. See this lecture by Jerry Mitrovica to understand why.

June 22, 2014 | Unregistered CommenterRob Painting

"Of course the thing about Cook et al that always get glossed over by contrarians, is that we got the authors of the peer-reviewed research to rate their own papers."

Not always. It was noted, for example that in 63% of cases the ratings differed, and in 25% of cases by more than one point. Cohen's 'kappa' statistic was calculated to be 0.08%, which is a tricky statistic to judge but sounds pretty miserable. Although since as you say they were about completely different things, I suppose that's neither surprising nor very helpful. Not that I've checked Tol's figures - I'm only noting here that it didn't "always" get glossed over.

"Maybe. Or perhaps contrarians would find the scientific consensus overwhelming too."

The scientific consensus on what exactly? I tried to get out of Dan Kahan what exact definition he was using, without success. There are a whole spectrum of statements and positions, ranging from zero effect to imminent Venusian runaway, and the percentage will be different for each. The percentages will also be different for different subsets of scientists, so we have to define exactly what population we're talking about, as well. While I appreciate the efforts to filter out informed opinions from uninformed, publication count is an absolutely lousy way of doing that.

So for example, in this case it's likely that sceptics would only have counted a quantified (more than 50%, or "most" observed warming) as endorsement, since there's lots of sceptics who would say there's *some* anthropogenic warming. You don't want to include Anthony Watts in your definition of "consensus"! As I understand it, that's your category 1, for which you got 64 out of 11944? That's 0.5%?

I think sceptics would also have made a distinction between an author merely citing the consensus position as one which is generally accepted (which is worth, what?) and claiming to provide (or at least to have checked) evidence for it. Most papers are about other things entirely, and only allude to global warming in passing. How many of those endorsements are informed ones, from people who have looked at and been convinced by the evidence? Actually, this would be a far more interesting number than the count of agreements, if we want to understand how reliable the consensus is as a source of information. Tom Wigley once said: "No scientist who wishes to maintain respect in the community should ever endorse any statement unless they have examined the issue fully themselves," but he acknowledged that they often did. How often?

However, the literature suffers from a lot of sampling bias. Even besides the well-known disputes about a higher bar being set for publication of scepticism, there are a lot of reasons to mistrust a publication count. It overweights more prolific authors, it overweights topics where papers can be produced at a faster rate, it overweights areas of active research over 'settled science', it overweights less collaborative topics in which authors publish papers individually rather than jointly. It overweights journals and topics where the review process is faster. Unless specifically studying the dynamics of the publication process itself, surveys of the scientists themselves are more useful.

A lot more work needs to be done on specifying the statements being endorsed more precisely. When surveyors ask "is the globe warming?" do they mean over the past 50 years, or over the past 15 years? Do they mean "significant" warming, and if so, according to what statistical model? So for example, if someone believes the global temperature did increase over the 20th century but that the data fits a model with a unit root and the trend is not significantly different from zero, should they be put into the same category as someone who thinks the thermometer measurements are screwy because they're stood next to the air conditioning?

When someone expresses agreement with a lower confidence, does that mean that they personally are uncertain of the answer, or that they are themselves very confident that the data is not available and nobody knows?

You also need to - and I cannot emphasise this enough - start asking *why* scientists believe as they do. Have they examined the issue fully themselves? Have they picked it up from the media, or the general scientific gestalt, or directly from talking to climate scientists, or from the literature? Have they just read the literature or have they downloaded data and checked it? How much climate science do they know? Don't use proxies like publication count for that - test them! (Or at least, validate the proxies.)

And so on. There's a vast amount of work to be done before you can even work out what the questions should be. You need to first do a more free-form survey of opinions to identify all the nuances of position and reasoning, and only then try to construct Likert-like multiple choice categories to capture them.

"Along with two previous research papers, Doran & Zimmerman [2009] and Anderegg et al [2010], that gives four approaches to estimating the scientific consensus with regard to global warming that give a value of around 97%. This suggests that our result is robust."

Doran and Zimmerman got 82% for the general "scientific consensus" figure - with other figures given for different subsets such as the highly-published climate scientists (94.9% with a very small sample, 95% CI 89.3%-98.7% using a Beta prior) and the meteorologists (63.9%, with 95% CI 48.1%-78.5%). It's apparent that it's very sensitive to the precise details of the population selected, and error bars are broad. That suggests that such results will not be robust. Simply getting the same (wrong) answer as someone else is not a sign of robustness! Anderegg has such a peculiar (and heavily biased) sampling method that I can't see how you can get any valid estimate of consensus percentages out of it.

A much better survey, both from the point of view of sampling (although criticisms are valid) and especially for the quality of the questions asked was Bray and von Storch, 2010. I think (from memory) that gave around an 85% endorsement, similar to Doran and Zimmerman. They also asked the more useful 'Is it dangerous?' question and again got around 80% endorsement.

There are a lot of interesting questions to be asked. It's a pity the topic is so distorted by the politics.

June 22, 2014 | Unregistered CommenterNiV

The relevant question in Doran & Zimmerman (2009) was:

Do you think human activity is a significant contributing factor in changing mean global temperatures?

97.4% of the people most qualified to answer this question, the climate scientists, answered yes.

As for Tol's critique of Cook et al, it's riddled with errors, but conjuring up 300 rejection papers from thin air, in order to lower the consensus to 91% is the largest flaw. He's been asked numerous times about this, but keeps avoiding it. That's not surprising, he has form for that sort of behaviour.

June 22, 2014 | Unregistered CommenterRob Painting

NiV -

You are pretty familiar with the literature on climate change. Can you point to any papers which use the results of GCMs as evidence in (at least partial) support of the authors conclusions but where you doubt that the authors would endorse the argument that ACO2 very likely explains more than 50% of recent warming?

==> " it overweights less collaborative topics in which authors publish papers individually rather than jointly. "

Why? Because it takes longer to write an article collaboratively? Do you know that is true on average? Seems to me that other factors affect the speed of production more strongly, and also dividing up the work to be done could shorten the time needed. Because journals prefer articles with multiple articles? It seems to me that the status of the authors is more important than the number. And then you might consider that having multiple authors may well have the effect of making articles better written and more robust - thereby meaning that articles with multiple authors might be more likely to get published(assuming you believe that being better written and more robust correlates with being more likely to get published).

June 22, 2014 | Unregistered CommenterJoshua

Rob -


"Skeptics" like to hang their hat on the theoretical argument that perhaps more than 3% of climate scientists think that ACO2 make a "significant" contribution to recent warming but would not believe that it is very/extremely likely that it causes more than 50% of the recent warming.


Personally, I think that this point made by Richard Tol is something better to hang my hat on:

“Published papers that seek to test what caused the climate change over the last century and half, almost unanimously find that humans played a dominant role.”

June 22, 2014 | Unregistered CommenterJoshua

"Cohen's 'kappa' statistic was calculated to be 0.08%, which is a tricky statistic to judge but sounds pretty miserable. Although since as you say they were about completely different things"

No, what I wrote was:

"Not that these two methods are measuring exactly the same thing of course, the full paper likely contains more information with which to make a determination."

This borne out by the research - where a large number of papers that we rated as neutral were rated as endorsements by the authors of the papers themselves. They are two methods by which to measure the level of endorsement in the scientific literature, but they are not the same.

So anyone using Kappa statistic to determine inter-rater agreement on data which is not the same, doesn't understand this statistical test.

June 22, 2014 | Unregistered CommenterRob Painting

Rob ref - Zimmermann (Doran Q2)

"Do you think human activity is a significant contributing factor in changing mean global temperatures?"

Here is some feedback form the scientists that actually took part in the survey cited by Doran/Zimmermann's EOS paper, with respect to that question and 'significant' (appendix F)

the survey was M Zimmermann's MSc thesis - "The Consensus on the Consensus"
http://www.lulu.com/shop/m-r-k-zimmerman/the-consensus-on-the-consensus/ebook/product-17391505.html


“Questions 1 asks if I think temperatures are warmer than the 1800s, but doesn’t indicate if I’m supposed to compare to today, the last 10 years, the last 50 years, or… Without telling me what I’m comparing to, I cannot answer the question.

Q2 then asks if I think that humans are “a significant” contributor to warming temperatures, but I can only answer yes or no. I happen to think that we are one among many contributing factors, so I answered yes, but I couldn’t explain this. The third question then asks me why I think humans are a major contributor, but is phrased in such a way that it’s implicit that I’m now listing them as THE significant factor. They are not the primary cause, but I had to stop the survey at this point because it was forcing me to answer queries about why I think they are.

As constructed, your responders will be unable to indicate that there are multiple causes to climate change, that climate change is the norm on Earth and has been going on throughout geologic time, and that there is strong evidence to indicate that climate change not only occurred before humans existed, but also was probably more extreme than the event we are living in today.”

And:


Your use of the word ‘significant‘. It seems clear that human activity has caused an increase in CO2 levels. That, in theory, might have caused an increase in global temperature. However, did it? If so, was it the only cause? If it was a cause, was it a significant cause?

And:


Not Fair: You changed the question from ‘significant’ to ‘contributing’ Significant= 25%. Contributing=75%

“What defines significant? If 1-2 degrees F is considered significant then I would agree that human input is significant

“what do you mean by significant? Statistically? A player in the total rise? sure we are! How much? I am not sure.

What is meant by significant? A major contribution, yes, but what is human activity compared with increased solar activity. So far, it is lost in the statistical models. While it certainly seems likely that human activity is at least partly responsible, I am not aware of data conclusively proving this. It has been documented that natural earth temperature cycles occur with, or without, human-based effects.

I entered an answer I did not intend. I think human activity is a significant component, but I do not know if it is 10%, 25%, 50% or more. (3c)


“I appologize, but as an objective scientist I do not communicate “opinions” or “attitudes”. These do not belong on the scientific agenda and certainly not in the classroom. Thus I decline to contribute to your survey.” (Zimmerman feedback)


-------------
lots more feedback from the participants in the 'Doran survey' in the actual survey paper itself
http://www.lulu.com/shop/m-r-k-zimmerman/the-consensus-on-the-consensus/ebook/product-17391505.html

June 22, 2014 | Unregistered CommenterBarry Woods

Rob,

"97.4% of the people most qualified to answer this question, the climate scientists, answered yes."

Wrong! As I explained just above it was 94.9%.

And the actual criteria were "those who listed climate science as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change". That's not the same thing as "climate scientists", let alone "the most qualified". If you want to go count up Doran's participants' qualifications and come back to me, please do. But that's not what the survey data as published shows.

And more to the point, when people say "97% of scientists say...", as many people do, they're wrong to use this statistic in support. The qualifications matter! The size of the differences in the result show that. That's my point.

"So anyone using Kappa statistic to determine inter-rater agreement on data which is not the same, doesn't understand this statistical test."

The differences indicate that the abstracts don't correctly represent the stance of the papers themselves, and therefore don't represent the literature. That they happen to yield the same percentage may be coincidence - they're measuring different quantities. And so you can't use the measurement of the contents of the papers to support a contention about the accuracy of a measurement of the abstracts.


--


Joshua,

"You are pretty familiar with the literature on climate change. Can you point to any papers which use the results of GCMs as evidence in (at least partial) support of the authors conclusions but where you doubt that the authors would endorse the argument that ACO2 very likely explains more than 50% of recent warming?"

As I said above, I rather doubt most sceptics who publish would disagree that it was "likely", if put into the proper terms. The CO2 rise 1950-2000 or 1950-2014 is an increase of either 23% or 29%, which is respectively 0.3 or 0.37 of a doubling. The rise in temperature over the same time is 0.6 C. So if "most" of that is anthropogenic, the transient sensitivity is at least 1.0 C/2xCO2 or 0.8 C/2xCO2, depending on time period. Everyone knows that observations are falling somewhat short of the models, this is another way of saying that.

So you would be looking for sceptics who claim transient sensitivities below 1 C/2xCO2 are likely. I can't think of any who have said so recently, most seem to go with Nic Lewis-like estimates around 1.0-1.5 C/2xCO2, but I do recall Idso reporting empirical estimates around 0.4-0.6 back in 1998. I don't know if he would still support that, or even if it counts. It's not quite the same thing.

In fact, I rather suspect the results from Doran etc. are a gross over-estimate of the people who actually believe such a thing. I would expect close to 100% of those asked believe it, but that people knowing what political use such survey results will be put to are actually answering the question "are you a sceptic?" rather than the one actually asked. They complained about how bad the questions were, as Barry points out above.

"Why? Because it takes longer to write an article collaboratively?"

No, because I was thinking n authors can produce n papers in a given time period writing them individually, but only n/m papers writing them in groups of m. It doesn't work out quite like that, but jobs requiring teams to do generally take more work in total per item produced. There are also the inefficiencies of large teams - waiting for other team members to do their bit, holding progress meetings, management overhead, seeking agreement, and so on.

June 22, 2014 | Unregistered CommenterNiV

I took a look at the - The ‘expertise’ of the 97% - a while back (ref Doran)
http://wattsupwiththat.com/2012/07/18/what-else-did-the-97-of-scientists-say/

On occasion when challenged about the 97% figure depending on 75 scientists from a survey of 10,000, it is usually met with a response that these were the experts in the field of climate science and this is what maters not the number that took part. A closer look at the methodology perhaps raises some concerns about the ‘expertise’ and selection bias as this as his result depends on 2 additional questions in the survey that were used to identify expertise in climate research (not an unreasonable goal) within the respondents


Q5 Which percentage of your papers published in peer reviewed journals in the last 5 years have been on the subject of climate change?

A: 1) less than 50% 2) 50% or more 3) not applicable

Q9 Which category best describes your area of expertise?

1) Hydrology/Hydrogeology 2) Geochemistry 3) Geophysics

4) Paleontology 5) Economic Geology (coal/metals/oil and gas)

6) Soil Science 7) Oceanography/MarineGeology

8) Environmental Geology 9) Geology/Planetary Science

10) Climate Science 11) Geomorphology 12)General Geology

13) Structure/Tectonics* 14) Petrology*

15) Sedimentology/Stratigraphy 16 Atmospheric Science*

17) Quaternary Geology* 18) Meterology*

19) Geography/Archeaology/GI 20 Engineering (Envr/Geo/Chem)*

21 Ecology/Biogeochemistry* 22) Glacial Geology*

23) Mineralogy* 24) Volcanology* 25) Other (*write in description)

(Zimmerman)

The survey used the answer to Q5 narrow down the expertise of the respondents, not unreasonably perhaps, and defined these as ‘active climate researchers’ (ACR), there was also criticism of the framing of this question in the feedback. This subset of respondents were then contacted to check the these claims and once verified, there were 244 respondents that met this criteria. This categorisation gave positive responses to Q1 – 95% and Q2 - 92%

The survey used the answer to Q9 to define those as [self] identifying as in the category of climate science (11) as having more expertise than the other listed categories. Question 9 resulted in 144 respondents self identifying in the category of climate science. This categorisation gave positive responses to Q1 – 95% and Q2 – 88.6%

Finally a category of experts was defined as those that responded as publishing more than 50% of papers AND self identifying in the survey as climate scientists, resulting in a group of 77


This categorisation gave positive responses to Q1 – 96.2% and Q2 – 97.4%

So is [Doran] Zimmermann defining expertise or introducing a selection bias here ? It has not gone unnoticed that perhaps those scientists that self identify as climate scientists, are perhaps those that are more activist minded for a consensus.

It is quite possible for example, in this survey for scientist or even colleagues with identical qualifications, to self identify differently. Thus in this survey respondents could even be co-authors of a paper, but this survey would categorise one as more expert than the other. Who knows if this happened or not, the fact that it is possible demonstrates the flaws in the thinking.

Additionally those that are in the 97% group are deemed to be more expert in climate science, keeping more abreast of the ‘whole’ field than the others.


“..The participants in this group are actively publishing climate scientists, and those most likely to be familiar with the theory and mechanisms of climate change, as well as have a thorough understanding of the current research and be actively contributing to the field..” (Zimmermann)

This I think is a huge assumption, ‘climate science’ is a huge multidisciplinary field.

Is a geologist that identifies as a ‘climate scientist’ any more an expert on astrophysics, atmospheric physics, statistics, etc than those classified as have less expertise in the categories identified above.

Additionally the responses may merely capture (only the last 5 years publishing Q5) those junior more activist post docs, etc that self identify as climate scientist, where perhaps the older more published ‘expert’ colleagues describe themselves by the qualifications, not as climate scientists. And of course, by the very nature of the survey, (which was commented on in the feedback) surveys of this type are potentially self selecting by the probability that those that are most concerned are more willing to take part.
-----------------------

June 22, 2014 | Unregistered CommenterBarry Woods

Rob [ Ed - "but I will give you a steady 3mm year even though the 3mm / year is likely the top end of a 60 yr cycle. Current sea level rise is anomalous within the context of the last the last 10,000 years. Numerous paleo sea level markers indicate that for the last 4-5000 years (prior to the Industrial Revolution) the volume of water in the ocean has been static. See this lecture by Jerry Mitrovica to understand why. ]

Rob, interesting, but irrelevant
Per the IPPC, humans have only had a significant effect on the earths climate since about 1950. All climate effects prior to about 1950 are natural in origin per the IPCC.

June 22, 2014 | Unregistered CommenterEd Forbes

Barry,

"Finally a category of experts was defined as those that responded as publishing more than 50% of papers AND self identifying in the survey as climate scientists, resulting in a group of 77. This categorisation gave positive responses to Q1 – 96.2% and Q2 – 97.4%"

Note the denominators in what D&Z say:

"In our survey, the most specialized and knowledgeable respondents (with regard to climate change) are those who listed climate science as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change (79 individuals in total). Of these specialists, 96.2% (76 of 79) answered “risen” to question 1 and 97.4% (75 of 77) answered yes to question 2."

There were 79 in the group, but two of then said no to question 1, and therefore were not asked question 2. There's no point in asking if you think the rise is anthropogenic if you've just declared you don't think it's risen. That means there were only 77 who were asked question 2. That's my understanding of the situation, anyway. I'd be interested if you know any different.

So the proper percentage is 75/79, rather than 75/77, unless you want to add "and thinks global temperatures have risen" to the long list of caveats on this subset.

June 22, 2014 | Unregistered CommenterNiV

I would love to know who the 2 were.. perhaps just rebelling against the questions?

Doran's co-author, Zimmermann seemed to have learnt something from her research (cf Cook et al)

“This entire process has been an exercise in re-educating myself about the climate debate and, in the process, I can honestly say that I have heard very convincing arguments from all the different sides, and I think I’m actually more neutral on the issue now than I was before I started this project. There is so much gray area when you begin to mix science and politics, environmental issues and social issues, calculated rational thinking with emotions, etc.” - M Zimmermann - The Consensus on the Consensus (Doran/Zimmermann)

I doubt if Dana and John Cook and the other 7 Skeptical Science co-authors of Cook et al have learnt anything, or are more neutral

June 22, 2014 | Unregistered CommenterBarry Woods

"I would love to know who the 2 were.. perhaps just rebelling against the questions?"

Possibly. I suspect that they just interpreted the question differently. Presuming you're only counting it as a rise if it is "significant", then those who don't regard the 20th century rise as statistically significant might reasonably answer "not risen". Or possibly they meant "not proven". We can't tell.

Actually, looking again at those numbers, it would appear that at least one person answered "not risen" to the first question, but answered the second one anyway. Maybe they went back to the MWP and said temperatures had "fallen"? Odd...

June 22, 2014 | Unregistered CommenterNiV

@Barry Woods:

Doran's co-author, Zimmermann seemed to have learnt something from her research

That quote is often cited by "skeptics", but I don't think it means quite what they imagine it does. The "neutrality" in question there is not, as "skeptics" seem to want to spin it, about whether anthropogenic warming is real. The quote is taken from a response from Zimmerman to someone asking whether she had already decided what the outcome to the survey would be before the results were received, due to the choice of language, and also warned Zimmerman to guard against possible bias. Zimmerman discusses how she strove to choose the most neutral, appropriate language, and that she had in fact become more neutral in her overall orientation. The context of this has nothing to do with causation of warming, but is rather about the sort of professional detachment that is encouraged when formulating and analyzing survey data.

It's possible that ZImmerman did herself become more neutral on the question of the cause of warming during the course of her finding overwhelming expert agreement. But I think if one wanted to demonstrate that to be the case, it would be best to ask her directly, rather than to take an answer she gave about a different topic out of context.

I doubt if Dana and John Cook and the other 7 Skeptical Science co-authors of Cook et al have learnt anything, or are more neutral

Speaking only for myself, I indeed learned a great deal during the course of analyzing the scientific literature and writing the manuscript. For example, I learned, during the literature review for the manuscript, about the expected trajectory of consensus endorsement over time- as the consensus becomes more established papers directly addressing the issue become less frequent, leading to a growing percentage of neutral papers while the consensus strengthens. This is one of those "obvious in hindsight" things that I slapped my forehead for not anticipating prior to reading about it.

As to whether I've become "more neutral", as in the actual rather imagined context of Zimmerman's quote, I'd say during the rating I was. Something about the rating process puts you (or me, anyway) in a dispassionate, analytical place mentally.

But in the imagined context (as in, with respect to the issue of human causation), I have become much more confident of the consensus position over time. The more I learn about the response of different components of the climate system to relatively small changes in radiative forcings, especially on longer timescales, the more concerned I have become. Likewise, the more I have learned about paleoclimatic and paleoecological changes on geologically rapid timescales, the more concerned I have become.

At the same time, I have probably become more policy neutral. I no longer have a preferred policy solution to the problem, and am now an unabashed utilitarian. Nuclear, GMOs, carbon capture, and all of those things that contrarians wonder why people don't support if the problem is as big as we say? If they work, I support them. I have also become much more optimistic that we will avoid the worst case scenario emissions trajectories, in a large part due to the growing movement towards evidence-based communications efforts.

On the whole, I am more convinced, more concerned, more policy neutral, and more hopeful. I know some people would call that combination more alarmist, some would call it more neutral, and others would call it putting my head in the sand.

I hope you find that response helpful.

June 22, 2014 | Unregistered CommenterPeter Jacobs

Rob – Of course the thing about Cook et al that always get glossed over by contrarians, is that we got the authors of the peer-reviewed research to rate their own papers. They did this based upon their full paper, not on the abstract - as we did. They gave a consensus value of 97.2%, where our abstract ratings found it was 97.1%.

Yea instead of a subset of the subset (to get 97%) they got a subset of the subset of the subset to get the 97%. From memory – Your group emailed 2,800+ authors & 1,200 + answered. Not very convincing IMHO.

Along with two previous research papers, Doran & Zimmerman [2009]

So at least 5 years of having the 97% message and still not able to convince folks there’s a disaster coming. I would try something knew.

I want to thank you for your participation in this discussion & others you have been in on different blogs & articles. Your not a “Homeboy.” (only post on sites that agree with you) I’ve never seen you be abusive (unless abused first) and you (from what I’ve seen) pretty much will stick around and defend your point of view. I don’t agree with your view, but you are reasonable when presenting it.

June 22, 2014 | Unregistered CommenterClyde

Out of context?

well Zimmermann writes that she received 500 emails from participants, and she included them in Appendix F...

She received lots of very sceptical feedback from survey participants, why not read it all.. so anybody can read for themselves and make their own judgement about that quote.

http://www.lulu.com/shop/m-r-k-zimmerman/the-consensus-on-the-consensus/ebook/product-17391505.html

And the reason why lots of sceptics quote it is probably because of this Watts Up article.. (I'm the author)
http://wattsupwiththat.com/2012/07/18/what-else-did-the-97-of-scientists-say/

which criticises how the paper is misused by others, who go far beyond what the paper actually found/looked at.. much like when Obama claimed Cook et al (all co-authors from) Skeptical Science, showed 97% of scientists agreed 'dangerous'.

now as one of Skeptical Science contributors (Michael Marriott, of Watching the Deniers blog fame) publically labelled my WUWT article 'Verified Bullshit' which Skeptical Science endorsed this, it is perhaps best that any reader here, look at the original paper and make their own judgement.

http://www.lulu.com/shop/m-r-k-zimmerman/the-consensus-on-the-consensus/ebook/product-17391505.html

I would hope that a reader would check for themselves, rather than just take for granted anything Rob, Andy, Peter, Dana & John (from Skeptical Science), or indeed myself (a WUWT contributor) has to say

June 22, 2014 | Unregistered CommenterBarry Woods

Barry, I guess I am also a member of the 97% as to how I would answer these 2 questions, with "significant" = 25%.

Q1: "When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?" RISEN
Q2: "Do you think human activity is a significant contributing factor in changing mean global temperatures?" YES

Q1 Warming coming out do the LIA should be considered a good thing
Q2 Human activity causes all sorts of of climate issues, clear cutting forests and major cities with their urban heat islands being 2 such.

Now do I consider the increase in CO2 to be significant (25% of temp increase due to the increase in CO2); NO

June 22, 2014 | Unregistered CommenterEd Forbes

Clyde - "So at least 5 years of having the 97% message and still not able to convince folks there’s a disaster coming. I would try something knew."

This is similar to the claim made by Dan Kahan. I don't know whether your claim, or Dan's, is true because no one has presented evidence in support. If there is some peer-reviewed research that has shown that the consensus message has been used in the public domain, and hasn't worked, I would certainly be interested in seeing it.

Only anecdotal I know, but my perception is that the 97% consensus is beginning to branch out beyond the blogosphere. Not only did President Obama's twitter account link to our research, but comedians, such as John Oliver, have too. I see it popping up more and more, but that could be that I'm now 'conditioned' to notice these references rather than any actual trend occurring.

Like I said, some peer-reviewed research might shed some light on this issue.

June 23, 2014 | Unregistered CommenterRob Painting

Niv - "The differences indicate that the abstracts don't correctly represent the stance of the papers themselves, and therefore don't represent the literature."

No. The Kappa statistic only applies when raters are categorising the same data. It cannot, therefore, be used in the case of the Cook et al raters and the author self ratings. They are not measuring the same thing - they are abstracts versus the full paper. So of course applying the wrong statistical test is not going to be informative.

I find this argument from you quite incoherent. Earlier you stated: "Although since as you say they were about completely different things, I suppose that's neither surprising nor very helpful."

If you agree that the Cook et al ratings and the author self-ratings are not the same, you also implicitly agree that the kappa statistic is uninformative, i.e. it's the wrong test to perform.

You claiming that "The differences indicates..... is a contradiction of your admission that they are not the same thing. The kappa statistic cannot be uninformative (the wrong test) and yet informative (the right test) at the same time.

June 23, 2014 | Unregistered CommenterRob Painting

yes


Obama linked to your research,he misrepresented it, saying 97% of scientists say dangerous, and the authors made no efforts to correct that misinformation..

is there any research into the effectiveness of consensus messaging to the public, when the public find that the message is false/misleading/misinformation..

June 23, 2014 | Unregistered CommenterBarry Woods

"I find this argument from you quite incoherent. [...] If you agree that the Cook et al ratings and the author self-ratings are not the same, you also implicitly agree that the kappa statistic is uninformative, i.e. it's the wrong test to perform."

I agree! I was arguing with the implication above that the rating of papers by authors supported the "robustness" of the ratings of abstracts.

"One suitable measure as to the worthiness of abstract ratings would be to compare the contrarian ratings to that of the authors of the papers. [...] Of course the thing about Cook et al that always get glossed over by contrarians, is that we got the authors of the peer-reviewed research to rate their own papers. They did this based upon their full paper, not on the abstract - as we did. They gave a consensus value of 97.2%, where our abstract ratings found it was 97.1%."

The near-coincidence of the two numbers is apparently taken to be a "suitable measure as to the worthiness of abstract ratings", both of them measuring a single quantity called the "consensus value".

I was pointing out that the two were measuring different things (as had been acknowledged), and while the percentages might be similar, the observed values did not match in detail. The kappa statistic is statistical evidence that they do not match, should there be any doubt. I wouldn't expect them to be the same, and neither should you, and therefore nothing should be read in to the near-coincidence of the percentages. It's not evidence of "worthiness" or "robustness" of the abstract count, or anything of the sort. (Or evidence against it.)

Nor is there a single number called the "consensus value" which both of these estimate. The degree of "consensus" depends on precisely what question you ask (or rather, what question the subjects think you're asking), and what sub-population you're talking about, with a different number for each combination. Both D&Z and Anderegg were asking different questions (whose interpretation was ambiguous and disputed), and there's no a priori reason to expect the numbers to be the same.

June 23, 2014 | Unregistered CommenterNiV

Rob – This is similar to the claim made by Dan Kahan. I don't know whether your claim, or Dan's, is true because no one has presented evidence in support.

If it was effective then the public would be more inclined to support raising taxes on carbon based fuel or other solutions to the so called problem. (Cook’s reasoning not mine) No need to peer-review everything to know if its true. Based on peer-review Cook’s 97% paper has been debunked. I’m gonna guess you don’t agree with those peer-reviewed articles.

Do you have peer-reviewed evidence we needed another paper claiming 97% consensus?

Not only did President Obama's twitter account link to our research, but comedians, such as John Oliver, have too. I see it popping up more and more, but that could be that I'm now 'conditioned' to notice these references rather than any actual trend occurring.

No need to worry about that. The 97% consensus gets plenty of exposure. That was the point I was trying to make. It’s been on every major nightly news cast in America. More blogs than I can count, more internet news articles than I can count etc etc. That was even before the Cook paper came out. If I’m not mistaken the site you blog at most SS has a blog about how much press coverage it got.

June 23, 2014 | Unregistered CommenterClyde

Clyde - "No need to peer-review everything to know if its true."

That's an absurd claim to make. The whole point is that we don't know that it's true. The Catholic church once knew the Earth was the centre of the universe, but we now know that that isn't true. This is the part where actual scientific evidence can provide answers.

Given that neither you, nor Dan Kahan, can provide references to the peer-reviewed literature backing up your claims, maybe it doesn't exist? Maybe consensus messaging does work? We won't know until someone actually researches the subject.

June 23, 2014 | Unregistered CommenterRob Painting

"Given that neither you, nor Dan Kahan, can provide references to the peer-reviewed literature backing up your claims, maybe it doesn't exist?"

Why would you look for "peer-reviewed literature" to back up claims? Surely what you should be looking for is "evidence"?

June 23, 2014 | Unregistered CommenterNiV

John Cook could you answer a question about the graph shown above with its axis labelled "Perceived Consensus" against "Free Market Support?"

Could you describe how you ascertained the measure of your subjects "Public perception of scientific consensus of climate change"?

If it was via a questionnaire, could you tell us the form and wording of that questionnaire?

Thanks.

June 24, 2014 | Unregistered Commentertlitb1

Rob – Given that neither you, nor Dan Kahan, can provide references to the peer-reviewed literature backing up your claims, maybe it doesn't exist? Maybe consensus messaging does work? We won't know until someone actually researches the subject.

Are you seriously claiming that unless somebody writes a paper & submits it to peer-review you won’t believe something?

If it was effective then the public would be more inclined to support raising taxes on carbon based fuel or other solutions to the so called problem. (Cook’s reasoning not mine)

So is Cook's reasoning flawed?

June 24, 2014 | Unregistered CommenterClyde

This discussion glosses over a crucial element in the messaging of climate science. Luntz is far more successful because he has the tools of Fox News and the right wing radio talk shows at his disposal to barrage his target conservative audience. Rush Limbaugh and Fox news are they vehicle to allow this to happen daily-
"You say it again, and you say it again, and you say it again, and you say it again, and you say it again, and then again and again and again and again, and about the time that you're absolutely sick of saying it is about the time that your target audience has heard it for the first time. And it is so hard, but you've just got to keep repeating, because we hear so many different things -- the noises from outside, the sounds, all the things that are coming into our head, the 200 cable channels and the satellite versus cable, and what we hear from our friends.”

June 26, 2014 | Unregistered CommenterGary Williams

"This discussion glosses over a crucial element in the messaging of climate science. Luntz is far more successful because he has the tools of Fox News and the right wing radio talk shows at his disposal to barrage his target conservative audience."

Al Gore had his own TV channel. An entire TV channel, all to himself! And the consensus generally has the rest of the media, not to mention all the governments, politicians, environmental NGOs, Euro-bureaucrats, UN bureaucrats, organisations of scientists like the Royal Society and the NAS, the EPA and other regulators, PR people, advertising agencies, protesters, 'corporate responsibility' officers, local government, renewable energy firms, and - last but by no means least - science communication scientists and pundits working to hone the climate science message to be more persuasive.

And yet, despite more than four decades of constant repetition, not even all liberals are persuaded, let alone conservatives.

We live in a world saturated with advertising. People have got quite good at recognising it. There comes a point when repetition has done all it can, and it shifts towards being counterproductive. But please don't believe me, and do carry on. :-)

June 26, 2014 | Unregistered CommenterNiV

Cook writes:
"My own data, measuring climate perceptions amongst US representative samples, confirms the liberal consensus gap. The figure below shows what people said in 2013 when asked how many climate scientists agree that humans are causing global warming. The x-axis is a measure of political ideology (specifically, support for free markets)."

I'm wondering if the definitions of liberal and conservative along the lines of free market support is the most informative way to determine a liberal consensus gap (or Conservative consensus gap for that matter). Rather than along an economic political position, would identification of liberal/conservative along positions on social issues be equally informative, more so, or less? Why was economic position chosen as the determining factor of political identity? Would a position on market regulation be as salient an identifier of liberal vs conservative in this instance, as say, whether one believes oneself in AGW or not?

June 27, 2014 | Unregistered CommenterCWG

"I'm wondering if the definitions of liberal and conservative along the lines of free market support is the most informative way to determine a liberal consensus gap..."

That's assuming that John Cook has measured "Free market" belief in a reliable way, I wouldn't be so sure. We don't know do we? ;)

However, assuming he has, I would still be far more interested in what precisely he asked the public when trying to ascertain the Y -Axis measure of their 'Perceived Consensus'.

I ask this because the alleged 'information deficit' shown there depends on the assumption that both the public and the scientists are considering the same information; i.e. they have been asked about the same 'consensus'.

Since the same author is providing both lines in that graph - showing the result of two of his studies - one showing the level of scientist of adherence to a consensus; and the other showing the public belief that scientists adhere to a consensus tracking against some other public ideology measure; we should be ideally placed to see a comparison right?

Well we can't because John Cook nowhere tell us how he has measured the Y-Axis of 'Perceived Consensus' of consensus in his second unpublished PhD study.

I've asked for background information of this graph in several places now: on the Guardian where the graph appears in an article by 97% co-author Dana Nuccitelli, here, Skeptical Science, and on twitter; and John Cook, Dana Nuccitelli, nor our host Dan Kahan here have offered any explanation of the origin of this data.

Outside the media spin that depict their study as having shown a consensus as being something like 97% of scientists hold the opinion that warming is >50% due to humans, or even "dangerous" when they have President Obama's help, and that Cook and his fellow authors help promulgate, the reality is that the Cook study doesn't demonstrate anything like a strong consensus on AGW quantity at all, let alone danger.

If you understand that the sub survey of the actual scientists has a 97% 'consensus' which consists of half of the scientists merely describing their work as showing:

Implicit Endorsement: paper implies humans are causing global warming. E.g., research assumes greenhouse gases cause warming without explicitly stating humans are the cause.

The you might understand why their consensus studies power is a lot weaker than their spin would lead you to believe.

And you might then understand why it is important to know what "consensus" exactly John Cook ask his 'US representative sample' to consider above.

For example I wonder, did John Cook ask the public this:

"Do you, or do you not, believe that scientists assume greenhouse gases cause warming without explicitly stating humans are the cause?"

This would be about fair I think if he was to compare its result to his own 97% study and in order to claim to show a deficit of belief.

Do you think the comparison is silly?

Then how would you do it?

Would you just draw a straight line and then a curve underneath without telling how you derived the curve? ;)

June 27, 2014 | Unregistered Commentertlitb1

==> "Why would you look for "peer-reviewed literature" to back up claims? Surely what you should be looking for is "evidence"?"

Wow. Look again at what he wrote. He asked for peer reviewed literature backing up [your and Dan's] claims.

That is asking for evidence - evidence of a certain sort. Evidence that has gone through at least some sort of vetting.

Why would you truncate his comment like that? I would consider that you did so evidence. Can you guess what I think it is evidence of?

June 30, 2014 | Unregistered CommenterJoshua

Rob Painting said:

"If there is some peer-reviewed research that has shown that the consensus message has been used in the public domain, and hasn't worked, I would certainly be interested in seeing it."

It only works on people who don't bother to read papers like Cook et al 2013 (like people who work at the White House and tweet on behalf of the President). Because people who read papers like Cook et al 2013 pick up on the fact that ONLY 64 abstracts out of 11,994 abstracts actually openly agreed with AGW and quantified it as greater than 50%.
And the fact that the 97% is actually 97% of 33% of abstracts that stated an opinion one way or another.
And the fact that 66% of abstracts said NOTHING about the causes of global warming, or how much etc.
And the fact that Cook and Company based their entire paper on the fabulously illogical idea that one can read an abstract written by multiple authors from multiple scientific disciplines and determine, based on their own interpretations, how/what those authors believe about global warming/climate change with some kind of accuracy in the first place.

And the most damning thing of all? That the authors and contributors of Cook et al 2013 ALLOW the actual findings and the limited evidence examined in their paper to be misrepresented over and over and over again on the internet and in the media and even by themselves, without uttering a word of clarity.

If "scientists" want people to believe in their research and trust them, maybe they should be willing to be brutally honest about their work, no matter what that work exposes. Otherwise they are viewed no differently than campaigning politicians or propaganda outlets. The idea that all "scientists" have to do is frame their message correctly and the masses will suddenly accept and embrace it, is arrogance and condescension of the highest order and even "common folk" know when they are being played.

August 22, 2014 | Unregistered CommenterSaudadia

Hi Dan -

you seem to believe the spin put on that quote... have you read the entire memo, for yourself

As far from trying to advocate doubt to the public -

In the memo he stated that the public doubted 'consensus messaging',he was recognising that the public did NOT believe in the consensus, and was trying top put forward ways to engage those people (as this was 2001, 'consensus on science was weaker then, anyway)

So he has been spun to say the exact opposite! by activist and people that blindly quote the quote (fulfilling own preconceptions), without realising it is totally out of context and spun to mean the opposite..

the memo, even discussed how republicans had been 'framed' as evil, and a ridiculous caricature by democrats, and that emotions were ahead of fact.. as there was no 'consensus' he recommended that Republicans stick to the science, quoting the science - Again the complete opposite of the spin, that has been put on that quote.

Please read the entire memo, and especially the recommendations..

read the whole memo.
http://nigguraths.files.wordpress.com/2013/03/luntzresearch_environment.pdf

Shub's take on it.
http://nigguraths.wordpress.com/2013/03/19/the-story-of-the-frank-luntz-memo/


Perhaps you might find it cynical ie how to better engage ratings wise on environmental issues - but they are saying not anti-environmental, but they had a different pro-environment strategy - that they thought was better - ie conservation is good - 'environmentalism' has negative activist connotations with voters -

this is a lot less cynical than Futerra recommendation to the UK government that they should Act is if the science s settled and that their is a consensus for public consumption (I think That position is worse, Futerra is going beyond science, Lunz was saying to acknowledge the uncertainty, but to always refer to science, which is just exactly what scientist are doing now - Tamsin Edwards recent TED-Cern talk for example)

in the memo, he was saying that weather event were being pointed at as global warming' and however many facts (the science) you point out, that this could not counter this emotional response!

so please read the whole 16 page memo....
http://nigguraths.files.wordpress.com/2013/03/luntzresearch_environment.pdf

than just unquestionably quote the spin put on it

November 22, 2014 | Unregistered CommenterBarry Woods

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>