follow CCP

Recent blog entries
« Local adaptation & field testing the science of science communication | Main | The two-channel strategy/model for satisfying the public's appetite to know what is known by science »
Tuesday
Feb192013

On science communication & the job of the scientist: a thoughtful response from a scientist

Below is an extremely thoughtful comment relating to my 2d post on my experience in giving a presentation to a group of public-spirited citizen scientists at the  North American Carbon Program a couple of weeks ago.

Just by way of context: I stressed that it is a mistake to think that the job of the scientist is to communicate as opposed to doing science -- not because scientists shouldn't communicate with the public (the ones who take that on that added demand are heroes in my book) but because a democratic society that expects or relies on its scientists to bear the responsibility for making what's known to science known to citizens necessarily doesn't get the central tenets of the science of science communication: (1)  that there is distinction between "doing" and "communicating" valid science; and (2)  that the latter demands its own science, its own professional training, and its own reliable implementing institutions and practices. Not getting (1) & (2) is the source of the persistent public conflict on climate science & risks squandering in general what is arguably our society's greatest asset -- the knowledge that science confers on how to secure collective health, safety, and prosperity.

But the one thing I am more confident is correct than this argument is that the surest means for remedying the deficit in our society's science-communication intelligence is through the process of conjecture and refutation that is the signature of science. Let's articulate as many experience-informed hypotheses as we can; and let's test them by doing and modeling them within our universities and within all the other settings in which science and science-informed policymaking are practiced.

So consider this inspired account of what's to be done. If it weren't an "n of 1," I myself would accept that it in itself refutes my claim that it's a mistake to think that we shouldn't conflate excellence in doing and communicating science.

from Paul Shepson:

Dan - you said in your revised post, that "Their job is not to communicate their science to non-experts or members of the public." This did strike me as a weird thing to say. When I am doing science, I try to do it in a scientifically defensible way. When I am communicating to the public about science, I try to do it in a way in which they learn something, and hopefully laugh a few times. But what my job is, that's for me and my employer to negotiate, and hopefully, for me to be creative about. My job is to feel good about what I do, and at the same time hopefully help people, and get to eat. But, as I said in my email to you, it is indeed our responsibility to do exactly this (communicate to members of the public), as I said, especially when the scientific results have large social, ethical, economic, human and ecosystem health impacts. And, it is the case that Federal agencies, e.g. NSF, that fund the scientific community REQUIRE that we communicate our science outside of the scientific community.

For me, doing this is an integral part of who I am as a scientist. I have learned, from a variety of personal experiences, like marriage counseling, and communicating about climate change to Rotarians, etc., that it is very important to "get into the heads of" the members of the audience. But, until your presentation at the NACP meeting, I didn't fully have the jargon about, and the better informed ideas about, the importance and impact of cultural cognition. This has helped me a great deal, and I am sure it will in future presentations; I am already implementing changes (in my head) as a result of your blogs and your presentation. But I don't typically expect scientists to communicate, as you have said, the "validity of valid science". Scientists more often are communicating about the process of science, which can be far more interesting and entertaining, than trying to hammer home the idea that some set of climate science-related conclusions are valid. For me, a quantitative scientist, to discuss the "validity" of my work requires the use of error analysis, and thus, for a general audience, might require them to use stimulants of some sort. People sometimes use the word valid or validate when referring to one of the most important tools of science, the model. But, models are almost never valid, they are a representation and most often simply a test of our understanding of a natural system, such as the Earth. It is hard for me to imagine an Earth System model as ever being valid. But what is fun to tell people about is the process of finding things out, to use a Feynman-ian-like term, since you have referred to Feynman in your blog. People will listen to stories about how hypotheses are developed, e.g. about warming in the Arctic, and then about how you went there to test it, and observed a similar warming, and a similar loss of sea ice, but how that loss of sea ice is occurring faster than the models predicted, and then how that comparison led you to think harder about what is wrong with a model. Models aren't ever valid, they are wrong, and it is learning about the wrongness that leads to scientific progress. The finding things out, and the wrongness is the excitement of science. People love to hear stories about what an Inupiat Eskimo taught you about ice that you never learned from other scientists, and how that helped you rethink your model. Science is a process, not a bunch of end results that are either valid, or not. Ah, but enough ranting.

Regarding making my University bear its share of the burden, I can't really make my University do much of anything. I have tried! But, I can motivate myself to try to inspire young people about the process of science, and to tweak peoples minds to think about things in a different way, and hopefully, in a positive, constructive way. So, when I asked you about taking a renewable energy engineer with me to the Rotary Club, I was suggesting that it might be effective for people who value individualism and a hierarchical world to see the unprecedented investment opportunities in renewable energy, which everyone on the planet will likely eventually need. Its a darn big market! And that pursuit of such investment opportunities might "symbolize human resourcefulness", in a way that is fully consistent with the values of the cultural group with which they identify. Shouldn't we try to take Warren Buffet with us to the Rotary Club? I think the climate science community should be communicating that everyone can win, and that includes the cultural groups with which they strongly identify, in the pursuit of the solutions to climate change.

While you might not think that I am, I will take the liberty of saying thank you for helping me to think more clearly.

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (5)

Paul,
Nicely stated.
As to models, I had never thought to vocalize that I think of them as always wrong. They are my way to make a tasty reduction of thousands of experiments so that I don't have to remember each of the experiments.
When the reduction has an odd taste, when it no longer fits the results, I make a different sauce/model. Some models survive longer than others.
The most seductive models are those that are 'elegant' but are actually wrong. I have to discard them as well. I am learning to discard the elegant ones a bit faster than I used to. I am helped in this learning process because, in my field of molecular biology, the elegant models have almost always thrown out most of the experimental results in the attempt to create an elegant model.
Thanks for your thoughts,
Eric

February 19, 2013 | Unregistered CommenterEric Fairfield

"People sometimes use the word valid or validate when referring to one of the most important tools of science, the model. But, models are almost never valid, they are a representation and most often simply a test of our understanding of a natural system, such as the Earth. It is hard for me to imagine an Earth System model as ever being valid."

When people talk about "validating" the model, they don't mean that the model has to be perfect.

It's a technical term from quality control. What they're usually talking about is showing that the model can reliably predict outcomes to a stated accuracy, within a stated range of circumstances. You need to know what assumptions the model relies on, what approximations are made, for what ranges of parameters it's been tested, and so on so that you will not try to apply the model outside its domain of known validity, And you have to know the accuracy of the result you get, to be able to tell whether it meets your requirement. A grossly inaccurate model may still be sufficient for your purposes, so long as you know about the inaccuracy.

It was George Box who said "All models are wrong, but some are useful." The art is in knowing which ones.

The same goes for scientific theories generally, which is another word for explanatory models (i.e. a model that allows not only numerical prediction of the outcome but causal explanations and generalisations). Just as with a numerical model, you need to know its limitations and boundaries to be able to use it safely. That's something that's often omitted from science education. People commonly assume that certain well-established scientific conclusions are perfect models of reality. But if you don't know where the boundaries are, you can never be sure you haven't crossed them.

Incidentally, some people argue that in a technical sense this really ought to be called "verification and validation". Verification is checking that a product meets its specification. Validation is checking that the specification meets the user's needs. Thus, verification is the process of measuring the accuracy and documenting the limitations of the model. Validation is looking at those documented limits and deciding whether the model is suitable for what you want to use it for. A validated model is then one that has been shown sufficiently accurate for the application.

But only quality auditors talk like that, and I think most normal people in science would understand a "validated model" more flexibly, as either verified or validated. If the distinction doesn't seem useful to you, just be thankful you don't have to deal with quality auditors... :-)

February 19, 2013 | Unregistered CommenterNiV

Prof. Kahan,

you recently made the fine observation that,

Everyone deserves an education and opportunity to attain … ‘ordinary science intelligence,’ but I think people can get by fine w/o it — it actually isn't a necessary faculty for recognition of what's known to science. But the likelihood that those people, and even a lot of people who are great at crticial thinking, will be misled about what's known if science-intelligence professionals don't think scientifically is pretty high.

Notwithstanding the arguments against expecting scientists themselves to teach us (the public) about science, one point comes to mind in its favor:

When we “get our science” from working scientists, we can be sure we’re getting it from scientifically literate people. We can be sure our teachers are capable of thinking scientifically.

Can such a guarantee exist otherwise?

Let’s look at the kind of “science” we’re getting from today’s “science communication” specialists.

The following passages are due to well-known public thinkers on, and propagators/propagandists of, “science”:

There are two aspects to scientific consensus. Most importantly, you need a consensus of evidence – many different measurements pointing to a single, consistent conclusion. As the evidence piles up, you inevitably end up with near-unanimous agreement among actively researching scientists: a consensus of scientists.

This is not only a turgid word-salad (“consensus of evidence”) but a fairy tale; a lie; a myth. You do not “need a [consilience] of evidence—many different measurements pointing to a single, consistent conclusion” in order for scientific consensus to occur. After all, scientific consensus is not magic; it is not a sacred rite; it undergoes no more quality control than a sartorial consensus, an architectural consensus or a sports-journalism consensus. We’re talking about a majority of primates of the same profession and level of ignorance sharing a particular opinion.

Science is inherently skeptical, and peer-review is the instrument by which scientific skepticism is pursued.

No it isn’t. “The instrument by which scientific skepticism is pursued,” to borrow a horribly clunky clause, is something called the scientific method. The peer-review system isn’t even part of it—it was tacked on the end as a response to the post-War glut in government-funded research.

The person responsible for this claim evidently doesn’t know the first thing about how science works. Well, OK, they know the first thing—that science is inherently skeptical—but not the second, third or subsequent things.

Finally:

In my experience, climate skeptics are nothing if not confident in their ability to challenge the science of climate change--and even to competently recalculate (and scientifically and mathematically refute) various published results. It’s funny how this high-level intellectual firepower is always used in service of debunking—rather than affirming or improving—mainstream science.

I hardly need to explain that the author of the above is radically ignorant of the point of science and has probably never given a thought in his/her life to how science differs from, say, religion or any other system of bare ideology.

To repeat, these 3 passages are from professionals paid to communicate / teach science. (By a colorful coincidence, all 3 authors are best-known for their adherence to a particular niche hypothesis in science.)

This is not good news for citizens.

APPENDIX

The sources for the above quotations are:

http://theconversation.edu.au/there-is-no-such-thing-as-climate-change-denial-11763

http://theconversation.edu.au/climate-change-denial-and-the-abuse-of-peer-review-1552

http://www.desmogblog.com/little-knowledge-why-biggest-problem-climate-skeptics-may-be-their-confidence

February 26, 2013 | Unregistered CommenterBrad Keyes

@Brad:
1. For sure scientists should *teach* science!
2. Agree that many professional science communicators are not good. Many of them don't have a scientific conception of science communication.
3. I think *everyone* can *think* like a scientist. Goal of education should be impart scientific habits of mind in everyone. Thinking like a scientist just "is thinking."

February 26, 2013 | Registered CommenterDan Kahan

Prof. Kahan,

thanks for your reply! You recognize that "Many of them don't have a scientific conception of science communication," but I was drawing attention to an even more disturbing problem (perhaps because I misinterpreted your initial quote): many of them don't think scientifically, period.

At least one of the paid science communicators I quoted above not only doesn’t think scientifically, he doesn’t even know how. He has an English degree and no scientific discipline at all, IIRC (which hasn’t stopped him selling patronising polemics about how half of America is scientifically handicapped by its bad neural machinery).

February 27, 2013 | Unregistered CommenterBrad Keyes

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>