follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Religiosity in the Liberal Republic of Science: a subversive disposition or just another manifestation of the pluralism that makes scientific knowledge possible? | Main | MAPKIA! episode 1: religiosity, science comprehension & climate change »

MAPKIA! "answer" episode 1: The interaction effect of religion & science comprehension on perceptions of climate change risk

Okay-- as promised: the "answer" to "MAPKIA!" episode 1!

As you'll recall, the "question" was:

What influence do religiosity and science comprehension have on (or relationship do they have with) climate change risk perceptions? 

Some players understandably found the query to be vague.  

It was meant to be in one sense.  I wanted to frame the question in a manner that didn't presuppose any position on the nature of the causal dynamics that could be generating any observed relationships; I wanted the players to have the freedom -- & to bear the explanatory burden -- to spell that out.  

Two players might have agreed, e.g., that religiosity would be negatively correlated with climate change risk perceptions but have disagreed on whether variance in the former was causing variance in the latter or instead whether the covariance of the two was being caused by some 3d influence (say, cultural outlooks or political ideology) operating on each independently.  

Or they might have agreed that the influence of religiosity or science comprehension on climate change risk perceptions was causal but disagreed about whether the effect was "direct" or  instead "mediated" or "moderated" & if  so what the mediator/moderator was. Etc.  

An essential part of the game (it says so in the rules!) is for players to venture a "cogent hypothesis," and I didn't want to rule anything out by suggesting any particular causal relationships had to be at work in whatever correlations a particular hypothesis might entail.

But I think reasonable players could have seen the vagueness as going to whether they were supposed to assume a particular causal relationship. That's no good!

So if I were to do it again, I would say (and when I do something like this again I will say) something like: 

If you had to predict someone's climate change risk perceptions, would your prediction be affected by information about that person's religiosity and science comprehension? If so, how and why?!

Okay, so now what's the "answer"?

I'm unsure!  But I can report that the two predictors interact. That is, one can't specify what the impact of either is without knowing the value of the other.

Actually, I was motivated to investigate this question myself because I had a vague hunch that would be true.  The reason is that I've now seen such an interaction in several other places.

One, which I've reported on previously, involves belief in evolution.  Science literacy (of the sort measured by the NSF indicators) predicts a higher probability that a person will say he or she "believes" in evolution (of the sort that operates without any "guidance" from God) only in people who are relatively nonreligious.

In relatively religious persons, the probability goes down a bit as science literacy increases (at least in part because the probability of believing in a "theistic" variant of evolution goes up).

This pattern is part of the reason that I think "belief in evolution" is an invalid measure of "science literacy" or "science comprehension" viewed as a disposition or aptitude as opposed to a simple score on a quiz (the latter is a bad way to investigate what "ordinary science intelligence" is & how to promote it).  Insofar as scoring high on other items in a valid science literacy or comprehension scale doesn't reliably predict saying one "believes in evolution," the "belief" item should be viewed as measuring something else--like some sense of identity that is generally indicated by low religiosity (indeed, saying one "believes" in evolution has no correlation with actually understanding natural selection, random mutation, and genetic variance-- the core element of the prevailing "modern synthesis" theory of evolution).

But if this particular indicator of one's sense of identity -- "belief in evolution" -- interacts with science comprehension, what about others?!

Actually, we know that there is such an interaction for various risk perceptions.  Perceptions of climate change risk increase with science comprehension for egalitarian communitarians, whose identities tend to be bound up with the perception that technology and commerce are dangerous, but decrease for hierarch individualists, whose identities tend to be bound up with the perception that technology and commerce are beneficial to human welfare.

Basically, when a position on some risk or other fact that admits of empirical investigation becomes a marker of identity, science comprehension becomes a kind of amplifier of the connection between that identity and the relevant position.  I've explained before why I view this as, in one sense, individually rational but, in another more fundamental one, collectively irrational.

So ... what about religiosity, science comprehension and climate change?

Here things get admittedly tricky. For sure, religiosity can be an indicator of some latent identity. Indeed, it seems to be an indicator of more than one kind -- and those varying sorts of identities might orient people in different directions with respect to some risk or comparable policy-relevant fact, not to mention all sorts of other things.

It's pretty clear, for example, that religion is bound up with certain forms of cultural identity for both whites and African Americans-- but also that the relationship between religiosity varies across with respect to race in a manner that makes religious African Americans differ politically from both religious conservatives and nonreligious liberals (or egalitarians).

Still, I happen to know that religion in general correlates with things like being conservative and hierarchical--indicators or forms of identity that tend to be bound up with climate change skepticism.  So it seemed possible to me that religion, understood as a fairly crude and noisy indicator of such an identity, might be correlated strongly enough with them to interact with science comprehension in exactly the same way with respect to climate change as do those forms of identity.

Or maybe not.  I wasn't all that confident & was curious -- both about the answer and what others' intuitions might be.

Actually, on what others' intuitions might be, I feel fairly confident that people who believe in climate change are likely to believe both that science comprehension correlates positively with climate change risk perceptions and that religiosity correlates negatively.  

They are wrong to believe the first point (just as people who are skpetical of climate change are wrong to believe that science literacy negatively correlates with perceived climate change risks).

But if they were right, they'd be making a good guess to think that religiosity is negatively correlated with climate change risk perceptions, because in fact (as is pretty well known) there is modest negative correlation between religiosity and various measures of science literacy & critical reasoning.

I mentioned this just a few weeks ago in my ill-fated "tea party science comprehension" post.  Measuring "religiosity" with a composite scale that aggregated  church attendance, frequency of prayer, and self-reported "importance of God" in the respondents' lives (α = 0.72) and "science comprehension" with a scale that aggregated eleven items (& "evolution" & "big bang," as the NSF itself recognizes makes sense if one is using their items as a latent-variable measure rather than as a "quiz" score) with an extended 10-item Cognitive Reflection Test battery ((α = 0.82), I found a modest negative correlation between the two, as one would expect based on previous research.

It doesn't follow, however, that science comprehension must be positively correlated with climate-change risk perceptions if religion is negatively correlated with it! The correlation of the former might be zero.  And it's also possible -- this is what I was curious about -- that the two interact, in which case it would be possible for science comprehension to be positively or negatively correlated with climate change risk perceptions depending on one's degree of religiosity.

But using the same N = 2300 highly diverse general population data (collected last summer) as I did for the "tea party" post, here is a "raw data" picture -- one in which the relationships are plotted with a lowess regression -- of the simple correlations between religiosity and science comprehension, respectively, and climate change risk perceptions (measured with the tried and true "industrial strength" measure).

Religiosity, it's pretty clear, is negatively correlated with climate change risk perceptions (r = -0.25, p < 0.01). But the relationship between climate change risk perceptions and science comprehension looks pretty flat; indeed, the correlation is -0.01, p = 0.76.

Hi. I'm a lowess plot of raw data for fig below! Click me!But now let's look at how the two interact!

Below is the a graphic representation of the results of a regression model (take a look at the "raw data," too, by all means!) that treats science literacy, religiosity, and their interaction as predictors of perceived climate change risk:

Yup, pretty clearly, the impact of science comprehension varies conditional on religiosity.

In the Figure, I've set the predictor at +1 standard deviation for "high religiosity" and -1 SD for "low." The model suggests that science comprehension has no meaningful impact on "low religiosity" sorts, who are pretty concerned about climate change risk. Among "high religiosity" sorts, science comprehension reduces concern.

Or in other words, being more religious predicts more concern about climate change only among those who are relatively low in science comprehension.

We should expect pretty much anything else we ask about climate change to show the same patterns-- assuming that what we ask is genuinely tapping into the general affective orientation that climate change risk perceptions comprise.

And we do see that if we examine the interaction of the effect of the two predictors on the probability that respondents in the study would say either that they agree there is "solid evidence" of "global warming" or that there is "no solid evidence" any warming in "recent decades." (There's a third option-- belief in "global warming" caused "mostly by natural patterns in the earth's enviroment"--that isn't that interesting unless one is trying to boost up the percent that one would like to report "believe" in "global warming" while obscuring how many of those respondents reject AGW--ususally about 50%).

click me... I'm like crack cocaine ...These figures also graphically convey the results of a regression model -- this time a multinomial logistic one -- that treats religiosity, science comprehension, and their interaction as predictors of the probability of selecting the indicated response (raw data, anyone?).


I have to say, those effects are bigger than I would have expected.

Again, I thought that there might be such an interaction, but only because religiosity might get a "big enough piece" of a latent identity-based predisposition (one founded, perhaps, on cultural outlooks) to be climate-change skeptical.

But I think there is more going on here.  And I'm really not quite sure what!

What's at stake, for me in my own reflections, is how to think about religiosity in modeling motivating dispositions in this and related settings.

I actually don't think "religiosity" in isolation is all that interesting.

Religiosity coheres with other characteristics in distinct patterns that indicate really interesting cultural styles. But the styles are diverse, and the contribution religion makes to them varies. So if one just grabs "religiosity" and treats it as a predictor, then one is getting some blurry hybrid indicator of discrete styles.

Anyone who thinks that "the thing to do" in this situation is construct a  multivariate model in which religiosity and various other characteristics are treated as independent variables, the joint effects of which are partialted out and the "unique" variance of which retained and measured in the predictor coefficients, is dead wrong.

If you agree with what I said a second ago about religion combining in distinct ways with discrete cultural styles, then using a multivariate regression model of this sort will only obscure what these styles are and how religion figures in them. The multivariate regression model measures the contribution of each predictor independently of its covariance with the other predictors.  But in the "heterogeneous indicator of diverse styles" view, religiosity is helping us to form a picture of who sees what and why only as a component of one or another particular combination of attributes.  The covariance of religion with these other indicators is the best measure of that style -- yet that covariance is exactly what is being partialed out of the parameter estimates in a multivariate regression model!

Stanley knows what he's talking about; people who think it makes sense to pile everything up on the right-hand side of a regression & see "what's significant," don't.Under these circumstances, the first-best modeling strategy is a latent-variable one that combines religion and other characteristics as indicators of the relevant styles. But that's hard to do becaues there really aren't any fully satisfactory (as far as I'm concerned) scaling or data-reduction techniques for mixed, nominal and ordinal plus continuous variables (factor analaysis doesn't work there; "cluster analysis" is not psychometrically valid; latent class analysis combines the variables but assigns each observation to one class, thereby ignoring heterogeneity in the strength of the relevant predispositions).

The next-best strategy is to form a decent latent-variable measure with indicators that do readily admit of scaling -- like the Likert items that are aggregated in the cultural worldview scales -- and resign oneself to ignoring the other indicators. If one could include them, the latent-variable measure would be even more discerning, but since what is being measured by the aggregate measure without them will correlate appropriately with the omitted indicators, the omitted ones are still "contributing" in an attenuated way, and their omission will not bias the measure. 

Okay.  But the point is that I'm looking only at religion alone here and seeing that it has a kind and degree of predictive power in conjunction with science comprehension that makes me think it is doing things that are too "big" and too interesting for me to keep thinking of it solely as an indicator that really has to be combined with others into appropriate packages before it can help one understand who sees what and why....

So what's going on?!

If people want to speculate on that, go ahead.  But story telling would be boring.  Offer an explanatory hypothesis -- a cogent one -- and specify a testing strategy for it & we'll play "WSMD? JA!"

As for the contest, there were multiple good entries (some made on G+, others sent by email, an extremely thoughtful but blatantly contest-rule-violating one on our neighbor site Anomalies & Outliers: Field Notes on a Human Tribe,  and still more hand-delivered by people who had driven to New Haven from Minnesota and Kentucky to be sure that their entries were received on time), but I'm going to declare @Ryan the winner of this episode of "MAPKIA!" 

Ryan figured that religiosity would be negatively (if weakly) correlated with climate-change concern via its status as an indicator of one or another risk-skeptical disposition that admits of even clearer specification.  He also offered that science comprehension would likely just result in "greater the confidence that the risk is high or the risk is low"-- the basic amplification effect I mentioned.

Ryan, your prize is in the mail!  But I do think you should now try to explain why the effect is bigger than I think your hypothesis would have led us to suspect, and tell us what we might observe to corroborate or refute your surmise.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (6)

Thanks for this excellent post.

I am curious how any of this maps (or doesn't) to the CCR Group-Grid schema . . . ?


November 24, 2013 | Unregistered CommenterIsabel Penraeth


Do you have a conjecture?

Of course, one question is how to specify the "this."

As mentioned, we know science comprehension of cultural conflagrations like one over climate.

But how *religion* figures is more complicated. There is an interesting race x religiosity interaction on climate. Also African Americans egalitarian communitarians are much much more religious than white ones...

November 24, 2013 | Unregistered Commenterdmk38


A conjecture? I have about 100 micro-conjectures.

To echo a little more concretely what you describe above about the, shall we say, fuzziness? that would result from trying to use religiosity as a measuring stick for anything meaningful . . . there is a limitation to using statistical survey aggregation to "form" groups compared to the use of observation and measurement of groups formed by affinity or history or geography . . .

For instance, even if the survey asked the respondent to designate a major American religious denomination like, say, Lutherans, within that you have ELCA Lutherans who ordain women and Wisconsin-synod Lutherans where I observed a lengthy discussion over whether or not women could be ushers since it put them in authority over men. By time you are done parsing that, you would probably have a sample size that is unsatisfactory, but nonetheless I would expect these two sorts of Lutherans to land in ever-so-slightly different places in the group-grid schema.

Interesting dilemma.



November 25, 2013 | Unregistered CommenterIsabel Penraeth


I agree that there will be some interesting dynamics that are too fine grained to be measured w/ survey & related measures. E.g., I'm pretty sure no one will be able to figure out what sorts of disposition motivates "vaccine hesitancy," since the number of persons in the general population who have the disposition is so small that any thing one can measure that "indicates" that disposition in them *won't* in orders of magnitude more people. So we will need another tool -- although the tool that revealed that *was* survey research, which corrects the misimpression that "vaccine hesitancy" is actually a meaningful source of contestation in society at large. (I also believe in convergent validity: what one can show w/ methods of the sort I am using should admit of corroboration by ethnographic means-- in which case both the ethnographic data & the psychometric data become worthy of greater confidence.)

But I think you are giving up too quickly on the possibility that religiosity can be treated as a meaningful indicator of a latent disposition in survey & related forms of empirical research. In any case,, what's needed to figure this out is a scaling or classification technique that allows religion to be treated as an indicator based on its co-occurrence w/ combinations of other diverse indicators. Actually, cross-product interaction terms of the sort I used here enable that to a limited degree & can show that religiosity definitely *means* something about who a person is that varies depending on whether they are white or African-American, e.g.

November 25, 2013 | Registered CommenterDan Kahan

I'm not giving up at all. But you yourself admit in your *next* post that:

DMK: "But I think the state of the evidence is very unsatisfactory, in large part b/c the measures of both ordinary science intelligence and assent to the "authority" of science's way of knowing are so crude."

And I think measurements of religiosity are in the same boat. How much you pray or attend services tells little to nothing about how one allocates moral/decision-making authority. And I, for one, see that as a key piece of this puzzle.


November 26, 2013 | Unregistered CommenterIsabel Penraeth

Dan, what do you mean by cultural styles? As a qualitative researcher, that caught my eye! Thanks.

December 4, 2013 | Unregistered CommenterRobin Veldman

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>