So I said in the last post that the "57% vs. 56%" factoid can't be read to indicate that one side has better comprehension of science.
It was (quite reasonably) pointed out to me in a comment that our Nature Climate Change study reported statistically significant correlations between science literacy and numeracy (and the composite science comprehension scale that aggregated them), on the one hand, and climate-change risk perception, on the other. What bearing does that have on the issue of who has greater science comprehension-- the "yeah, but whose is bigger?" question--in the climate change debate?
To start, there's no tension between the statistical computations here. If one looks at the correlation between the continuous science comprehension (science literacy plus numeracy) scale and climate change, it's negative (r = -0.09) and significant (t-statistic = -3.35). But the difference in the mean scores of the most concerned and least concerned halves of the sample is not significant. That can certainly happen when one splits the sample and treats two continuous measures as categorical ones (the opposite can happen, too).
But that's particularly likely to happen when the correlation between the continuous variables is tiny. That's so here.
People who are numerate are likely to suspect that r = -0.09 (r = -0.05 for science literacy by itself!) is small (which is how the paper characterizes this effect)--way too small to be responsible for the intensity of the climate change debate in our society.
But it's actually pretty bad craft to expect anyone to figure out whether an effect size is meaningful from bare correlation coefficients. Readers should be shown the effect in some way that conveys its practical importance.
The question under investigation in our study was what explains climate change conflict--differences in science comprehension or differences in cultural outlooks? One shouldn't really have to know statistics to see the answer in a figure like this:
I won't say anything more about the difference between statistical significance and practical significance because there's an excellent post that addresses it in the context of science comprehension and climate change at the Blackboard, in appreciation of & gratitude for which I am posting this:
But that leaves room to discuss another, and in my mind, more interesting point about statistical illiteracy that is reflected in obsessing over that effect: it’s meaningless in real-world terms.
The principal finding was that science comprehension interacts with cultural predispositions: individuals who are predisposed by their group values to skepticism become more skeptical, and those predisposed to concern become more concerned, as science comprehension increases.
So it is in fact misleading to characterize greater science comprehension as having any “main effect” toward either skepticism or acceptance. It has one or the other depending on other characteristics.
The only thing the “main effect” really conveys in these circumstances is the frequency of the two types (or maybe the intensity of the effect in one or the other, if it varies meaningfully) in one’s sample. Moreover, that’s true even if one’s “sample” is in fact a census: if the correlation when one looks at the population as a whole is negative, then there are simply more people out there predisposed (and/or more strongly predisposed) to fit the evidence to a "skeptical" conclusion; and if the correlation is positive — then the number predisposed (or predisposed more intensely) to see evidence as justifying concern is greater. End of story.
I talked to a researcher recently who tried to convince me that one should see a small positive correlation between science literacy & some other issue that had an interaction with ideology as meaning that when one “controls” for ideology, science literacy increases concern …” I kept trying to tell him to think about what it was he was actually modeling and how silly it is to describe the sample “mean” as the “effect controlling for” something that interacts with a characteristic that varies systematically in people in the real world.
I felt like I was arguing with the guy from spinal tap who kept saying, “mine goes to 11.”