follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Aren't you curious to see the published version of "Science Curiosity and Political Information Processing"?! | Main | Presentation jeopardy: here's the answer; what's the question? »
Tuesday
Jan242017

WSMD? JA! Political outlooks & Ordinary Science Intelligence

This is approximately the 2,92nd episode in the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

Tom De Herdt formed an interesting conjecture, which he posed as follows:

[I]t may well be possible that the increased polarisation (visible in the left-hand graph [from Science Curiosity & Political Inforamtion Processing]) is a result not so much of OSI [Ordinary Science Intelligence], but rather of a selection effect: as OSI increases, many people are convinced of higher risk and hence “switch” camp towards the liberal/democrat voters. Only the “stubborn” republicans remain and, by implication, the perceived risk by highly scientifically intelligent republicans decreases.

In other words: in the “high” OSI group, there would be much more democrats than republicans compared to the “low” OSI group?

It must be easy for you to prove this hypothesis wrong (or to confirm it) but i don’t seem to find these data very explicitly mentioned in your paper(s).

My response:

That's an interesting surmise; for sure it is worth considering whether this kind of endogeneity could be creeping in when one assess how ideological or cultural values influence risk perception.

But here I'd say that the evidence we have on hand makes it unlikely that the results you are curious (science curious, in fact) about reflect flight from the Republican to the Democratic party, thereby causing the Ordinary Science Intelligence (OSI) to become top heavy with left-leaning Americans.

Maybe first I should explain what you obviously know, which is why that possibility wouldn't show up in the figure you are looking at. The two graphs are comparing concern about climate change among left- and right-leaning subjects conditional on their having same OSI scores. So even if there were a disparity in the proportion of right-leaning who score high on OSI, the figures would look exactly the same.

But we can easily look & see if there is such a disparity lurking in the data. Here's what we'd see on relationship of OSI to partisanship:

As reflected in these probability density distributions, those on left & those on right don't differ to any meaningful degree in their OSI scores. The correlation between OSI and scores on the "Left_right" political disposition scale (which is formed by aggregating resposes to liberal-conservative & party-identification items) is - 0.06-- it's hard to get much closer to zero than that! (Indeed,people can look pretty foolishif they think a "statistically significant" difference that paltry matters).

Or at least that's how it looks to me.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (18)

"As reflected in these probability density distributions, those on left & those on right don't differ to any meaningful degree in their OSI scores."

The question isn't whether left and right differ in their average OSI scores, but whether the subset at the top end of the OSI score differ meaningfully from those in the middle. It's noticeable that the blue line is above the red one at the top end, and quite a way below it in the middle. To know whether this difference is "meaningful" would require error bars on that density graph.

Incidentally, why does the graph cut off at the 99th percentile? And what do the percentiles 1-16-50-84-99 refer to? The sum of the two lines? Or each individually? Why those numbers? Are they based on Gaussian 1-sigma and 3-sigma percentiles somehow? How are the densities normalised? And does it look to you like the area to the right of the 99th percentile (if the curve is continued smoothly) would be equal to the area to the left of the 1st percentile? Weird!

"(Indeed,people can look pretty foolish if they think a "statistically significant" difference that paltry matters)"

That's quite ironic, really. The correlation between the 1700-1729 step of Michael Mann's famous 'Hockeystick' graph of global temperatures, and the actual measured temperature, was 0.0055. That's an r-squared of 0.00003! [ref Ammann & Wahl 2006]

I think climate scientists described that particular correlation as "robust", and when questioned about the correlation (at the NAS panel in 2006) said "We didn’t calculate it. That would be silly and incorrect reasoning" (although it's obvious from the original paper that they had, and had even quoted it for a later step; they just didn't publish the adverse results). And the rest of the scientific community seemingly concurred. Looking "pretty foolish"? Sometimes they do, sometimes they don't.

It's not relevant to your point, but I thought it was amusing. In climate science, apparently, it's thinking that teeny tiny correlations mean the result isn't meaningful that's actually "foolish"!

January 24, 2017 | Unregistered CommenterNiV

@NiV

Maximum and minimum scores on the test will constrain the shape of the tails. I could just take the mean & SD and draw the curves -- using out of sample scores to make tails end less abruptly -- but this is more realistic & intersting. The upper tail is "fatter" than the lower end one b/c there are more people crashing into the top of the scale than bottom.

But all in all, it's a reasonably normal looking curve. The percentils, as I'm sure you know, reflect median score (50th) & -1 +1 SD b/c those are convenient reference points.

I think w sample w/ 2267, you can learn a lot from raw data. Not much marginal value in modeling the distribution w/ error bars etc, just lots of noninformational clutter.

One thing that might add more information, though, would be to try to estimate the respective likelihoods of the two groups at or at & above a threshold, like 90th percentile etc. In that case, I'd use some bootstrapping strategy to account for error-- in other words, just draw randomly 2267 respondents w/ replacement from the respective distributions some 1000 times & then create another PDD for the two at the critical value or range; values at those PDD's 2.5 & 97.5 percentiles would give you precision on means equivalent to analytically derived 0.95 CIs. I could do that for you, if you think it would be interesting.

January 24, 2017 | Registered CommenterDan Kahan

Is there any way to look at what the respondents might mean by Human Health Safety or Prosperity? Maximizing hypothetical opportunities for Individuals (I can be Trump too someday).Or greatest long term common social and environmental good? (Obviously my version of this question has just a bit of push poll aspect to it. )

My bias being that industrial Titans have always leaned towards profit now worry about environmental and social consequences later. While coaxing the Public along with the promise of jobs or the threat of job loss.

Which IMHO is what actual climate change denial or of falsely and dilberatly fostering denialism others is all about. Way less important to many than jobs now. Some idea that they as individuals will come out ahead that way or that "Scientific Progress" will come to the res us in the nick of time.

January 25, 2017 | Unregistered CommenterGaythia. Weis

!@Gaythia --

more to your comment, I realize, but check out this post on the risk peception measure

January 25, 2017 | Registered CommenterDan Kahan

Off topic -

W/r/t the interaction effect, or interdependency, between strong ideological orientation and displayed reasoning across context, I offer the very interesting reaction of Sarah Palin to Trump's government interference in the economy/"crony capitalism" in the Carrier situation.

What does it mean that someone who is so often considered to be low on the critical thinking end of the spectrum shows a more consistent orientation w/r/t espoused values, in comparison to many "conservatives" who might be more scientifically literate or score higher on cognitive reasoning tasks?

As Trumps conflation of the "invisible hand" theory of the economy with his sticking his own hands (small as they might be) into economic policy to direct it in his desired direction, this question seems to me to become ever that much more vital.

January 25, 2017 | Unregistered CommenterJoshua

"The upper tail is "fatter" than the lower end one b/c there are more people crashing into the top of the scale than bottom."

I think you misunderstood my point. The meaning of "1st percentile" and "99th percentile" is that 1% or 99% of the data is less than that value, respectively. So there should be 1% of the shaded area to the left of the first point, and 1% of the shaded area to the right of the 99th percentile point. The two shaded areas should be equal, by definition.

"The percentils, as I'm sure you know, reflect median score (50th) & -1 +1 SD b/c those are convenient reference points."

Then they should be labelled as +/-1 SD, not as percentiles, because they're not percentiles.

The relationship between percentiles and SD multiples depends on the distribution. Different distributions give different values, and you can't assume any particular relationship without knowing what the distribution is (which in this case I assume you don't know).

I did wonder if you was assuming the distribution was Gaussian and plotting multiples of the SD, since the values are 2.27%, 15.9%, 50%, 84.1%, and 97.7%. The middle three values match. But the outer two values don't - the 1% and 99% points ought to be further out (at +/-2.33 SD), and the distribution is fairly obviously not Gaussian anyway, so I figured you must be doing something else.

"I could do that for you, if you think it would be interesting."

Don't bother. It's not important enough to do any significant work.

If we consider the top 25% (my guestimate at the percentage of the total area to the right of the point where the blue line passes above the red), that's a sample of n=567. Assuming a binomial distribution with a fixed common 50% probability of being red/blue, you'd expect about 283 +/- 24 on each side. That's about 8.5% variation. Again, by eye (hard to tell when the data's been 'smoothed' for graphing), I'd think the difference from the midpoint was about that size. It's probably not significant, but it's a bit marginal.

However, if we also note that the spread on the blue curve is wider, that casts my fixed common 50% assumption into doubt, and makes it quite a bit less likely to be significant.

So my quick, back-of-an-envelope conclusion would be that there's insufficient evidence of a significant difference, indicating any 'conversion-to-the left' at high OSI, but it's not quite "obvious" that this is so. On the other hand, a difference up to 20% might be feasible. Not sure if that could explain the difference in risk judgements, but I'd have pretty low confidence that it's been firmly disproven.

Good job I had such strong priors that the hypothesis is untrue, eh? :-)

January 25, 2017 | Unregistered CommenterNiV

"What does it mean that someone who is so often considered to be low on the critical thinking end of the spectrum shows a more consistent orientation w/r/t espoused values, in comparison to many "conservatives" who might be more scientifically literate or score higher on cognitive reasoning tasks?"

It means the belief that Sarah Palin is on the 'low end of the spectrum' is only commonly held by people on the 'left end of the spectrum'. Motivated reasoning.

"As Trumps conflation of the "invisible hand" theory of the economy with his sticking his own hands (small as they might be) into economic policy to direct it in his desired direction"

He's just quoting a soundbite he heard somewhere without really understanding what it's about. He wants small government because it means low taxes. It's good for the economy (for 'invisible hand' reasons), yes, but it's also good for business, which is what Trump understands and is interested in.

It's always difficult to tell with Trump, but so far as I know, he's strongly Protectionist in his economics, and so very unlikely to espouse much economics of the 'invisible hand' sort, given that it's diametrically opposed to Protectionism and regards it as stupid and destructive.

Of course, as Adam Smith himself pointed out "People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices." Such Protectionism of this classic sort is promoted by big business, as a way of increasing their own profits at the expense of the nation by excluding foreign competition. It's very likely he picked up the philosophy from this environment, and possible that he's aware of the economics but representing his own and his community's business interests.

He also promotes that other type of Protectionism, where the same technique is applied to the labour market. Workers also seek to raise their own wages by excluding cheaper competition, gaining a monopoly over its supply. This is how labour unions, guilds, and professional bodies work - you can only supply work if you're a member. And it works when applied to Nationalism, where you can only supply labour if a member of the selected nation. It's essentially a union closed shop on a grand scale, and about as left-wing as it gets.

And Adam Smith's invisible hand argument explains why it's equally disastrous for the economy. The 'invisible hand' uses competition to guide the economy to the point where supply and demand are coordinated optimally, with no governmental control or legal restriction. Excluding competition disables the hand, enables manufacturers and workers to raise prices, making everything more expensive, and overall making society poorer. People always stop thinking at the point where they figure they can charge more by excluding the cheaper competition, without getting as far as realising that everyone else can do the same. You lose more from everyone else inflating their prices than you gain from inflating your own.

Trump's being consistent in pursuing his priorities, but is wrong. Palin is consistently supporting a different set of priorities, and would have been a much better President. But she was a woman (and a Republican), and so easy to categorise as 'on the low end of the critical thinking spectrum'.

January 25, 2017 | Unregistered CommenterNiV

On attempting to define Human Health, Safety and Prosperity, or on the impossibility of separating out science based policies as separate issues, such as Climate Change Denialism from the effects of EPA environmental or OSHA regulation,

How would you or could you parse those three things as separate items?

Earlier today I was in Wheeling, West Virginia, the state with the the highest level of disability payments in the nation. http://247wallst.com/special-report/2016/07/15/states-with-the-most-americans-on-disability-2/11/
1. West Virginia
> Pct. receiving disability benefits: 8.9%
> Total monthly benefits received: $122.4 million
> Labor force participation rate: 52.7% (the lowest)
> May unemployment rate: 6.2% (tied-4th highest)

It also ranks as least educated:
1. West Virginia
> Bachelor’s degree or higher: 18.9%
> Median household income: $41,253 (3rd lowest)
> Pct. below poverty level: 18.5% (10th highest)

West Virginia's economy has been, of course, highly dependent on coal, high sulfur coal. This is coal was an extraction industry that fueled much greater wealth elsewhere, in the steel mills of Pittsburgh or power plants throughout much of the Northeast. Or computer servers.

Covering a lot of bases: http://www.theintelligencer.net/news/top-headlines/2016/12/murray-helps-trump-claim-win-closes-powhatan-coal-mine/

"“It is a repudiation of the administration of Barack Obama and his appointed bureaucrats, and the Silicon Valley pay-to-play billionaires who are on the dole from America’s taxpayers, and the liberal elitists on Wall Street and in Hollywood, radical environmentalists, and some industries, such as the makers of costly windmills and solar panels,” Murray Energy Corp. Chairman, President and CEO Robert E. Murray said the day after Trump’s election. “It is a victory for the working men and women of America.”

But then:

"Although Murray hailed the Trump win in the spirit of keeping coal miners employed, the Powhatan No. 6 Mine is now closed, eliminating several hundred job opportunities.'


Having a job is,of course, directly connected to personal prosperity, although mining of coal certainly risky and takes a toll on health,and frequently also the environmental quality of one's surroundings. But providing for one's family is a considerable driving force.

At times, climate change denial is a code phrase for rejection of the concept that one has no future.

We haven't created a path forward through the energy and information technology changes we face.

January 25, 2017 | Unregistered CommenterGaythia Weis

@NiV-- The percentiles are based on the IRT model of the disposition. The PDD plots reflect the scores of the subjects in the study.
So you don't want to see how close your envelope calculations are to the ones I proposed to extract from the data?

January 25, 2017 | Registered CommenterDan Kahan

So overall there is only a slight difference in the opinions between liberal democrats and conservative republicans in their perception of risk involved regarding global warming correct? With the liberal- democrats being slightly* more alarmed and republicans being slightly less...

It seems like political partisanship is not a reliable indicator of the perceived dangers of global warming/climate change. Based on the graphs and information in your blog, one cannot rely on them to make a judgement about either side.

January 25, 2017 | Unregistered CommenterGabriela Cortes Legra

Also off-topic.


This is great. Not when I ever saw such a perfect illustration of the power of motivated reasoning.


This is reminiscent of 'They Saw A Protest" ....except on steroids;


--snip--

For the other half, we asked a very simple question with one clearly correct answer: “Which photo has more people?” Some of these people probably understood that the image on the left was from Trump’s inauguration and that the image on the right was from Obama’s, but admitting that there were more people in the image on the right would mean they were acknowledging that more people attended Obama’s inauguration.

Would some people be willing to make a clearly false statement when looking directly at photographic evidence — simply to support the Trump administration’s claims?

Yes.

[...]

For the question about which image went with which inauguration, 41 percent of Trump supporters gave the wrong answer; that’s significantly more than the wrong answers given by 8 percent of Clinton voters and 21 percent of those who did not vote.

But what’s even more noteworthy is that 15 percent of people who voted for Trump told us that more people were in the image on the left — the photo from Trump’s inauguration — than the picture on the right. We got that answer from only 2 percent of Clinton voters and 3 percent of nonvoters.

Even when the photographic evidence was directly in front of them and the question was straightforward, one in seven Trump supporters gave the clearly false answer.


--snip--


https://www.washingtonpost.com/news/monkey-cage/wp/2017/01/25/we-asked-people-which-inauguration-crowd-was-bigger-heres-what-they-said/?utm_term=.aeb8c4a6e1bd

January 26, 2017 | Unregistered CommenterJoshua

@Gabriela--

take a look at this post & tell me if you still think partisan differences here are small. (You can also play the fun data jeopardy game.)

January 26, 2017 | Registered CommenterDan Kahan

I know that Trump supporters are not more likely to be affected by cultural cognition than other groups delineated by political ideology, but jeez...

https://mobile.twitter.com/foodnpolitics/status/824440392271605760/photo/1

I mean are they? 38% can't REALLY believe that MOST of the woman marchers were paid by Soros, right% gotta think these poll results are more about who they are than about what they REALLY believe.

https://mobile.twitter.com/ppppolls?ref_src=twsrc%5Etfw

January 26, 2017 | Unregistered CommenterJoshua

" The percentiles are based on the IRT model of the disposition."

Ah! You didn't say!

So your model predicts 1% of the results above the maximum possible?

"So you don't want to see how close your envelope calculations are to the ones I proposed to extract from the data?"

If you was going to do it anyway, I'd be interested. What I meant was that I didn't want to put you to any trouble over what was no more than idle curiosity on my part.

--

"Would some people be willing to make a clearly false statement when looking directly at photographic evidence..."

Yes. That's the well-known Asch social conformity effect.
https://en.wikipedia.org/wiki/Asch_conformity_experiments

Although as I think I've mentioned before, people asked dishonestly political questions will tend to give equally dishonest political answers. The question they will ask themselves is "How will this response be used in tomorrow's newspapers by this clearly biased surveyor?" The headline will either be "Not even Trump's own supporters believe him!" or "Trump's supporters equally deluded!" The latter is less politically damaging.

It's a classic "have you stopped beating your wife yet?" question asked solely to generate a partisan political attack and cheer up the Democrat base, and nobody with any sense would take it seriously as any kind of reflection of people's real opinions. I think American sports have a phenomenon called "trash talk"? It's like a political version of that.

January 26, 2017 | Unregistered CommenterNiV

Dan, Had a look at the paper. Congrats on the publication, of course.

But oh dear!

Farmers, for example, have been observed to use information on climate change to form identity-congruent beliefs when they are behaving as citizens but to form truth-convergent ones when they are engaging in the task of farming, where they have an end—succeeding as farmers—that can be satisfied only with that form of information processing

You mean farmers showed an entirely consistent truth-seeking belief in *natural* climate change, which resulted in them simultaneously rejecting claims that climate change was anthropogenic while accepting that natural climate change would require changes to farming practices, as it always has.Tch! Tch!

The result is that these citizens, unlike their less curious counterparts, react more open mindedly and respond more uniformly across the political spectrum to the best available evidence.

Well, it probably means they a least tend to converge on what's presented in science documentaries. What evidence do you have that they are able to discriminate which bits are " the best available evidence"? That's more like OSI, isn't it?

As I noted before, the way to test it is to look at a subject where the science documentaries get it wrong. (For example, it's recently been decided that saturated fats are not as bad for you as decades of health advice have claimed.) Are high-SCS subjects more able to spot the lack of sufficient evidence for the false position *prior* to it becoming orthodox enough to appear in science documentaries? Or do they simply uncritically believe whatever the most recent science documentaries currently say?

If you show them a science documentary with logical flaws in the arguments, are they more able than the average citizen to correctly identify them? Did they spontaneously notice them? Can they be more easily be swayed to false positions by a man in a white lab coat?

I note that both sides of the political aisle converged for high SCS on the position that fracking is more dangerous. Is that what you think "the best available evidence" says? What's the evidence?

But as you say, this research remains at a formative stage. It's a refreshing change from the usual approach.

January 27, 2017 | Unregistered CommenterNiV

@NiV-- on the theoretical & observed OSI levels: the bunching up of the observations at the maximum of scale (2% got all 18 questions; and another 4% got 17 of 18) suggests that OSI_2.0 isn't discriminating as well as it could at t the high-scoring tail & I should try to add 1 or 2 more super hard questions to see if I can separate them. But the degree of discrimination that OSI_2.0 has in this regard is pretty good in relation to other tests that are common for trying to estimate some sort of proficiency related to science comprehension. Take a look at OSI_2.0 paper to see how much more of the range of ability OSI_2.0 can measure with precision than other related scales do on their own. But even OSI_2.0 has fairly high measurement error at 99th percentile -- about 40% of the variation at that point is measurement error rather than true variance in ability

January 28, 2017 | Registered CommenterDan Kahan

@NiV-- I could re-argue the Ky Farmer but if you accept the Pakistani Dr evinces "cognitive dualism," then I'll rest my (provisional as always) case there

January 28, 2017 | Registered CommenterDan Kahan

"I could re-argue the Ky Farmer but if you accept the Pakistani Dr evinces "cognitive dualism," then I'll rest my (provisional as always) case there"

I wasn't convinced the Pakistani Dr did, either, but that was largely because I didn't manage to track down any precise statement of what they did or didn't say. Apparently, the researchers *did* ask subjects for explanations, but only a few excerpts the researchers thought particularly relevant are quoted. There's no full transcript I can see.

I found the survey results here: http://link.springer.com/article/10.1186/1936-6434-6-2

But all it says on the subject is this:

Indeed, most participants asserted that the theory of evolution played a role in modern medicine (Figure 6). Unsurprisingly, this included every participant who accepted evolution. Significantly, however, two participants who rejected evolution as well as four participants who had other views also considered the theory to be relevant to medicine. Several participants provided examples, including two participants who considered the theory of evolution to be essential to the study of stem cells.

So there were two survey participants who rejected evolution but who also considered the theory "relevant to medicine". We don't know if they were among those who offered "examples" or not.

That's not quite the same as "members of religious groups that subscribe to creationist strictures but “believe” in it when discharging their duties as science-trained professionals". Thinking it "relevant" doesn't mean believing/using it.

There's a lot of scope for subtle distinctions here. People might think it relevant in a historical or contextual sense, or to understand what other researchers are talking about. (For example, I believe that understanding the conventional "back-radiation" explanation of the greenhouse effect is essential to studying climate science, even though I think it's wrong. You can't argue against a theory unless you understand it.) People might think of it as a useful model for certain purposes, but one that when examined sufficiently closely isn't true. (For example, physicists know that classical Newtonian physics is "wrong", but will routinely use it as if it was true for those areas where relativity and quantum mechanics have negligible effects.)

There's nothing remotely contradictory about the above combinations of views, and yet they seem to fit the descriptions given in E&H. On the other hand, I can't say for certain that the Pakistani doctors are *not* being inconsistent (nor would I find it particularly surprising in religious people - lots of their beliefs are mutually contradictory), and there may be more information elsewhere you have access to I've not seen, so I'm not about to say you're definitely wrong about it, either. In contrast, in the Kentucky Farmer case the numbers were sufficiently detailed to figure out what was going on, and I'm reasonably confident that their views were not inconsistent.

A vague third-hand report of two people responding to a survey at a conference that could be interpreted that way doesn't seem much to hang a theory of human reasoning on. But I don't find the Pakistani Dr to be anywhere near as irritating as the Kentucky Farmer, so you can have it if you like. Sure, go ahead and rest your case.

January 28, 2017 | Unregistered CommenterNiV

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>