More evidence of AOT's failure to counteract politically motivated reasoning 
Monday, October 30, 2017 at 2:39AM
Dan Kahan

Notice 2 things about this Figure:

1st, Stata 15 can now do transparencies!

2nd, this is even more evidence that “Actively open-minded thinking,” as commonly measured, furnishes no meaningful protection against politically motivated reasoning.

The results here are based on the same experimental design featured in the CCP Motivated Numeracy paper (Kahan, Peters et al. 2017). Subjects were asked what inference was supported by data presented in a 2x2 contingency table.  In one condition, the data were described as results of an experiment to test a new skin-rash cream.  In another, the data were described as results of an experiment to determine whether banning the carrying of concealed handguns in public increased or decreased crime.

In Motivated Numeracy, we found that individuals of opposing ideological orientations were substantially more likely to get the correct answer in the gun-control version if the data, properly interpreted, supported (or “affirmed”) the position associated with their ideology; when the data, properly interpreted did not not support their ideological group's position, individuals were more likely to select the wrong answer.

What’s more, the effect was stronger among the subjects of the highest degree of Numeracy, an aptitude to reason well with quantitative information.

The data here are pretty similar to those in Motivated Numeracy, except now it's “Actively Open-minded Thinking” (AOT) that is being shown to interact with ideology.  On the effectiveness of the new skin cream, individuals who score highest on a standard measure of AOT do better than those who score low, regardless of their political outlooks.

In the “gun control” condition, those who score highest on AOT do only slightly better on the version of the problem that presents ideologically congenial data. 

In the version that presents threatening or ideologically uncongenial evidence, however, those who score highest on AOT do no better than those who score the lowest.

This is not what you’d expect.

AOT is supposed to counteract ideologically motivated reasoning along with kindred forms of “my side bias” (e.g., Stanovich 2013; Baron  1995). Accordingly, in the "identity threatened" condition, one would expect those highest in AOT to do just as well as their high-scoring counterparts in the "identity affirmed" condition. One would expect, too, that the performance of those high in AOT would not show a level of degradation (-30%, +/- 14%) comparable to the degradation in performance shown by low scoring AOT subjects (-23%, +/-10%).

But it didn’t work this way here.

It also didn’t work that way in a study that Jon Corbin and I did last year, in which we showed that those highest in AOT, far from converging, were even more politically polarized on the danger posed by climate change (Kahan & Corbin 2016).

What to make of this?

Well, again, one possibility is that the version of AOT we are using simply is not valid.  I don’t buy that, really, because the measure has been validated in various settings (e.g., Baron et al. 2015).

The other possibility, which I think is more plausible, is that AOT--like Numeracy  (Kahan, Peters et al. 2017), Cognitive Reflection (Kahan 2013), and Ordinary Science Intelligence (Kahan 2016)—magnifies identity-protective reasoning where certain policy-relevant facts have become entangled with group-based identities (Kahan 2015).  Basically, where that’s the case, people use their critical reasoning proficiencies, of which AOT is clearly one, not to figure out the truth but rather to cement their status and relations with other group members (Stanovich & West 2007, 2008; Kahan & Stanovich 2016).

But I don’t want to be closed-minded toward other possibilities. 

So what do you think?

Refs

Baron, J. Myside bias in thinking about abortion. Thinking & Reasoning 1, 221-235 (1995).

Baron, J., Scott, S., Fincher, K. & Emlen Metz, S. Why does the Cognitive Reflection Test (sometimes) predict utilitarian moral judgment (and other things)? Journal of Applied Research in Memory and Cognition, 265-284 (2015).

Kahan, D. & Stanovich, K. Rationality and Belief in Human Evolution (2016), CCP/APPC Working paper available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2838668.

Kahan, D.M. & Corbin, J.C. A note on the perverse effects of actively open-minded thinking on climate-change polarization. Research & Politics 3 (2016).

Kahan, D.M. ‘Ordinary science intelligence’: a science-comprehension measure for study of risk and science communication, with notes on evolution and climate change. J Risk Res, 1-22 (2016).

Kahan, D.M. Climate-Science Communication and the Measurement Problem. Advances in Political Psychology 36, 1-43 (2015).

Kahan, D.M. Ideology, Motivated Reasoning, and Cognitive Reflection. Judgment and Decision Making 8, 407-424 (2013).

Kahan, D.M., Peters, E., Dawson, E.C. & Slovic, P. Motivated numeracy and enlightened self-government. Behavioural Public Policy 1, 54-86 (2017).

Stanovich, K.E. Why humans are (sometimes) less rational than other animals: Cognitive complexity and the axioms of rational choice. Thinking & Reasoning 19, 1-26 (2013).

Stanovich, K. & West, R. On the failure of intelligence to predict myside bias and one-sided bias. Thinking & Reasoning 14, 129-167 (2008).

Stanovich, K.E. & West, R.F. Natural myside bias is independent of cognitive ability. Thinking & Reasoning 13, 225-247 (2007).

 

Article originally appeared on cultural cognition project (http://www.culturalcognition.net/).
See website for complete article licensing information.