follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Cultural "fact polarization" trumps cultural "value" polarization -- a fragment | Main | "Inherent internal contradictions" don't cause bad institutions to collapse; they just suck ... "Rules of evidence are impossible," part 3 (another report for Law & Cognition seminar) »

Weekend update: Is critical reasoning domain independent or domain specific?... a fragment of an incomplete rumination

An adaptation of a piece of correspondence--one no longer, really, than this-- w/ a thoughtful person who proposed that people have "corrective mechanisms" for the kind of "likelihood ratio cascade" that I identified with "coherence based reasoning" and that I  asserted makes "rules of evidence" impossible:

What are these corrective mechanisms?

I ask not because I doubt they exist but because I suspect that they do -- & that their operation has evaded full understanding because of a mistaken assumption central to the contemporary study of cognition.

That assumption is that reasoning proficiencies--the capacity to recognize covariance, give proper effect to base rates, distinguish systematic relationships from chance co-occurrences, & perform like mental operations essential to making valid inferences--are more or less discrete, stand-alone "modules" within a person's cognitive repertoire.

If the modules are there, and are properly calibrated, a person will reliably summon them for any particular task that she happens to be doing that depends on that sort of mental operation.

Call this the "domain independent" conception (DI) of cognitive proficiency. DI is presupposed by standardized assessments like the Cognitive Reflection Test (Frederick 2005) and Numeracy (Peters et al. 2006), which purport to measure the specified latent reasoning capacities "in general," that is, abstracted from anything in particular one might use them for.

Another conception sees cognitive proficiency as intrinsically domain specific. On this view--call it the DS conception--it's not accurate to envision reasoning abilities of the sort I described as existing independently of the activities that people use them for (cf. Heatherington 2011).

Accordingly, a person who performs miserably in a context-free assessment of, say, the kind of logical-reasoning proficiency measured by an abstract version of a the Wason Selection Task-- one involving cards with vowels and numbers on either side -- might in fact always (or nearly always!) perform that sort of mental operation correctly in all the real-world contexts that she is used to encountering that require it. In fact, people do very well at the Wason Selection Task when it is styled as something more familiar--like detecting a norm violator (Gigenrenzer & Hug 1992).

In sum, reasoning proficiencies are not stand-alone modules but integral components of action-enabling mental routines that are reliably summoned to mind by a person's perception of the sorts of recurring problem situations those routines, including their embedded reasoning proficiencies, help her to negotiate.

DS is suspicious of standardized assessments, including the usual stylized word problems that are thought by decision scientists to evince one or another type of "cognitive bias."  By (very deliberately) effacing the contextual cues that summon to mind the mental routines and embedded reasoning proficiencies necessary to address recurring problem situations, such tests grossly overstate the "boundedness" of human rationality (Gigenrenzer 2000).

Indeed, by abstracting from any particular use to which people might put the reasoning proficiencies they are evaluating, such assessments and problems are actually measuring only how good people are at doing tests. In fact, people can train themselves to become very proficient at a difficult type of reasoning task for purposes of taking an exam on it and then evince complete innocence of that same sort of knowledge in the real-world settings where it actually applies (DiSessa 1982)!

DI and DS have different accounts of "expertise" in fields that involve reasoning tasks that are vulnerable to recurring cognitive biases. DI  identifies that expertise with the cultivation of general, context-free habits of mind that evince the disposition to use "conscious, effortful" ("system 2") forms of information processing (Sunstein 2005).

DS, in contrast, asserts that "expertise" consists in the possession of  mental routines, and their embedded reasoning proficiencies, specifically suited for specialized tasks. Those mental routines  include the calibration of rapid, intuitive, pre-conscious, affective forms of cognition (or better, recognition) that reliably alert the expert to the need to bring certain conscious, effortful mental operations to bear on the problem at hand. The proper integration of reciprocal forms of intuitive and conscious forms of cognition tailored to specialized tasks is the essence of professional judgment.

Nonexperts can be expected to display one or another bias when confronted with those same problems.  But the reason isn't that the nonexpert "thinks differently" from the expert; it's that the expert has acquired through training and experience mental routines suited to do things that are different from anything the ordinary person has occasions to do in his or her life  (Margolis 1987, 1993, 1996). 

Indeed, if one confronts an expert with a problem divorced from all the cues that reliably activate the cognitive proficiencies she uses when she performs professional tasks, one is likely to find that the expert, too, is vulnerable to all manner of cognitive bias.

But if one infers from that that the expert therefore can't be expected to resist those biases in her professional domain, one is making DI's signature mistake of assuming that reasoning proficiencies are stand-alone modules that exist independent of mental routines specifically suited for doing particular things  (cf. Kahan, Hoffman, Evans,Luci, Devins & Cheng in press) ....

Or that at leas is what a DS proponent would say.

She might, then, too agree that the reason-eviscerating quality of "coherence based reasoning" supplies us with grounds to professionalize fact-finding in legal proceedings.

Not because "jurors" or other "nonexperts" are "stupid." But because it is stupid to think that doing what is required to make accurate findings of fact in legal proceedings does not depend on the cultivation of habits of mind specifically suited for that task.

I tend to think the DS proponent comes closer to getting it right. But of course, I'm not really sure.


DiSessa, A.A. Unlearning Aristotelian Physics: A Study of Knowledge‐Based Learning. Cognitive science 6, 37-75 (1982).

Frederick, S. Cognitive Reflection and Decision Making. Journal of Economic Perspectives 19, 25-42 (2005).

Gigerenzer, G. Adaptive thinking : rationality in the real world (Oxford University Press, New York, 2000).

Gigerenzer, G. & Hug, K. Domain-specific reasoning: Social contracts, cheating, and perspective change. Cognition 43, 127-171 (1992). 

Hetherington, S.C. How to know : a practicalist conception of knowledge (J. Wiley, Chichester, West Sussex, U.K. ; Malden, MA, 2011).

Kahan, D.M., Hoffman, D.A., Evans, D., Devins, N., Lucci, E.A. & Cheng, K. 'Ideology'or'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment. U. Pa. L. Rev. 164 (in press).

Margolis, H. Dealing with risk : why the public and the experts disagree on environmental issues (University of Chicago Press, Chicago, IL, 1996).

Margolis, H. Paradigms and Barriers (1993).

Margolis, H. Patterns, thinking, and cognition : a theory of judgment (University of Chicago Press, Chicago, 1987).

Peters, E., Västfjäll, D., Slovic, P., Mertz, C.K., Mazzocco, K. & Dickert, S. Numeracy and Decision Making. Psychol Sci 17, 407-413 (2006).

Sunstein, C.R. Laws of fear : beyond the precautionary principle (Cambridge University Press, Cambridge, UK ; New York, 2005). 


PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (14)

Nice post. Thanks.
To me, the idea of 'modules' is mostly misleading. There are 'modules' in the sense of bounded sets but modules often overlap with each other.

November 28, 2015 | Unregistered CommenterEric Fairfield

==> "I tend to think the DS proponent is right. But of course, I'm not really sure."

My guess is that it's rather like the nature/nurture argument about development: 1), it's a mistake to think that it's an either/or and 2), the academic argument discussion is somewhat less important than determining how to respond to the situation irrespective of which answer is "correct" (or more "correct").

They used to think that by studying Latin, students would become more logical thinkers. Such thinking underlies the more modern practice of teaching "problem-solving" skills. But from what I've seen, there's little solid evidence of some automatic transfer of problem-solving skills. On the other hand, is it realistic to think that we can greatly help students by taking a serial approach to teaching problem-solving skills in an infinite line of contexts?

IMO, as a teacher, responding to the situation irrespective of whether cognitive skills are DS or DI are more operative, the goal is to focus on helping students apply problem-solving techniques in a particular context and then, examining, explicitly, meta-cognitive strategies for translating the application of those skills in one context to another context, then another, then another. Of course, no matter how long you work together the total sum of contexts you work in will be just a drop in the bucket....but IMO, the goal is to get students to be empowered to autonomously apply a discrete set of strategies...such as planning an approach, evaluating outcomes, revising strategies based on that evaluation, evaluating the revised strategies, etc. If nothing else, such an approach concentrates on process rather than outcomes...and the process is directed at helping students to see themselves as empowered, autonomous, self-directed learners who engage in learning activities without being overly-self critical in the face of failure, and to explore new ideas because of an intrinsic motivation.

November 28, 2015 | Unregistered CommenterJoshua

Joshua or others,
When I was a teacher, I believed the things that you said. Now I am trying, cell by cell, to understand the workings of a brain. How might we all establish a common vocabulary with well defined terms so that we can have more productive discussions, in particular on the DS and DI topics?
For instance, I do not have any clear meaning for 'meta-cognitive strategies' while 'learning activities without being ...' seems to have a different meaning from every person that I talk to.

November 28, 2015 | Unregistered CommenterEric Fairfield

Eric -

==> " I do not have any clear meaning for 'meta-cognitive strategies' 'while learning activities without being ...' "

An interesting question. And yes, I acknowledge a general ambiguity to the terms, certainly if you're going to dig down to the level of what's actually going on/being structured in the brain at the cellular level.

On the other hand, I think that mostly what I'm talking about references an affective/emotional domain. Certainly, that domain is rife with unknowns...but at some level we don't need to know how to define emotions in some precise manner at the micro-level, or even at more macro-levels, to know that emotions affect learning.

Having a deliberative and controlled impact at the emotional level on how students go about learning is certainly a complicated process. General approaches certainly don't work with all students. I think that you need to look long-term and give space for developmental processes to unfold, and you have to try not to fall into an illusion that you control the process - with a result of students conceptualizing learning as something that is extrinsically motivated. Are a student's emotional exchange with learning hard-wired, a product of genetics, impervious to controlled external influences? Or is it not necessarily 100% genetic, but largely genetically predictable in conjunction with the influence of external influences very early in development (pre-natal, prior to any impact from schooling)?

Perhaps...but we certainly know that children learn as they develop, and we certainly know that there are associations between affective/emotional characteristics of students and their processes of learning. And perhaps my reasoning is self-sealing ("motivated" by my identification with being a teacher), but from my biased experiences I think that it's a pretty safe bet that by creating a well-designed didactic environment that helps direct students to advance along a natural developmental path, we can affect the emotional/affective life of a student even if we can't precisely define what we're doing at the micro- or even more macro-levels.

None of that is to say that efforts to understand better, as you're doing, isn't extremely important.

November 28, 2015 | Unregistered CommenterJoshus

Can you supply some examples of current and older 'meta-cognitive strategies' and compare them to cognitive strategies? Also, are 'meta-cognitive strategies' a rebranding of strategies that I would already know about? I am still confused as to what is meant by the term. Thanks.

November 28, 2015 | Unregistered CommenterEric Fairfield

Typo in the paragraph that starts with "DI and DS have different accounts..."

November 28, 2015 | Unregistered CommenterNathan N

@Nathan N:
Thx. You obviously are very high in System 2!

November 28, 2015 | Registered CommenterDan Kahan

Eric -

Not sure I can do better than give you some Google links:


Googling will return more, that are mostly similar.

To make it more concrete....

I have worked a lot with students - at a variety of levels (e.g., undergraduate students, graduate students, MBA students, executives taking advanced management courses) - on developing metacogngitive strategies for reading expository texts for academic purposes. As one ex;ample: I worked a lot with Japanese executives taking courses where they used case studies. Many of them approached the case studies by reading every word, from the first to the last, underlining each word they didn't understand (including idiomatic expressions, slang, etc.) and looking up the words they didn't know in an English-Japanese dictionary. Such a process of reading would take them many, many hours, and at the end, they would often be completely unprepared for the case study discussions - which largely focused on having students think outside the text to analyze problems presented in the cases and on having students think of how their own business experiences in their own market sector could be useful for the group to consider when evaluating the case study.

We would then discuss meta-cognitive strategies for reading those case studies...where they would spend considerably less time reading the material but in the end, be far better prepared for engaging in the case study discussions. I could describe the strategies in more detail (they aren't that complicated), ;and why their preexisitng approach was largely ineffective - but the point is that probably you'd find that they strategies I'm describing are similar to those that you apply when reading expository texts, but that the Japanese execs would be completely unfamiliar with because of the cultural framework in which they approach learning tasks. Similarly while some students at graduate levels, or probably all academics, apply similar types of strategies as those I'm talking about, such strategies are rarely taught explicitly to American students. Indeed, quite a few graduate students are not deliberate about employing such strategies and in my experience very few undergraduate students or students below those levels are likely to employ explicit or even less deliberative strategies when they approach a reading task.

November 30, 2015 | Unregistered CommenterJoshua

That was very helpful. Thanks.

November 30, 2015 | Unregistered CommenterEric Fairfield

Very helpful post. Reminds me of E.D. Hirsch's work on reading comprehension. He strongly believes that DI reasoning is highly exaggerated. With respect to reading, content knowledge (i.e. DS-type skills) matters much more than generalized critical thinking.

November 30, 2015 | Unregistered CommenterPraj

==> "With respect to reading, content knowledge (i.e. DS-type skills) matters much more than generalized critical thinking."

I'm hoping you could explain that a bit more. I would say that what you said depends on what you consider "generalized critical thinking," and what you mean by "with respect to reading," and what task you're measuring.

Generalized critical thinking could mean something like the meta-cognitive strategies I described above, and I would say that they are very important with respect to students making meaning out of what they read. Certainly, stimulating and connecting with prior knowledge about a particular subject (i.e., "content knowledge") before and while reading is an important part of making meaning out of what we read, but those steps are, I would say, component pieces of the overall critical thinking aspects of reading.

Further, while it would seem logical to say that all else considered equal (such as the degree to which two readers employ critical thinking strategies when reading), prior content knowledge largely helps to explain reading comprehension, that isn't particularly useful for understanding how people make sense of material that they are unfamiliar with.

November 30, 2015 | Unregistered CommenterJoshua

Eric -

Cheers. Glad it was helpful.

November 30, 2015 | Unregistered CommenterJoshua

@Joshua: thanks for the follow-up. I was just trying to regurgitate what I had read about E.D. Hirsch's work, and say that this discussion reminded me of it. Wasn't taking a hard stance myself.

December 2, 2015 | Unregistered CommenterPraj

Praj --

Ok, thanks for the follow-up to the follow-up. I'll do some Googling to scratch my itch.

December 2, 2015 | Unregistered CommenterJoshua

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>