follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Cultural consensus worth protecting: robots are cool! | Main | The Goldilocks "theory" of public opinion on climate change »

Two common (& recent) mistakes about dual process reasoning & cognitive bias

"Dual process" theories of reasoning -- which have been around for a long time in social psychology -- posit (for the sake of forming and testing hypotheses; positing for any other purpose is obnoxious) that there is an important distinction between two types of mental operations.

Very generally, one of these involves largely unconscious, intuitive reasoning and the other conscious, reflective reasoning.

Kahneman calls these "System 1" and "System 2," respectively, but as I said the distinction is of long standing, and earlier dual process theories used different labels (I myself like "heuristic" and "systematic,” the terms used by Shelley Chaiken and her collaborators; the “elaboration likelihood model” of Petty & Cacioppo uses different labels but is very similar to Chaiken’s “heuristic-systematic Model”).

Kahneman's work (including most recently his insightful and fun synthesis “Thinking Fast, Slow”) has done a lot to focus attention on dual process theory, both in scholarly research (particularly in economics, law, public policy & other fields not traditionally frequented by social psychologists) and in public discussion generally.

Still, there are recurring themes in works that use Kahneman’s framework that reflect misapprehensions that familiarity with the earlier work in dual process theorizing would have steered people away from.

I'm not saying that Kahneman — a true intellectual giant — makes these mistakes himself or that it is his fault others are making them. I'm just saying that it is the case that these mistakes get made, with depressing frequency, by those who have come to dual process theory solely through the Kahneman System 1-2 framework.

Here are two of those mistakes (there are more but these are the ones bugging me right now).

1. The association of motivated cognition with "system 1" reasoning.  

"Motivated cognition," which is enjoying a surge in interest recently (particularly in connection with disputes over climate change), refers to the conforming of various types of reasoning (and even perception) to some goal or interest extrinsic to that of reaching an accurate conclusion.  Motivated cognition is an unconscious process; people don't deliberately fit their interpretation of arguments or their search for information to their political allegiances, etc. -- this happens to them without their knowing, and often contrary to aims they consciously embrace and want to guide their thinking and acting.

The mistake is to think that because motivated cognition is unconscious, it affects only intuitive, affective, heuristic or "fast" "System 1" reasoning. That's just false. Conscious, deliberative, systematic, "slow" "System 2" can be affected be affected as well. That is, commitment to some extrinsic end or goal -- like one's connection to a cultural or political or other affinity group -- can unconsciously bias the way in which people consciously interpret and reason about arguments, empirical evidence and the like.

This was one of the things that Chaiken and her collaborators established a long time ago. Motivated systematic reasoning continues to be featured in social psychology work (including studies associated with cultural cognition) today.

One way to understand this earlier and ongoing work is that where motivated reasoning is in play, people will predictably condition the degree of effortful mental processing on its contribution to some extrinsic goal. So if relatively effortless heuristic reasoning generates the result that is congenial to the extrinsic goal or interest, one will go no further. But if it doesn't -- if the answer one arrives at from a quick, impressionistic engagement with information frustrates that goal -- then one will step up one's mental effort, employing systematic (Kahneman's "System 2") reasoning.

But employing it for the sake of getting the answer that satisfies the extrinsic goal or interest (like affirmation of one's cultural identity defining group). As a result, the use of systematic or "System 2" reasoning will thus be biased, inaccurate.

But whatever: Motivated cognition is not a form of or a consequence of "system 1" reasoning. If you had been thinking & saying that, stop. 

2.  Equation of unconscious reasoning with "irrational" or biased reasoning, and equation of conscious with rational, unbiased.

The last error is included in this one, but this one is more general.

Expositors of Kahneman tend to describe "System 1" as "error prone" and "System 2" as "reliable" etc.

This leads lots of people to think that that heuristic or unconscious reasoning processes are irrational or at least "pre-rational" substitutes for conscious "rational" reasoning. System 1 might not always be biased or always result in error but it is where biases, which, on this view, are essentially otherwise benign or even useful heuristics that take a malignant turn, occur. System 2 doesn't use heuristics -- it thinks things through deductively, algorithmically  -- and so "corrects" any bias associated with heuristic, System 1 reasoning.

Wrong. Just wrong. 

Indeed, this view is not only wrong, but just plain incoherent.

There is nothing that makes it onto the screen of "conscious" thought that wasn't (moments earlier!) unconsciously yanked out of the stream of unconscious mental phenomena. 

Accordingly, if a person's conscious processing of information is unbiased or rational, that can only be because that person's unconscious processing was working in a rational and unbiased way -- in guiding him or her to attend to relevant information, e.g., and to use the sort of conscious process of reasoning (like logical deduction) that makes proper sense of it.

But the point is: This is old news! It simply would not have occurred to anyone who learned about the dual process theory from the earlier work to think that unconscious, heuristic, perceptive or intuitive forms of cognition are where "bias" come from, and that conscious, reflective, systematic reasoning is where "unbiased" thinking lives.

The original dual process theorizing conceives of the two forms of reasoning as integrated and mutually supportive, not as discrete and hierarchical. It tries to identify how the entire system works -- and why it sometimes doesn't, which is why you get bias, which then, rather than being "corrected" by systematic (System 2) reasoning, distorts it as well (see motivated systematic reasoning, per above).

Even today, the most interesting stuff (in my view) that is being done on the contribution that unconscious processes like "affect" or emotion make to reasoning uses the integrative, mutually supportive conceptualization associated with the earlier work rather than the discrete, hierarchical conceptualization associated (maybe misassociated; I'm not talking about Kahneman himself) with System 1/2.

Ellen Peters, e.g., has done work showing that people who are high in numeracy -- and who thus posses the capacity and disposition to use systematic (System 2) reasoning -- don't draw less on affective reasoning (System 1...) when they outperform people who are low in spotting positive-return opportunities. 

On the contrary, they use more affect, and more reliably.

In effect, their unconscious affective response (positive or negative) is what tells them that a "good deal" — or a raw one — might well be at hand, thus triggering the use of the conscious thought needed to figure out what course of action will in fact conduce to the person's well-being.

People who aren't good with numbers respond to these same situations in an affectively flat way, and as a result don't bother to engage them systematically.

This is evidence that the two processes are not discrete and hierarchical but rather are integrated and mutually supportive.  Greater capacity for systematic (okay, okay, "system 2"!) reasoning over time calibrates heuristic or affective processes (system 1), which thereafter, unconsciously but reliably, turns on systematic reasoning.

So: if you had been thinking or talking as if  System 1 equaled "bias" and System 2 "unbiased, rational," please just stop now.

Indeed, to help you stop, I will use a strategy founded in the original dual process work.

As I indicated, believing that consciousness leaps into being without any contribution of unconsciousness is just incoherent. It is like believing in "spontaneous generation."  

Because the idea that System 2 reasoning can correct unconscious bias without the prior assistance of unconscious, system 1 reasoning is illogical, I propose to call this view "System 2 ab initio bias.”

The effort it will take, systematically,  to figure out why this is an appropraite thing for someone to accuse you of if you make this error will calibrate your emotions: you'll come to be a bit miffed when you see examples; and you'll develop a distinctive (heuristic) aversion to becoming someone who makes this mistakes and gets stigmatized with a humiliating label.

And voila! -- you'll be as smart (not really; but even half would be great!) as Shelly Chaiken, Ellen Peters, et al. in no time!


Chaiken, S. & Maheswaran, D. Heuristic Processing Can Bias Systematic Processing - Effects of Source Credibility, Argument Ambiguity, and Task Importance on Attitude Judgment. Journal of Personality and Social Psychology 66, 460-473 (1994).

Chaiken, S. & Trope, Y. Dual-process theories in social psychology, (Guilford Press, New York, 1999).

Chen, S., Duckworth, K. & Chaiken, S. Motivated Heuristic and Systematic Processing. Psychol Inq 10, 44-49 (1999).

Dunning, E.B.a.D. See What You Want to See: Motivational Influences on Visual Perception. Journal of Personality and Social Psychology 91, 612-625 (2006).

Giner-Sorolla, R. & Chaiken, S. Selective Use of Heuristic and Systematic Processing Under Defense Motivation. Pers Soc Psychol B 23, 84-97 (1997).

Hsee, C.K. Elastic Justification: How Unjustifiable Factors Influence Judgments. Organ Behav Hum Dec 66, 122-129 (1996).

Kahan, D.M. The Supreme Court 2010 Term—Foreword: Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law Harv. L. Rev. 126, 1 (2011). 

Kahan, D.M., Wittlin, M., Peters, E., Slovic, P., Ouellette L.L., Braman, D., Mandel, G. The Tragedy of the Risk-Perception Commons: Culture Conflict, Rationality Conflict, and Climate Change. CCP Working Paper No. 89 (June 24, 2011).

Kahneman, D. Thinking, fast and slow, (Farrar, Straus and Giroux, New York, 2011).

Kahneman, D. Maps of Bounded Rationality: Psychology for Behavioral Economics. Am Econ Rev 93, 1449-1475 (2003).

Kunda, Z. The Case for Motivated Reasoning. Psychological Bulletin 108, 480-498 (1990).

Peters, E., et al. Numeracy and Decision Making. Psychol Sci 17, 407-413 (2006).

Peters, E., Slovic, P. & Gregory, R. The role of affect in the WTA/WTP disparity. Journal of Behavioral Decision Making 16, 309-330 (2003).


PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (2)

In around the turn of the century, the dominant Carol-Horn-Cattell theory of intelligence, introduced a cognitive factor known as - gLR, the brain's long-term memory and retrieval factor. This factor was a new addition to the handful of already established cognitive factors. When gLR was analyzed, it revealed latent factor g loadings which exceed even that of Fluid intelligence - formal logic. In laymen's terms this means, that gLR includes mental tasks which involve highly complex underlying processes; and thus correlate with, and are important to, general intelligence. A factor that had been ignored for some time (and still is on other tests of intellectual functioning), comes out of nowhere, to become a major intellectual sub-factor. And while this may have only been interesting news to some, it certainly was a shock to others. A quiet panic must have ensued in the minds' of the logicians, as they wondered how another underlying factor could possibly rival logic?! Was our art of reason under threat? And what must be done to preserve it?

The recent motivations behind dichotomizing human thought, of course, to suggest the alleged superiority of the (various) ''System 2' constructs, which are all arbitrarily restricted to, and implcitly refer to underlying Short term/Working memory processes, where many (but not all) of the processes of formal logic take place. I won't speak directly, therefore, of these invalid systems, but the refer to certain sub-processes (often associated with the systems), which will guide you through my point.

There are number of elements that reflect biases against heuristic-associative processes (generally referred to as system 1). Firstly, the associative-heurisitc is often defined as being impulsively 'quick', and part of the subconsious. But this has always been based on task performance where associations have been paired with a behavoural response. Associative thinking in humans, is generally distinctive of the emotionally linked pavolovian mechanism observed in animals, though it certainly evolved from it. Yes, humans can still be conditioned in the same way as animals, but their is no reason to present the entirety of the associative-heuristic, as a function of classical conditioning. The most important process of the associative-heuristic involves deep concentrative searches, where subtle cues are hard-targeted and returned. This actually invovles a high degree of consious thought, much much, higher than the fire-and-forget type of processing, one is used to from WM/STM processing. Secondly, the associative-heuristic is often blamed as being prone to bias. More recent evidence, as you have generously suggested yourself, has clearly shown that both systems are prone to bias. Thirdly, is the notion that the associative-heuristic is often deemed as being 'contextual', probably in order to mislead one towards the idea that it does not involve novel thinking. While it is true that it is contextual, it is by no means dependent on 'privileged knowledge' (calculus, economics, literature ect). The Heuristic-associative process applys to all forms of knowledge (free knowledge) , but most importantly - it is still clearly involved in generating 'novel' solutions to problems (as is 'System 2'). Third, and related, the associative-heuristic process is widely suggest to involve 'non-abstract' thought. This is plainly false. 'Abstract', by it's definition, can be referred to anything which is even slightly implicit - all problems are technically abstract, though limited in abstraction based on their individual (logical, associative, spatial, verbal ect;) 'relational complexities' - both the associative heuristic process and WM/STM processes are clearly 'abstract' thought processes, as they require the application of some complex method. Furthermore, the associative-heuristic processes is 'abstract' in even a second sense, in that it requires a person to think 'outside of an immediate problem' in order to establish a relationship. Fourth, is the most blatantly biased notion, that 'hypothetical thinking' is restricted to logical processes. The associative-heuristic process, is the factor directly involved in retrieval (and generating ideas), which are required for establishing and generating hypothetical propositions. Of course Short Term memory/working memory ('system 2') processes must finalize the hypotheticals by evaluating there logical consistency. The WM/STM system is capable of generating ideas in of itself, but is severely restricted in it's range of hypotheticals - it can only generate ideas if faced with elements that contain purely logical relationships AND only based on what information is limited in STM/Working memory. 'System 2', almost literally, cannot think outside of it's restricted box. If it is faced with a problem marked by 'associatively' abstract elements, or a problem too large for it's capacity, it will call on the associtive-heuristic processes. Finally, some of the authors intend to further polarize human thought, with the crafty implication that the associative-heuristic is not related to general intelligence. However, a supportive argument for such a claim is only achieved by, again, selectively restricting their version of 'system 1' processes, to only a type of primitive Pavlovian, behavioral response. (with the same arbitrary 'system 1 paradigm' established, they then refer to it's 'quickness' and error-proneness, as reflected in certain tasks (wason selection)). As I have mentioned, the cognitive factor underlying associative-heuristic thought, glr, actually correlates quite nicely with general intelligence.

Of Course, these two systems do not exist, as the way they are typically defined, let alone independently of each other. Both processes are relevant to complex cognition and problem solving. Sure logic may be more important , but associative-heuristics deserves credit. Until it does, the unfortunate are stuck with the bot-theory of human reasoning.

An abstract associative/logical relationship

Analyze.... ... ... ... no evident logical relationship between DOG:: 9 LIVES (Whether formal logic actually goes first or not, is irrelevant)
4) Engage ASSOCIATIVE-HEURISTICS, BEGIN Hypothetical Thinking:

NINE LIVES.......[search] .... ... . .CAT (match)
Establish relationship between CAT: NINE LIVES : superstituous quality about all cats

Apply general form A ---> B
Generate Hypothetical proposition: A ---> Superstisious quality(Probablistic Opposite (A))

November 14, 2013 | Unregistered CommenterTJ Henkle

Very interesting. Not an expert here, but if I may, I would add:

ELM and HSM - propose two System 2 processes - low effort v/s high effort. Indeed, one could argue that there is NO System 1 in ELM or HSM. See Evans (2011) and Ferguson and Bargh (2007) for these arguments.

Kahneman (more explicit about this in his other work e.g., Kahneman & Frederick (2005), uses a model in which System 1 judgments occurs by default, while System 2 is sometimes activated. If activated, System 2 can endorse, correct or override System 1 judgments. Evans calls it a "default - interventionist" view of decision.

May 7, 2014 | Unregistered CommenterCurious

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>