Reflections on “System 2 bias,” part 1 of 2 (I think)

Some thoughts about Miller & Sanjurjo, Part 1 of 2:

Most of the controversy stirred up by M&S centers on whether they are right about the methodological defect they detected in Gilovich, Vallone, and Tversky (1985) (GVT) and other studies of the “hot hand fallacy.”

I’m fully persuaded by M&S’s proof. That is, I get (I think!) what the problem is with GVT’s specification of the null hypothesis in this setting.

Whether in fact GVT’s conclusions about basketball shooting hold up once one corrects this defect (i.e., substitutes the appropriate null)  is something I feel less certain of, mainly because I haven’t invested as much time in understanding that part of M&S’s critique.

But what interests me even more is what the response to M&S tells us about cognition.

The question, essentially, is how could so many extremely smart people (GVT & other empirical investigators; the legions of teachers who used GVT to instruct 1,000’s of students, et al.) have been so wrong for so long?! Why, too, does it remain so difficult to make those intelligent people get the problem M&S have identified?

The answer that makes the most sense to me is that the GVT and others were, ironically, betrayed by intuitions they had formed for sniffing out the general public’s intuitive mistakes about randomness.

The argument goes something like this:

I. The quality of cognitive reflection depends on well calibrated non-conscious intuitions.

There is no system 2 ex nihilo. Anything that makes it onto the screen of conscious reflection (System 2) was moments earlier residing in the realm of unconscious thought (System 1).  Whatever yanked that thought out and projected it onto the screen, moreover, was, necessarily, an unconscious mental operation of some sort, too.

It follows that reasoners who are adept at System 2 (conscious, deliberate, analytical) thinking necessarily possess well behaved System 1 (unconscious, rapid, affect-laden) intuitions. These intuitions recognize when a decisionmaking task (say, the detection of covariance) merits the contribution that System 2 thinking can make, and activates the appropriate form of conscious, effortful information processing.

In anyone lucky enough to have reliable intuitions of this sort, what trained them was, most likely, the persistent exercise of reliable and valid System 2 information processing, as brought to bear over & over in the process of learning how to be a good thinker.

In sum, System 1 and System 2 are best though of not as discrete and hierarchical modes of cognition but rather as integrated and reciprocal ones.

II.  Reflective thinkers possess intuitions calibrated to recognize and avoid the signature lapses in System 1 information processing.

The fallibility of intuition is at the core of all the cognitive miscues (the availability effect; hindsight bias; denominator neglect; the conjunction fallacy, etc.) cataloged by Kahneman and Tversky and their scholarly descendents (K&T et al.).  Indeed, good thinking, for K&T et al., consists in the use of conscious, effortful, System 2 reflection to “override” System 1 intuitions when reliance on the latter would generate mistaken inferences.

As discussed, however, System 2 thinking cannot plausibly be viewed as operating independently of its own stable of intuitions, ones finely calibrated to recognize System 1 mistakes and to activate the sort of conscious, effortful thinking necessary to override them.

III. But like all intuitions, the ones relfective people rely on will be subject to characteristic forms of failure—ones that cause them to overestimate instances of overreliance on error-prone heuristic reasoning.

It doesn’t follow, though, that good thinkers will never be misled by their intuitions.  Like all forms of pattern recognition, the intuitions that good thinkers use will be vulnerable to recurring illusions and blind spots.

The sorts of failures in information processing that proficient thinkers experience will be predictably different from the ones that poor and mediocre thinkers must endure.  Whereas the latter’s heuristic errors expose them to one or another form of overreliance on System 1 information processing, the latter’s put them at risk of too readily perceiving that exactly that form of cognitive misadventure accounts for some pattern of public decisionmaking.

The occassions in which this form of “System 2 bias” will affect thinking are likely to be rare.  But when they occur, the intuitions that are their source will cling to individuals’ perceptions with the same dogged determination that the ones responsible for heuristic System 1 biases do.

Something like this, I believe, explains how the “ ‘hot hand fallacy’ fallacy” took such firm root.

It’s a common, heuristic error to believe that independent events—like the outcome of two coin flips—are interdependent. Good reasoners are trained to detect this mistake and to fix it before making a judgment.

GVT spotted what they surmised was likely an instance of this mistake: the tendency of fans, players, and coaches to believe that positive performance, revealed by a short-term string of successful shots, indicated that a player was “hot.”

They tested for this mistake by comparing whether the conditional probability of a successful basketball shot following a string of successes differed significantly from a player’s unconditional probability of making a successful shot.

It didn’t. Case closed.

What didn’t occur to them, though, was that where one uses the sampling method they used—drawing from a finite series without replacement—Pr(basket|success, success, sucses) – Pr(basket) should be < 0. How much below zero it should be has to be determined analytically or (better) by computer simulation.

So if in fact Pr(basket|success, success, sucses) – Pr(basket) = 0, the player in question was on an improbable hot streak.

Sounds wrong, doesn’t it? Those are your finely tuned intuitions talking to you; yet they’re wrong. . . .

I’ll finish off thise series “tomorrow.™”  In the meantime, read this problem & answer the three questions that pertain to it.


Gilovich, T., Vallone, R. & Tversky, A. The hot hand in basketball: On the misperception of random sequences. Cognitive Psychology 17, 295-314 (1985).

Leave a Comment