follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Lecture on Science of Science Communication at Penn State (lecture slides) | Main | A fragment: The concept of the science commmunication environment »

Some data on education, religiosity, ideology, and science comprehension

No, this blog post is not a federally funded study. It's neither "federally funded" nor a "study"! Doesn't it bug you that our hard-earned tax dollars pay the salary of a federal bureaucrat too lazy to figure out simple facts like this?

Because the "asymmetry thesis" just won't leave me alone, I decided it would be sort of interesting to see what the relationship was between a "science comprehension" scale I've been developing and political outlooks.

The "science comprehension" measure is a composite of 11 items from the National Science Foundation's "Science Indicators" battery, the standard measure of "science literacy" used in public opinion studies (including comparative ones), plus 10 items from an extended version of the Cognitive Reflection Test, which is normally considered the best measure of the disposition to engage in conscious, effortful information processing ("System 2") as opposed to intuitive, heuristic processing ("System 1").  

The items scale well together (α= 0.81) and can be understood to measure a disposition that combines substantive science knowledge with a disposition to use critical reasoning skills of the sort necessary to make valid inferences from observation. We used a version of a scale like this--one combining the NSF science literacy battery with numeracy--in our study of how science comprehension magnifies cultural polarization over climate change and nuclear power.

Although the scale is designed to (and does) measure a science-comprehension aptitude that doesn't reduce simply to level of education, one would expect it to correlate reasonably strongly with education and it does (r = 0.36, p < .01). The practical significance of the impact education makes to science comprehension so measured can be grasped pretty readily, I think, when the performance of those who have and who haven't graduated from college is graphically displayed in a pair of overlaid histograms:

The respondents, btw, consisted of a large, nationally representative sample of U.S. adults recruited to participate in a study of vaccine risk perceptions that was administered this summer (the data from that are coming soon!).

Both science literacy and CRT have been shown to correlate negatively with religiosity. And there is, in turns out, a modest negative correlation (r = -0.26, p < 0.01) between the composite science comprehension measure and a religiosity scale formed by aggregating church attendance, frequency of prayer, and self-reported "importance of God" in the respondents' lives.

I frankly don't think that that's a very big deal. There are plenty of highly religious folks who have a high science comprehension score, and plenty of secular ones who don't.  When it comes to conflict over decision-relevant science, it is likely to be more instructive to consider how religiosity and science comprehension interact, something I've explored previously.

Now, what about politics?

Proponents of the "asymmetry thesis" tend to emphasize the existence of a negative correlation between conservative political outlooks and various self-report measures of cognitive style--ones that feature items such as  "thinking is not my idea of fun" & "the notion of thinking abstractly is appealing to me." 

These sorts of self-report measures predict vulnerability to one or another reasoning bias less powerfully than CRT and numeracy, and my sense is that they are falling out of favor in cognitive psychology. 

In my paper, Ideology, Motivated Reasoning, and Cognitive Reflection, I found that the Cogntive Reflection Test did not meaningfully correlate with left-right political outlooks.

In this dataset, I found that there is a small correlation (r = -0.05, p = 0.03) between the science comprehension measure and a left-right political outlook measure, Conservrepub, which aggregates liberal-conservative ideology and party self-identification. The sign of the correlation indicates that science comprehension decreases as political outlooks move in the rightward direction--i.e., the more "liberal" and "Democrat," the more science comprehending.

Do you think this helps explain conflicts over climate change or other forms of decision-relevant science? I don't.

But if you do, then maybe you'll find this interesting.  The dataset happened to have an item in it that asked respondents if they considered themselves "part of the Tea Party movement." Nineteen percent said yes.

It turns out that there is about as strong a correlation between scores on the science comprehension scale and identifying with the Tea Party as there is between scores on the science comprehension scale and Conservrepub.  

Except that it has the opposite sign: that is, identifying with the Tea Party correlates positively (r = 0.05, p = 0.05) with scores on the science comprehension measure:

Again, the relationship is trivially small, and can't possibly be contributing in any way to the ferocious conflicts over decision-relevant science that we are experiencing.

I've got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I'd be shown a modest negative correlation between identifying with the Tea Party and science comprehension.

But then again, I don't know a single person who identifies with the Tea Party.  All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).  

I'm a little embarrassed, but mainly I'm just glad that I no longer hold this particular mistaken view.

Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for. I just no longer assume that the people who happen to hold those values are less likely than people who share my political outlooks to have acquired the sorts of knowledge and dispositions that a decent science comprehension scale measures.

I'll now be much less surprised, too, if it turns out that someone I meet at, say, the Museum of Science in Boston, or the Chabot Space and Science Museum in Oakland, or the Museum of Science and Industry in Chicago is part of the 20% (geez-- I must know some of them) who would answer "yes" when asked if he or she identifies with the Tea Party.  If the person is there, then it will almost certainly be the case that that he or she & I will agree on how cool the stuff is at the museum, even if we don't agree about many other matters of consequence.

Next time I collect data, too, I won't be surprised at all if the correlations between science comprehension and political ideology or identification with the Tea Party movement disappear or flip their signs.  These effects are trivially small, & if I sample 2000+ people it's pretty likely any discrepancy I see will be "statistically significant"--which has precious little to do with "practically significant."

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (1)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    Will he now do a study to document the cognitive biases of the left, their tendency to demean as nearly subhuman all those who disagree with them?

Reader Comments (284)

Here's a topic of a future study for you. Assess the level of bias in a scientist(?) who makes the following statements:
1. "I've got to confess, though, I found this result surprising." (that the results of his own analysis of his own study shows that tea party members had a higher than average level of science comprehension).
2. "I don't know a single person who identifies with the Tea Party. All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico)."
3. "Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for."

The second part of your study might be to find common sense solutions to this clearly unacceptable bias toward a group of people whom you admittedly know virtually nothing about or worse yet base your opinion on the clearly biased opinions of others. Here's a hint. Start associating with intelligent people who may have a different political viewpoint than the one you currently hold. Start consuming news/commentary from sources that have diverse viewpoints.

By the way, as a person who identifies himself as a Tea Party guy I find your stated goal (I believe I read this in another of your posts) of using clear scientific evidence to inform public policy to be admirable. My apologies if I've misunderstood or misstated your goals.

October 21, 2013 | Unregistered Commentertonyguy01


1. You understand my goals -- & by asking me these questions you show that you support them! thx

2. Show me someone who does empirical reasearch who never says, "I was surprised to find ..." -- or better, in public has never reported that were contrary his or her hypotheses -- & I'll show you an empirical researcher whose results you shouldn't trust.

3. Here's a study that shows that we should worry about whether scientists are likely to recognize the validity of evidence that surprises them. Scientists know this can happen & appropriately worry a lot about it.

4. Here's a study that shows that people who are high in science comprehension -- of which there clearly are many in the Tea Party, as there are in the Democratic and Republican Parties -- are vulnerable to the same type of reasoning bias when considering scientific evidence on the effectiveness of a politically charged issue (gun control). That worries me, and I suspect it worries you-- even though we likely disagree about lots & lots of really important political matters.

5. I said learning something that surprised me about a fact -- that people in the Tea Party are exactly the same in science comprehension as everyone else (despite the correlation between Tea Party membership and religiosity, which is negatively correlated w/ science comprehsnion--interesting! not that I think the negative correlation between science comprehension and religioisity means anything important about whether religious people's views on politics are right or wrong)-- didn't change my disagreement with the Tea Party's stances on issues of fundamental importnace to it: Obamacare, say, or progressive income taxation. Those positions-- the ones you have and the ones I do -- aren't based on scientific facts; they are based on our values. So learning that Tea party members are like everyone else in science comprehension (something that in my view is actually not particularly relevant to political disagreements) gives me no reason to change my views on those policy issues! My goal, really, was to try to show people what a bad error it is to think that people who disagree w/ them on policy matters must be "stupid" (this bugs me a lot-- I was trying to get people who share my values to stop making this mistake). But it wasn't to show that people should therefore agree w/ anyone in particular policy matters; differences in values are important. You and I need to discuss the matters where are values are in conflict & work them out through democratic deliberation -- that's what democratic politics is about.

6. But if we *agree* that it's a problem that people who have either of our sets of values are vulnerable to misreading scientific evidence on controversial matters -- including "whose side knows more science"; what a childish exercise! -- then we should probably spend some amount of our time and attention trying to figure out if there's something we can do to keep this problem from distorting or interfering with our democratic deliberations. The same way that scientists spend time trying to come up w/ procecdures that make their temptation to credit evidence that supports results they like more readily than evidence that contravenes their expectations. Agree?

October 21, 2013 | Registered CommenterDan Kahan

@Dan Kahan:
"didn't change my disagreement with the Tea Party's stances on issues of fundamental importnace to it: Obamacare, say, or progressive income taxation. Those positions-- the ones you have and the ones I do -- aren't based on scientific facts; they are based on our values."
This is only true if economics, sociology, and statistics are unscientific.
Like, for example, that less than 0.5% of medical spending can be attributed to the voluntarily uninsured, the targets of Obamacare's individual mandate; that because Obamacare's individual mandate guarantees customers to health insurance companies, they're jacking up their rates and making health insurance even less affordable*; or that economists generally recognize that income and corporate taxes inhibit economic growth. You might also find it interesting that even Karl Marx found bureaucracy to be redundant, error-prone, and generally bumbling (not to mention the fact that he seemed to think they were of the same nature as corporations).

* additional requirements for compliant insurance may also be a factor

October 21, 2013 | Unregistered CommenterNate Fries


Thanks for your thorough reply to my comment.

When you speak of science and our biases interfering with properly evaluating and subsequently utilizing the results of scientific study in informing public policy I think we are in nearly perfect agreement. You'll notice that I use the word "informing" when describing the proper role of science in public policy issues. It should be used to provide knowledge to policy makers and more importantly to the electorate in general. But it should not be the only determinant in policy making. I suspect or at least hope that you would agree with that.

The purpose of my comment though was not to clarify your position in regard to science/public policy matters. It was actually to help you develop as a person by overcoming your own biases. What concerns me is that your self imposed prison of thought regarding those with different political beliefs is preventing you from a more complete understanding of what motivates people both individually and as groups. I repeat your quote from the article: "Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for." and what concerns me most is your moral assessments. I think that if you exposed yourself to people who identify themselves as Tea Party people and read their blogs or heard their comments on news/commentary shows you would come to find that their motivation is less self interest (or greed if you prefer) and more pro freedom and justice. I see the typical Tea Party person as more Libertarian (Classic Liberal) than the typical Democrat who now refers to himself as a Liberal. With this understanding I think you may find the average Tea Party person to be just as highly moral as any Progressive pressing for his Utopia.

October 21, 2013 | Unregistered Commentertonyguy01


But our values & commitments are important enough to us that we will form negative views of those who harbor different ones. Classical Liberal principles, which I have also inherited & feel immeasurable gratitude and commitment toward, don't oblige me to suspend moral judgment of those who come to conclusions different from mine about how to live; they oblige me only to treat those people w/ respect as reasoning and free people, and to abide by laws and institutions suited for protecting the equal right of people endowed with those capacities to pursue their own understanding of happiness by means of their choosing.

It would trivialize my values & yours -- trivialize your reason and mine -- if I just shrugged and said "enh-- who cares?" when we found ourselves debating a matter of political significance in which our values impelled us to opposing stances.

The only sort of "moral" judgment that anyone could make of you, & others of me, that you & I both ought to treat as inappropriate is one that says our welfare or our status or our freedom count less in the eye's of the law b/c of the conclusions about the best way of life you or I have reached through the exercise of our reason and our freedom.

I see the impulse to dismiss your positions as grounded in a deficit in reason as a sad, unconscious outgrowth of that sort of judgment, actually. If I catch it in myself, I want to correct it; if I catch it in someone else, I will point it out -- especially if that person shares my commitements, b/c then I'll know I'm in a special position to help him or her see that, and b/c I know that someone who shares my commitments shudders to think that such a distortion of his or her reason, and a denigration of his or her most fundamental commitments, could occur.

I'm guessing you feel the same way -- guessing b/c it is fairly implicit in what you are saying but also b/c it has been said to me by others now who I think in fact share your outlooks.

If we can agree on that much, we can handle all the other sorts of disagreements, including ones that, understandably, leave us both doubting the moral judgment of the other. The Classical Liberal insight we can live w/ each other despite disagreements of that sort -- that is the greatest contribution that any system of political or philosophical thought has ever made to human wellbeing.

October 21, 2013 | Registered CommenterDan Kahan

"This is only true if economics, sociology, and statistics are unscientific."

They're what are called the "soft sciences," which means that they're not really based on the Scientific Method of experimentation and observation.

"Progressive taxation" is nothing more than thinly veiled theft, based on the politics of envy.

October 21, 2013 | Unregistered CommenterRich Grise

The greatest insight to the furtherance of human well-being is that we are all made in God's image. That perception engenders the maximum possible status to all people and the concomitant respect deserved by all people. Liberalism attempts to replace this with a Darwinian valuation of humanity which accords respect based on species competition and survival. Consequently, mutual respect for differing viewpoints is only justified if it confers a survival benefit to one organism over another. Just as often, disrespect for a rival viewpoint ensures that mine will prevail along with increase with my status and monetary compensation. We witness this every day. Liberalism tries to replace theism with Darwinism and the soul of man lives in isolation.

October 21, 2013 | Unregistered CommenterMike Nelson


I think our positions aren't so different, although obviously the language differs -- and in ways that I don't mean to say aren't importantly different. Indeed, I think it would trivialize how we both feel to say "oh, well, we just use different words." But I think we do get to a fairly broad area of agreement, at least as it is going to affect how we live together in a single political community: "Maximum possible status to all people and the concomitant respect deserved by all people." If we are at the same place on that, then I think it would be ridiculously rude & *disrespectful* for me to object that how you got there is via a different path.

REalize too that I was using Liberalism b/c @Tonyguy01 declared himself -- w/ a pride that I share -- a "Classical Liberal." I'm pretty sure you @Tonyguy01 agree about a lot more thnigs than I & @Tonyguy01 do! So it must be the case that the sort of Liberalsim that I think both he & I see ourselves as the proud inheritors of can be reconciled pretty readily with the political phiolosophy you hold (and which I not only respect but admire).

What you say about Liberalism & Darwinism and survival value & the like, though -- I have to say, that sounds completely alien to me. You aren't describing anything I or others who I share commitements with endorse. I am sure it is related to someone's views in some way but it's just not meaningfully connected to mine or to the views of lots of people like me.

I suspect there are equivalent mistaken characterizations of your views in circulation among people I spend most of my time with. I certainly hear them being circulated sometimes & I try to point out that they are dopey, cartoon versions of thigns other people believe, or ways that they talk. The things that the real versions of those people say might be things that are worthy of disagreeing with -- but going after the cartoons shows only that one hasn't really engaged with others.

Indeed, one of the cartoons that I very much despise is the one that suggests that those who have commitments that I assocaite with conservativism or with the Tea Party or with religiosity are "anti-science." This post was actually an outgrowth of that; I was trying to say how ridiculous it is to think that the reason there is polarization about a lot of policy issues that involve judgments of fact that admit of scientific or empirical inquiry reflects one or another group's ignorance of or hostility to science.

I had a great discussion once w/ someone about how astonishing it was that plants make use of certain really funky quantum-mechanical dynamics to metabolize energy via photosynthesis. We were both saying "whoa! how could that be???? Will it kill the plant if we look at it?! ha ha ha!" -- you have to be really weird science freak to even know what that joke means, much less think it's funny, but we both were slapping our thighs & howling.

And then when we caught our breath, he said to me, "Yeah, God makes some amazing things in his workshop."

No one I had ever hung out with had ever said anything remotely like that.

What's more, I realized that the vast majority of the people who I hang out with -- and get along with well -- don't share the same awe and fascination about what science enables us to learn about the mysteries of nature as this guy shared with me. And I guessed that, yeah, probably most of his friends -- who talk the way he does -- don't find science as cool as the two us either (that's okay!).

So we had an interest in common that connected us to each other that was completely independent of those kinds of connections.

Those other connections are important -- and no doubt the source of lots of values and concerns and commitments that would divide this guy & me on important issues. Fine.

But after that conversation I realized that anything that prevents anyone from seeing that there are people who don't talk the way they do, or value what they do in matters of public life, but who actually do experience the same awe and wonder as they do about what we are able to know through science (and I'm not saying there aren't other ways to know things, or things one must know by other means) -- that is something I despise.

October 21, 2013 | Registered CommenterDan Kahan

@Mike Nelson:
"The greatest insight to the furtherance of human well-being is that we are all made in God's image. That perception engenders the maximum possible status to all people and the concomitant respect deserved by all people. Liberalism attempts to replace this with a Darwinian valuation of humanity which accords respect based on species competition and survival. Consequently, mutual respect for differing viewpoints is only justified if it confers a survival benefit to one organism over another. Just as often, disrespect for a rival viewpoint ensures that mine will prevail along with increase with my status and monetary compensation. We witness this every day. Liberalism tries to replace theism with Darwinism and the soul of man lives in isolation."
I do not think that acceptance of atheism and Darwinism are the cause of "liberal" arrogance and disrespect. Their primary news sources paint their political opponents as stupid, deceitful, or hateful; while suggesting that "liberalism" is the only rational, honest, and kind political viewpoint a person can have. Of course, conservative news hosts frequently depict "liberals" and libertarians as foolish (and occasionally hateful) as well. The problem is that neither way of thinking is hateful, or even necessarily wrong, and yet our primary sources of information are telling us that they are. The sad truth is that The Daily Show is probably the most honest and objective news program in the country, despite John Stewart's obvious leftward bias (also note that he is the only "liberal" news host who will interview prominent conservatives and libertarians without treating them like morons or bigots).
I also don't see how believing that we're made in God's image is any sort of unique boon for human well-being. The realization that except for a very small number of sociopaths, we're all essentially the same, and all in this together, should be more than enough.

October 21, 2013 | Unregistered CommenterNate Fries

I found your study quite interesting. I've been aware that people tend to use studies and data which supports their views while ignoring those which support the opposite view or insisting such studies/data are incorrect. My mother used to say that people see what they want to see and also used to say that even if there isn't any significant correlation, people tend to find what they are looking for (i.e. notice certain sets of data more often than other sets of data, like those who notice certain numbers more often than other numbers and think this means something special). And, no, I'm not immune to this weakness even though I'm aware of it.
When I first stumbled upon an article stating that you were surprised that identifying with the Tea Party had a positive correlation with science comprehension (not exactly how the article stated it), I thought perhaps someone had "punked" the writer of the article, since, once I looked up your study, I understood that this meant they were more likely to ignore data which contradicted their views.
Have you checked to see if this group which identified with the Tea Party more correctly answered the questions which contradicted their religious or political views? Quite frankly, I'd be interested to know if there are any subsets in the scientifically literate groups which correctly answer those questions which challenge their views...did the study also identify subsets within the liberal group?

October 22, 2013 | Unregistered CommenterJMPN

@JMPN Have you checked to see if this group which identified with the Tea Party more correctly answered the questions which contradicted their religious or political views?

FWIW, the two (very short) questionnaires used by Professor Kahan have no "moral/political" questions on them.

I can't speak for any of the other "Tea Party members", but THIS particular TP'er is more than capable of separating her religious, her political, and her scientific views from each other.

Science is SCIENCE; religion and politics have no bearing on hard science and/or math - it never ceases to amaze me that those who consider themselves as oh-so-enlightened condescendingly assume that those who don't share their political views don't know that.

It might surprise them to learn that most people who identify with the Tea Party are quite aware of those distinctions.

I wish those people who view members of the Tea Party with disdain would try venturing outside of their conformational bias bubble once in a while and hob-nob with the common folk - they might find that the stereotypes are nothing more than overly-exaggerated caricatures which bear no resemblance whatsoever to reality.

prej·u·dice \pre'-jə-dəs\

1.opinion formed beforehand: a preformed opinion, usually an unfavorable one, based on insufficient knowledge, irrational feelings, or inaccurate stereotypes

2.holding of ill-informed opinions: the holding of preformed opinions based on insufficient knowledge, irrational feelings, or inaccurate stereotypes

3.irrational dislike of somebody: an unfounded hatred, fear, or mistrust of a person or group

October 22, 2013 | Unregistered CommenterTeresa in Fort Worth, TX


I do anticipate that members of the tea party who are higher in critical reasoning skills of the sort necessary to comprehend science are more likely to display motivated reasoning in their assessment of evidence relating to culturally charged issues -- but the reason I do is that this is true for all groups, right & left (or up & down if one uses a 2-dimensional cultural cognition scheme).

So yes, I think having "more" members who are science comprhending is nothing to brag about (& as far as I can tell, no political or cultural group does have "more"). It is a reason to worry about the stupefying effects that living in a polluted science communication environment can have.

I haven't tried to test, though, whether members of the tea party per se are more or less prone to identity-protective cognition -- my expectation that they are "like everyone else" in that regard is based on all the existing studies showing that identity-protective cognition is magnified by science literacy, numeracy, cognitive reflection, etc..

I suppose I could specifically investigate this issue. But there are 2 reasons I am disinclined to do so.

1st, I'm sick of this "whose is bigger" bull shit.

2d, doing studies that furnish evidence that "whose is bigger" is a bull-shit way to think about the problem of political conflicts over decision-relevant science just provokes everyone to get their rulers out & start measuring. Isn't that obvious? Shouldn't I have figured that out by now?


October 22, 2013 | Registered CommenterDan Kahan

I seem unable to track down the actual battery of questions used. Can you post a link?

October 22, 2013 | Unregistered CommenterLookingForData

@Looking For Data - the CRT is only 3 questions; the NSF is only 10 or 11 (although Professor Kahan said that the gentleman who developed the CRT has a "longer" version, in looking at his data, the additional questions are more a measure of a person's willingness to take risks based on stated probabilities - those questions don't have a "right" answer, but appear to be used for a different metric)

The link for the CRT is at the top of this post ("Cognitive Recognition Test") -

The link for the NSF battery is

(Professor Kahan was kind enough to link both of these in an earlier response)

October 22, 2013 | Unregistered CommenterTeresa in Fort Worth, TX



did you get the long version CRT items? They aren't risk perception or risk preference questions; they are word problems, like the original 3, that are designed to see if test takers resist a common but mistaken reasoning error and instead use the information in the problem to draw a valid inference.

October 24, 2013 | Registered CommenterDan Kahan

You must determine a way to measure and adjust for the likelihood of response distortion from those who might be mistrustful and fear the possibility of exposure and public ridicule of an obviously bias scientific community.

October 24, 2013 | Unregistered Commenterc-ray


Sorry I dropped out of the conversation for a few days.

After reading your (and others) comments and your responses to them I must say that I am still confused and somewhat disappointed by the comments in your post in which you state that you still hold exclusively negative moral judgements of what you believe the Tea Party stands for. As an obvious proponent of the scientific method and someone who is devoting his time to eliminating bias to further the political process, your acceptance of your own bias is stupefying.

I'll again suggest getting a portion of your news and information from other sources and broadening your circle of acquaintances to include people of a different political ilk. Consider it an experiment. I know and associate with many people whose political viewpoints are 180 degrees opposite of mine and we often have spirited discussions. But I don't judge them to be morally deficient. We often have the same broad general goals (which I expect would be judged as morally positive) but differ greatly regarding the role of government in achieving those goals.

By the way - awesome story about the plant photosynthesis.

October 25, 2013 | Unregistered Commentertonyguy01


I don't know that my views are "exclusively negative," but likely they are predominantly so. I am guessing you would feel the same about my political outlooks. There is nothing about that that suggests closed minded ness or prejudice, necessasarily; people who are free to form their own views on political matters will inevitably form independent and varying conclusions. So long as they respect each other's rights to hold those views and act on them in a manner consistent with the equal liberty for all (including their right to appeal to others through argument and advocacy), then they are certainly not being unfair when they form positive or negative views of others based on their moral & political commitments. This is the normal experience of democratic people, don't you think?

October 26, 2013 | Unregistered Commenterdmk38

Haha, I remember my time in academia, when I thought much like you do.

Eventually over time you realize the way your information is filtered is important. Thus people on the right generally have a more complete picture of things -- they cannot avoid the leftwing viewpoints that saturate news and media (as polling of those institutions has shown for decades). By contrast, people in your position tend to know only your side of things.

Once you realize this, if you aspire to intellectual honesty you have to implicitly discount your "objective" sources, realizing that there are probably aspects of virtually any information they transmit that skew to their ideological bent.

Of course that doesn't mean the right is generally or even usually right on its positions, it just means their viewpoint is more likely to be right than it probably appears, because of the filtering disparity.

October 27, 2013 | Unregistered CommenterTallDave

TallDave thinks that we academics get filtered information. That is just wrong. There are plenty of conservatives in the academic world. And they give talks, and we go to the talks. Some of us even read and seek out intelligent discussions on the other side.

That said, speaking for myself, I lose patience with bumper-sticker arguments on both sides, and I see a lot of those in these comments. Bare assertions of a position, without apparent recognition that the position is quite well known, and that many arguments have been made against it, arguments I could provide if anyone would listen. These bumper-sticker sized arguments are offered as if we are supposed to say, "Oh, now I see! I never thought of that before." But the fact is that I did think of it, and read about it, and rejected it, more than once.

If we academics seek out things that are "filtered", it is not by position but rather by the quality of argument. I try to read things that take into account arguments I have already thought of, and go beyond them. That is what academics get credit for doing. That is my filter. When I see one-sided assertions, written as if the readers were so stupid as not to have thought about the issue, I glaze over. If that is a filter, I confess to filtering. But it is not a filter by political position.

Nobody will read this. But I'm just saying it anyway. I am now going to unsubscribe from comments, so I don't have to get upset anymore.

October 27, 2013 | Unregistered CommenterJon Baron

Your correlation calculated (with a p of 0.05), was 0.05. THIS MEANS (with my rusty stats) THAT THERE IS NO CORRELATION whatsoever (not that "there is a small plus correlation"). Professor, you just proved that "tea-party" concept has nothing to do with "science" concept, they're orthogonal. I know you expected a strong minus correlation coef, but you just showed that there's no correlation at all.

October 28, 2013 | Unregistered Commenterranjix


Your stats are indeed "rusty." You need to brush up on your "p's" & "r's"

The correlation is a measure of covariance. A p-value is a measure of the likelihood that such a covariance would be observed by chance if in fact the "true" correlation = 0.

A correlation coefficient of "0.05" w/ a "p-value" of "0.05" doesn't mean "no correlation." It means -- a correlation of 0.05 that one would expect to see by chance in only 5% of one's samples if the true correlation is "0."

I could also have a correlation of 0.50 & a p-value of 0.05 & that would mean that I should expect to see a correlation that big or bigger in only 5% of the my samples if the true correlation is "0."

One could have an r = 0.01 or .000001 for that matter & p = 0.05 (or p = 0.049 or whatever you decide the threshold should be). That would still be a correlation & it would still be "statistically significant" (by that convention).

Whether there's any practical significance to any of these relationships is an entirely separate matter -- one that can't be answered by looking at either "r's" or "p's" but only by considering what's being measured & why one cares.


October 28, 2013 | Unregistered Commenterdmk38

Thanks for answer, didn't realize you posted a response. While the p-value is important, and I appreciate your quick reminder about p-values and r, I was talking about the strength of the correlation, r. The general take on interpreting r is that a value between 0 and 0.1 (in absolute value) shows "little or no relationship between variables". You say yourself, several times in a row, that "the relationship is trivially small". Personally, since it becomes an issue of interpreting the result, I would have ignored it and written about something more interesting, something with hopefully a higher r value.
Thanks again.

November 15, 2013 | Unregistered CommenterRanjix


It all depends.

Attaching labels to effect sizes -- "large," "medium," "small" -- etc. abstracted from the phenomenon that is being measured is a bad idea.

What do you think the effect size is for smoking & mortality in men is over 8-yr period?

r = 0.04

How about taking aspirin & heart attack?

r = -0.03.

Both of these are considered "clinically significant" associations b/c the risk of death was doubled (for smokers & non-aspirin users).

So don't say "r = 0.05" "MEANS. .. THAT THERE IS NO CORRELATION."

Say there is a correlation of 0.05 & figure out if that is practically significant--i.e., use judgment--statistics don't obviate the need for that, as is proven over & over by people mindlessly following statistics rituals like setting thresholds for effect sizes & p-values.

Also, if one expects to see a decent-size correlation & finds there isn't one, that is</> interesting. Saying that "null effects" aren't important or worth writing about is another example of judgment-free use of statistics.

November 15, 2013 | Registered CommenterDan Kahan

"What do you think the effect size is for smoking & mortality in men is over 8-yr period?" Over 8 year period? Frankly I don't expect it to be much, smoking doesn't kill in 8 years (at least don't think so, plus, 8 seems so arbitrary). 0.04? So be it. Also, in this case, it became relevant when compared with the others, non-smokers. In the case of tea partiers, we compare it with the other categories you mentioned (religious, etc), and with those segments in mind, it looks like this correlation becomes more ignorable.
Anyway. It seems that your surprise was that the correlation is not negative. Mine was that it was so small (plus, we have to indeed look at who's answering that they're tea partiers, at that point the selection is not that random anymore, maybe that explains the non-zero correlation).
"Also, if one expects to see a decent-size correlation & finds there isn't one, that is interesting". You're right, but unfortunately a very small correlation is pretty hard to be taken as a starting point towards an even more interesting conclusion.

November 16, 2013 | Unregistered Commenterranjix

"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'"
- Isaac Asimov

November 16, 2013 | Unregistered CommenterEric Buhrer

I'm a 'tea party' person. And I have most of my teeth. I regularly prescribe medicine to people I call patients. They hope I know something about science. Love ya, wouldn't want to be ya!

December 17, 2013 | Unregistered Commentertonymann

And the mail keeps coming in! It's telling that this article has stirred up such a vigorous and extended response. Apparently a lot of conservative-minded folks feel slighted by the author's self-confessedly pre-judiced opinions toward them, and are eager to help him augment his information base. Is he listening?

December 17, 2013 | Unregistered CommenterBrooks Alexander

It is always entertaining to see "science" people selectively doing flips & twists dismissing new data they are uncomfotable with. As if scientists should ever be "uncomfortable" with data for any reason other than it's poor quality - if that were the case. The same phenomena occurred back when the Big Bang filled so many academics with mysterious, deep angst.
Did anyone else notice that the ONLY thing anyone here found truly problematic in Dr Kahan's conclusions was that ONE single correlation?
The level of ideological insulation many academics maintain is both dangerous & truly frightening. And the ease with which the good doctor admits cooly that he knows NO ONE who identifies with the TP, that was truly revealing. That speaks to the degree to which many of our most brilliant end up having absolutely NO idea what's going on around them.

December 18, 2013 | Unregistered CommenterC Donald,RN

Regarding your findings for Tea Party members/non-Tea Party members and science comprehension, has anyone teased out the affect of the sample size difference? The non group was so much larger that I wonder if it was influenced by regression to the mean and that resulted in the apparent difference between the groups. I would suggest that you randomly select a number of the non subjects equal in number to the Tea Party subjects and run it again.


December 18, 2013 | Unregistered CommenterRobert McDonald, Ph.D.

Interesting theory. The political party that's best at budget math can do science, too. Whoulda thunk it?

December 18, 2013 | Unregistered CommenterLibertyAtStake

I believe a statistical flaw occurred in these analyses.

I assume that tea party members are predominantly conservative republicans.

If this assumption is true then the histogram for the tea party members (grey bars) come from the previous histogram of the conservative republicans (red bars)

Looking at these carefully the tea party histogram is shifted slightly to the right (higher science knowledge) indicating the tea party members are more nowledge than non-tea party conservaties.

In the histogram the non-tea party members (light blue bars) is a combination of the conservative republicans with lower science knowledge (from the non-tea party group) + the liberals with overall higher science knowledge.

The result of combining these two groups (the lower tail of the conservative republicans and the higher knowledge group liberals) is that the mean is shifted downwards.

The analysis as done compares tea party to ALL others when perhaps we should compare the tea pqrty to liberals or compare all three groups -- tea party, liberals, and conservatives not in tea party.

December 19, 2013 | Unregistered CommenterBill Shannon


you are right that the tea party members are being compared w/ non-tea party members both republican & democrat.

It's a mere description of the data, so I don't see how it can be "flawed."

If I were drawing an inference of some kind that those observations didn't support, that could be "flawed," sure.

The inference you want to draw presumably is that TPers have lower comprehension scores than Democrats. Yes, it would make sense to compare TP to Dems only to figure that out. I haven't done it but am willing to bet you $10,000 the differences -- just like all other ideology-science-comprehension differences have always turned out to be.

Don't you find this "whose is bigger" game boring & childish?

Plus remember: the prize for the side who "wins" the "greater science comprehension score sweepstakes" is being revealed to be the most biased in its reasoning about politically disputed issues that turn on decision-relevant science. Woo hoo!

December 20, 2013 | Registered CommenterDan Kahan


I call your "regression to the mean" & raise you a "central limit theorem."

Or more simply put, correlation is tailor-made for figuring out whether the frequency of some characteristic varies in relation to another characteristic that varies in frequency in the population. Your proposal to throw out a bunch of observations would achieve nothing except to limit the statistical power of the analysis

December 20, 2013 | Unregistered Commenterdmk38

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>