follow CCP

Recent blog entries
« Lecture on Science of Science Communication at Penn State (lecture slides) | Main | A fragment: The concept of the science commmunication environment »
Tuesday
Oct152013

Some data on education, religiosity, ideology, and science comprehension

No, this blog post is not a federally funded study. It's neither "federally funded" nor a "study"! Doesn't it bug you that our hard-earned tax dollars pay the salary of a federal bureaucrat too lazy to figure out simple facts like this?

Because the "asymmetry thesis" just won't leave me alone, I decided it would be sort of interesting to see what the relationship was between a "science comprehension" scale I've been developing and political outlooks.

The "science comprehension" measure is a composite of 11 items from the National Science Foundation's "Science Indicators" battery, the standard measure of "science literacy" used in public opinion studies (including comparative ones), plus 10 items from an extended version of the Cognitive Reflection Test, which is normally considered the best measure of the disposition to engage in conscious, effortful information processing ("System 2") as opposed to intuitive, heuristic processing ("System 1").  

The items scale well together (α= 0.81) and can be understood to measure a disposition that combines substantive science knowledge with a disposition to use critical reasoning skills of the sort necessary to make valid inferences from observation. We used a version of a scale like this--one combining the NSF science literacy battery with numeracy--in our study of how science comprehension magnifies cultural polarization over climate change and nuclear power.

Although the scale is designed to (and does) measure a science-comprehension aptitude that doesn't reduce simply to level of education, one would expect it to correlate reasonably strongly with education and it does (r = 0.36, p < .01). The practical significance of the impact education makes to science comprehension so measured can be grasped pretty readily, I think, when the performance of those who have and who haven't graduated from college is graphically displayed in a pair of overlaid histograms:

The respondents, btw, consisted of a large, nationally representative sample of U.S. adults recruited to participate in a study of vaccine risk perceptions that was administered this summer (the data from that are coming soon!).

Both science literacy and CRT have been shown to correlate negatively with religiosity. And there is, in turns out, a modest negative correlation (r = -0.26, p < 0.01) between the composite science comprehension measure and a religiosity scale formed by aggregating church attendance, frequency of prayer, and self-reported "importance of God" in the respondents' lives.

I frankly don't think that that's a very big deal. There are plenty of highly religious folks who have a high science comprehension score, and plenty of secular ones who don't.  When it comes to conflict over decision-relevant science, it is likely to be more instructive to consider how religiosity and science comprehension interact, something I've explored previously.

Now, what about politics?

Proponents of the "asymmetry thesis" tend to emphasize the existence of a negative correlation between conservative political outlooks and various self-report measures of cognitive style--ones that feature items such as  "thinking is not my idea of fun" & "the notion of thinking abstractly is appealing to me." 

These sorts of self-report measures predict vulnerability to one or another reasoning bias less powerfully than CRT and numeracy, and my sense is that they are falling out of favor in cognitive psychology. 

In my paper, Ideology, Motivated Reasoning, and Cognitive Reflection, I found that the Cogntive Reflection Test did not meaningfully correlate with left-right political outlooks.

In this dataset, I found that there is a small correlation (r = -0.05, p = 0.03) between the science comprehension measure and a left-right political outlook measure, Conservrepub, which aggregates liberal-conservative ideology and party self-identification. The sign of the correlation indicates that science comprehension decreases as political outlooks move in the rightward direction--i.e., the more "liberal" and "Democrat," the more science comprehending.

Do you think this helps explain conflicts over climate change or other forms of decision-relevant science? I don't.

But if you do, then maybe you'll find this interesting.  The dataset happened to have an item in it that asked respondents if they considered themselves "part of the Tea Party movement." Nineteen percent said yes.

It turns out that there is about as strong a correlation between scores on the science comprehension scale and identifying with the Tea Party as there is between scores on the science comprehension scale and Conservrepub.  

Except that it has the opposite sign: that is, identifying with the Tea Party correlates positively (r = 0.05, p = 0.05) with scores on the science comprehension measure:

Again, the relationship is trivially small, and can't possibly be contributing in any way to the ferocious conflicts over decision-relevant science that we are experiencing.

I've got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I'd be shown a modest negative correlation between identifying with the Tea Party and science comprehension.

But then again, I don't know a single person who identifies with the Tea Party.  All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).  

I'm a little embarrassed, but mainly I'm just glad that I no longer hold this particular mistaken view.

Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for. I just no longer assume that the people who happen to hold those values are less likely than people who share my political outlooks to have acquired the sorts of knowledge and dispositions that a decent science comprehension scale measures.

I'll now be much less surprised, too, if it turns out that someone I meet at, say, the Museum of Science in Boston, or the Chabot Space and Science Museum in Oakland, or the Museum of Science and Industry in Chicago is part of the 20% (geez-- I must know some of them) who would answer "yes" when asked if he or she identifies with the Tea Party.  If the person is there, then it will almost certainly be the case that that he or she & I will agree on how cool the stuff is at the museum, even if we don't agree about many other matters of consequence.

Next time I collect data, too, I won't be surprised at all if the correlations between science comprehension and political ideology or identification with the Tea Party movement disappear or flip their signs.  These effects are trivially small, & if I sample 2000+ people it's pretty likely any discrepancy I see will be "statistically significant"--which has precious little to do with "practically significant."

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (1)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    Will he now do a study to document the cognitive biases of the left, their tendency to demean as nearly subhuman all those who disagree with them?

Reader Comments (284)

Next time, use the following measures:

1., Instead of a test of factual knowledge about science, use a test of understanding of how and why science works. I'm not sure that such a test exists.

2. Instead of CRT and numeracy - which are also heavily influenced by factual knowledge about (e.g.) algebra - use a measure of actively open-minded thinking (AOT). Even an 8-item measure of beliefs about how thinking should be conducted is quite useful for predicting many things.

3. Instead of your religion measure, use one that assesses the belief that morality comes from God and cannot be understood through human thought. This correlates very highly (negatively) with AOT.

I predict that you will find that the low-AOT, low belief in the power of thought, and lack of understanding of how science works will correlate with each other, with politics, and with resistance to counter-evidence. (The last will be difficult to test because you need to equate strength of belief, and high-AOT people will be on the average less certain about their beliefs at the outset.)

October 15, 2013 | Unregistered CommenterJon Baron

@JB:

will do!

I assume too that you think people who score high on this measure will be resistant to politically motivated reasoning.

I am also interested in measures of curiosity ...

October 15, 2013 | Registered CommenterDan Kahan

Hey Dan,

I have my doubts about the validity of these measures of science comprehension (I mean what to they measure, really?), but I wonder what would have happened with this analysis had the study been conducted over a period of time. My guess is that actually, such a longitudinal design might be informative about "motivated reasoning." We've discussed a bit before how easy it is to look at a cross-sectional analysis and overevaluate how much you're learning about a dynamic mechanism.

What if there were a trend that showed that over time - as the number of people who identify with the Tea Party have dropped, and as the popularity of the movement has waned - the correlation between identification with the Partiers and science comprehension changed in magnitude? Say the study compared sampling from when the Tea Party first coalesced, then the summer 2012 time period when you collected your sample, and then more currently. Would that be meaningful?

My reaction to this research is generally: The more longitudinal the more informative. I don't think that you can find a meaningful relationship between what someone thinks and how someone thinks by using political ideology as a yardstick. But I do think that it might be useful to compare how someone identifies, politically, over time, to "how" they think. How are people who strongly identify with the Tea Party in October of 2013 different from those who identified with the Tea Party in October of 2010, or October of 2012? What can we understand about why some people have doubled-down on their dedication to the Tea Party rhetoric while others have moderated in their support for the Partiers?

This goes back to something that I've felt for a while: that it isn't that people who are more (scientifically or otherwise) literate who are more invested in using their knowledge to confirm their biases, but that people who are more invested in confirming their biases have more motivation to become more (scientifically, or otherwise) literate.

October 15, 2013 | Unregistered CommenterJoshua

@Joshua:

but do you doubt them b/c of what they say about the tea party?

I think the NSF science indicators are very inadequate. I've discussed this before..

the only thing that makes me willing to do anyting w/ the NSF science literacy batter is that it does seem to scale well w/ validated measures of critical reasoning dispositions, like CRT & Numeracy. Besides "better" than the NSF science literacy test by itself, I'm not sure "how good" the resulting 'science comprehension' measure is, except that I can see (a) it does a good job at predicting performance on reasoning tasks that indicate a facility with valid inference from observation (like the covariance problem), (b) isn't reducible to education, (c) outperforms education in tasks that have a connection to comprehension of empirical data, and (d) has the sorts of correlations w/ other characteristics that one would expect a "valid" science comprehension measure to have. But I certainly think it is possible to design something that one would have even more confidence in (the amazing thing is how little confidence people ought to have in what pass for standard measures of science understanding).

The longitudinal stuff would be interesting. What is your hypothesis? That the correlation w/ science literacy grows *stronger* as the popularity of a fad-ish "movement" like Tea Party declines? (I thnk the # of people "identifying" is declining; & likely those sticking it out have 'thought it through" more deeply etc, so that's plausible. But flight of those thinking it through or seeing consequences is also plausible-- and would predict opposite sign, I guess, in correlation.)

I sincerely doubt, though, that people who like the Tea Part became more science literate because they wanted to improve their ability to "dig in" and support their positions-- if that is your hypothesis. But I like to be surprised.

October 15, 2013 | Unregistered Commenterdmk38

Dan -


I sincerely doubt, though, that people who like the Tea Part became more science literate because they wanted to improve their ability to "dig in" and support their positions-- if that is your hypothesis. But I like to be surprised.

I think that what would change is the concentration of "scientifically literate" among those who remain Pariters over time, not the scientific literacy of individuals over time. Dependent vs. independent variables?

but do you doubt them b/c of what they say about the tea party?

I'm not sure of the "them" there. The tests? The Tea Partiers? Do I doubt the tests because of what they (the tests) show about Tea Partiers? No, definitely not. My judgement of the tests predate reading your your analysis. If my negative assessment of the tests is due to my motivated reasoning, it isn't because of any correlation with Tea Partiers, but my "motivations" related to being an educator.

My expectation is that there is no meaningful distinction between political affiliation and the degree to which someone displays motivated reasoning, because I see motivated reasoning as an outgrowth of fundamental components of human cognition (we are pattern-finding organisms that construct meaning through patterns) and human psychology (we are fragile beings that need to confirm identity, and often through the identification of "other" and through group affiliation).

So what if the tests show a positive correlation between Partying and science literacy? Does that mean I'm wrong?

Well, even if I don't know what the tests show, they do show something. So it would be important to investigate further what they do actually show. My thinking about "scientific literacy" has to be reconciled with my belief in multiple domains of intelligence. I see people who would fail in tests of "scientific literacy" but who are very skilled in "critical thinking" in domains that they're familiar with and care about. What does that mean?

But if the science literacy tests align with CRT, does that mean that we can postulate that CRT is positively correlated with Partying? And would that be meaningful? Would that mean that because I am anti-Partying, I am not a reflective person? Does it mean that because my "group" is anti-Partier, my "group" comprises unreflective people, relatively, whereas Sarah Palin and Michelle Bachman are statistically more likely to be deep thinkers than my "group" and I? Well, that would suck, wouldn't it?

So let me throw some reflexive motivated reasoning out there.

That is all part of why I think longitudinal data would be important. My guess is that hardcore Partiers are more likely to be "reflective" thinkers, because they are people who are generally more "motivated" to investigate issues associated with their identifications. The harder core, the more they are motivated - but the causality goes [motivation] ====> [literacy]. People more heavily identified with particular affiliations have more motivation to affirm their identifications and then have more motivation to pursue education, to read newspapers, to develop math skills, etc. People who are less identified are more likely to watch American Idol, work on restoring muscle cars, go hunting, etc.

So looking at those examples, I wonder about class as opposed to political affiliation. But hey, that would be problematic too, because I run across people of all classes who strike me as reflective thinkers. In fact, I have a strong "motivation" to disprove that class is associated with reflectivity.

But flight of those thinking it through or seeing consequences is also plausible-- and would predict opposite sign, I guess, in correlation.

Interesting speculation that people who think it through would be more likely to abandon the Tea Party - but I think that hypothesis is actually not consistent with your findings on motivated reasoning: Those who tend to think things through more are more likely to use that proclivity to affirm their identifications. So abandoning the Party would not be the result of thinking it through more, but maybe a less complex dynamic of seeing the impact - over time - of Tea Party philosophy, and determining an adverse outcome. I don't know that assessing outcomes over time is the same thing as "thinking it through." The former is more visceral and emotional and the later is more abstract and theoretical and mental.

Sorry for the rambling. As usual, just see if there's anything interesting and throw out the rest.

October 15, 2013 | Unregistered CommenterJoshua

Well, for starts, thanks to the shutdown, I can't go to the NSF website and check out what might be on the NSF Science Indicators test. But all tests are structured to have an upper limit. They aren't testing the full range of knowledge from kindergarten through graduate school. And tests themselves are rigid in nature and those who respond well to them very well may be fond of such authoritarian environments in other situations as well.

I think it would help to consider the nature of our education system, and how one might ideally want to navigate through it in search of a real, broad based education. Then one would need to administer a test such as the one above with some understanding as to the backgrounds of the students. On an anecdotal basis, I offer the following education outline;

At one end, we have "dates, calculations and facts" memorization based learning. Extreme supporters of this sort of education might use religious based texts, but many advocates of this sort of education use a curriculum as given in the Core Knowledge books written by E.D. Hirsh . This is frequently linked with a math sequence called Saxon math which is based on the curriculum used in Singapore. These curricula are frequently used in public but charter schools and are often popular with authoritarian conservatives although there are many others who like it because they find the general public school offerings to be too weak. If you want your kid to actually know history as a timeline of events and also to be able to add, subtract, multiply and divide, I think that these curricula actually can be good within those limitations. It is equalizing in that it offers every child a set base of standards, one that is laid out so that a reasonably qualified teacher could be expected to teach to such a standard. It does create a framework within which abstract reasoning and creativity are not actively encouraged, and in implementation, in a charter school run by and for conservative parents escaping the liberalizing and inclusive influences of the usual public school offerings, this can be quite stifling, IMHO. But for standardized text taking, this system is excellent.

At the other extreme there is self directed, experiential based learning. I think this is actually highly dependent on the nature of the individual family. I know of "unschoolers" whose kids run wild through the woods. Most parents in my experience get tired of this after a bit and frequently switch to something else. Rudolf Steiner (Waldorf Schools) offer a curriculum that is hands on oriented in early grades and then in many ways open to creativity throughout. But in my observations, these schools in practice tend to an extreme. It may be perfectly ok to not introduce reading until a later age than done in public schools, but the way this is implemented sometimes, IMHO, quashes the interest in reading of a child who is ready earlier. And heaven forbid if your child plays with plastic, rather than properly natural wooden toys. These schools, I believe, attract parents who are actively anti-authoritarian, and this biases such schools towards such things as the anti-vaxx movement, animal rights, or anti-GMO all in what I would loosely describe as "New Age" manners. So in practice, these schools are not likely to be hotbeds of science education.

A different format of experiential learning is offered by Montessori schools. These are hands on, but with much structure behind the scenes by the teachers as to how materials are introduced to an individual child. This educational method is also very oriented to the individual as opposed to the group. Many of the pedagogical materials are very designed to teach such things as basic spacial or mathematical principles, and the teachers are trained to introduce materials as the child's physical and mental abilities are ready to handle them. These are not called toys, and what the children are doing, usually in a very individual way is called work, not play. In a good setting, the kids don't know that, and have a great time. In more poorly run systems this can be quite dogmatic.

Next in line, generally within the public school system are methods like "Whole Language" or "Connected Math". These tend to advocate an orientation that is much more oriented towards reasoning and creative thinking, and not rote memorization. In practice, however, I think that children can't really do much without the proper tool kit. So whole language might mean handing them a journal in which they could draw or attempt to write. Math might be such things as looking at purchasing fish to put in a fish bowl and spending great amounts of time drawing said fish. In my opinion, much about this sort of curriculum, if badly implemented, leads to situation in which success is most likely to happen for children who have educational inputs beyond the classroom. When coupled with elementary school teachers, who may not be all that strong in math skills themselves, this tendency to draw pictures rather than learn facts gets aggravated. And the mechanisms by which the authors of the texts thought that they were structuring a discovery of new principles may be lost on the teachers themselves. But with the right teachers these methods are very useful. In general though, I think these tend to be coupled with teacher belief systems that say that testing is bad for children. This is probably aggravated by the fact that these classroom teachers are frequently subject to big personal career pressures using the worst forms of standardized tests.

At the high school level, I believe that the International Baccalaureate program offers a well thought out, well rounded educational program. But this is strictly oriented towards already high achieving, college bound students and does not greatly impact even a sponsoring high schools achievement levels for other students.

At any rate, the above is a long winded way of saying that I think that the structure of Core Knowledge/Saxon math would create winners in many testing environments, although not necessarily in creating a citizenry capable of complex decision making and abstract reasoning.

From the perspective of parents (or societies) trying to navigate and maximize educational outcomes, the problem is that our tribal identities make it difficult to come up with a little bit of all of the above type solutions.

October 15, 2013 | Unregistered CommenterGaythia Weis

@Gaythia-- I noticed that about the NSF site, but figured for sure they'd have the govt up & running by noon ... go figure!

try: this.

And be prepared to be underwhelmed. The CRT questions are *much* more discriminating of the aptitude that the two togeteher are measuring, although science literacy does add something.

October 15, 2013 | Registered CommenterDan Kahan

Gaythia -

FYI - I trained (in Italy) as a Montessori teacher, and taught in that system for a number of years.

In my subseqent years of studying educational phiilosophy and methodolog, epistemology, and developmental psychology, and working as an educator in elementary, middle, and high schools, community colleges and university settings and welfare-to-work programs and adult basic education and workplace training initiatives, I have not found a discrete sysem that I think is better adapted to promote creative thinking in accordance with cognitive development (I know that sounds like I"m tooting my own horn there, but my point is to say that although my perspective is obviously subjective, I have worked in a lot of different educational contexts). Specific to kids, and as a fan of Piaget's work, and although Piaget criticized Montessori education in that it reflected a desgin superimposed onto a child's learning, I think that Montessori methodology most closely aligns (of the educational paradigms I've seen) with the fundamental truths about epistemology that Piaget revealed. But...

". In more poorly run systems this can be quite dogmatic."

This is absolutely true, IMO. There is an intrinsic ridigidy to Montessori methodology (she was a product of her times, and, for example, thought that abstract art was a abomination), and like Stiener schools, when they are run by "true beleivers" - they can be insufficiently flexible. But that is true for any educational methodology, isn't it? A key factor in successful educational paradigms is that they can be shifted to meet the individual needs of any given student. And IMO, in the hands of a flexible teacher who has a diverse bag of tricks, there are elements of Montessori methodology that are very powerful and effective. And I feel that there are key elements to Montessorian philosophy (e.g. that learning happens not through "instruction" but through intrinsically motivated exploration in didactic environments) that serve as a valuable guide for educators at any level.

So I would be interested in knowing whether you have criticisms of Montessori that really are unique to that system - not something more generic like saying that it doesn't work well if the practitioners are too rigid.

October 15, 2013 | Unregistered CommenterJoshua

"But then again, I don't know a single person who identifies with the Tea Party."

If you did, they would probably keep it quiet! :-)

I'm not an American, so I can't really comment with too much authority. But I know a fair number of Tea Party types via the internet, and I quite like them and their policies, so far as I can distinguish them. Does that count as "identifying with"? Would you say that you "knew" me? :-)

"Of course, I still subscribe to my various political and moral assessments--all very negative-- of what I understand the "Tea Party movement" to stand for."

But you do recognise that your view is at least partly a product of your ideological position and the media you read, rather than necessarily absolute and unarguable truth, which is good.

I have, as you can imagine, a rather different ideological viewpoint. The basic belief is that you can't spend more than you earn indefinitely, that you can't afford everything you want, that simply running up a huge debt will lead to disaster, that they're taxing people more than they should to pay for it, and that you need to balance your national budget and encourage growth by shrinking government spending massively, and cutting taxes.

The main problem, as in all cases where people spend far more than they earn or can possibly pay back, is that their opponents are firmly convinced that they cannot possibly survive on any less. (And for many, that's probably true.) The subsidiary problem, as in all cases where people spend other people's money, is that their opponents don't think it fair that they should have to cut back when there is still anything left to take. (Which to be fair there probably is.) They seem to regard it as rich people's attempt to shirk their duty to give everything they have to the poor. While Tea Partiers believe that if there's no personal benefit to creating wealth, people won't.

There is both a moral dimension and an economic dimension to the eternal battle between Left and Right. The former is about how we would like the world to be. The latter is about how it is, and what can be made to work. The Left give priority to the moral dimension, and try to force the economics to fit. The Right argues that it doesn't work, and that when you chase down all the reasons why, you find that the market imposes some moral values of its own - a sort of merciless justice - in which the Left's methods are in turn seen as immoral. But the right do sympathise with many of the Left's aspirations regarding helping the poor out of poverty, which is something the Left rarely seem to understand. They assume that because the Right don't like their methods, that they're therefore callous and uncaring of their goals.

When I try to explain the Left-Right argument to the young, I sometimes use a classroom example. There are some kids in the class who get top marks, and a lot of kids who don't. The teacher wants *all* the kids to get top marks, so they can go on to be doctors and inventors and so on for the benefit of all. (Or at least, not stuck being burger-flippers.) Therefore he takes marks from the bright kids, and gives them to the thick ones. What happens? Does it work?

Kids may often be resentful of the fact that others get higher marks than they do, despite all their efforts, but they do understand that the reason for giving marks is to encourage students to learn, and that it is really the learning that is the goal, and the point of the entire exercise, not the marks. If you redistribute the marks, that doesn't mean there's any more learning to go round, and the bright kids no longer see any point in making the effort. That doesn't mean that by marking kids homework strictly you don't want them to succeed, or that you have no sympathy with their deep unhappiness about that F-. But simply handing out higher marks is the wrong way to deal with it. You just get grade inflation.

You might not agree with it, of course. You almost certainly don't. But it's not a stupid viewpoint. Just a different one.

That's just my view, of course, and it is at least partly the result of the people I have listened to and the books and articles that I have read. Where people stand in this great debate has nothing to do with intelligence. The moral dimension is pretty much independent, and the economic dimension complicated enough that a great many highly intelligent people on both sides have presented elaborate arguments in favour of either view. Not even the economists agree.

So I don't think it's at all surprising that there's no particular correlation when you ask ordinary members of the general public. The Tea Party is a reaction to the economic times. Whether you like them or not depends to a large extent on your economic situation.

October 15, 2013 | Unregistered CommenterNiV

"Whether you like them or not depends to a large extent on your economic situation."

???

In what way? Which economic situations correlate with which views on the Tea Party?
Evidence?

October 15, 2013 | Unregistered CommenterJoshua

"Kids may often be resentful of the fact that others get higher marks than they do, despite all their efforts, but they do understand that the reason for giving marks is to encourage students to learn, and that it is really the learning that is the goal, and the point of the entire exercise, not the marks. If you redistribute the marks, that doesn't mean there's any more learning to go round, and the bright kids no longer see any point in making the effort."

Hmmm. Where to start?

Much of my educational experience was working, specifically, with kids who don't get good grades. I would say that in general, very few of them see the goal of giving marks as encouraging students to learn. IMO, they see the point of grades as to make them feel bad about themselves, or to tell them that they are stupid, or lazy.

Studies show that largely, our educational system, one that relies heavily on grades as a "motivational" method, produces students with a passive attitude about learning.

I would say that most kids who get good grades (I've worked a lot with them also) think that the point of getting good marks is to get good marks. Many of them don't particularly care about learning, and when you take away the system by which they do what they are told to do in order to get good marks, they are left without any real sense of themselves as learners. This is, largely, why many students undergo such a dramatic shift as they advance through educational levels, where intrinsic motivation for learning rather than extrinsic motivation, becomes more important. It is one reason why so many students struggle so much when there is no longer someone there instructing them in such detailed ways about which hoops to jump through to demonstrate the behaviors that nets them good grades.

And the following part of that comment is the most incorrect, IMO.

"...and the bright kids no longer see any point in making the effort."

Well, of course it does depend on how you define "bright," but IMO, what I would call the "brightest" students are those who care the least about grades. Keep in mind, my determination of who is more "bright" and who is less "bright" is to a some degree independent of who gets good grades (although they do sometimes correlate). Yes, those who get good grades are the most motivated to get good grades. Why wouldn't they be? Who doesn't like positive reinforcement? Who doesn't like being told that they are superior?

Implicit in your whole discussion of grades is that they are an effective educational tool. What emperical evidence do you base that opinion on?

October 15, 2013 | Unregistered CommenterJoshua

Joshua,

If you're funded by the tax-payer, you tend to want to keep them paying. If you're on the edge of going out of business because you're paying far more to the government than you're getting back, you often don't.

October 15, 2013 | Unregistered CommenterNiV

"Much of my educational experience was working, specifically, with kids who don't get good grades. I would say that in general, very few of them see the goal of giving marks as encouraging students to learn. IMO, they see the point of grades as to make them feel bad about themselves, or to tell them that they are stupid, or lazy."

Ah! There you go, you see! Different ideologies!

"I would say that most kids who get good grades (I've worked a lot with them also) think that the point of getting good marks is to get good marks."

By default, maybe, given the way some kids are taught nowadays. You do have to explain it to them. But it's not a difficult point to understand.

"Many of them don't particularly care about learning, and when you take away the system by which they do what they are told to do in order to get good marks, they are left without any real sense of themselves as learners."

Which is to say, if you take away the grades, they no longer see any point in learning. Right?

October 15, 2013 | Unregistered CommenterNiV

By default, maybe, given the way some kids are taught nowadays. You do have to explain it to them. But it's not a difficult point to understand.

Kids are not fools. Grades are the engine that drive motivation within our educational paradigm - "explaining" to them that they shouldn't be motivated by grades doesn't work if grades are the mechanism by which they are evaluated.

The basic problem with that paradigm is that students pass over the responsibility for the evaluation of their learning process to someone else. That is counterproductive to them becoming engaged in evaluating their own learning. Seeking out feedback is important, but that isn't what happens, for the most part, with grades.

To give a generalization, try giving students, who generally get good grades, a less than excellent grade on a paper along with extensive feedback on why they didn't get a high grade. See where their focus is (again, generally speaking0 : (1) on processing the information as to why they didn't get the high grade and incorporating that into future efforts or (2) focus on the grade itself, the "fairness" of the grade, the implications of the grade to their GPA, their application to medical school, their funding, their parents' reaction, etc. What to you think might happen if you just try changing that grade and seeing whether or not they just move on without ever paying one iota further of the feedback that you gave them, or whether the grade in and of itself was what was primarily important.

The situation is even worse with students who don't generally get good grades. If they take a particular test and get a good grade, they attribute that result to "luck," or that the test must have been easy. The whole interaction is externalized. (again, generalizing).

IMO, "meta-cognition" is the key - the extent to which a learner assumes "executive control" over their own learning process - not the extent to which they hand over the responsibility of assessing their performance to someone else. If you want to use grades, ask them to grade themselves. It's far more educational.

"Which is to say, if you take away the grades, they no longer see any point in learning. Right?"

So then giving them grades makes them see the point in learning? In other words, the point in learning is to get good grades? Unfortunately, that perspective is all too prominent.

Actually, the "bright" students learn for intrinsic reasons. They learn because they enjoy it. They find it enriching. They find it consistent with their values, and the values of their social environment. Actually, they do see a point in learning beyond getting good grades, which is why adding to the paradigm of conflating good grades with learning is counterproductive - because it reinforces a different message.

If students think that the point of learning is to get good grades, then they don't understand learning. If you take the grades away and they don't see a reason to learn, then they don't understand learning.

It's a harsh analogy, but if you take away the rewards from the lab rats, the behaviors diminish. Why treat students as if they are lab rats - and should be motivated by external rewards.

Again, all generalities, of course....

It is a misconception that the best students will no longer be motivated to learn if they don't get good grades. You take away the grades and the intrinsic motivation remains behind.

October 15, 2013 | Unregistered CommenterJoshua

NiV -

If you're funded by the tax-payer, you tend to want to keep them paying. If you're on the edge of going out of business because you're paying far more to the government than you're getting back, you often don't.

Do you have any idea of the % of Republican voters and people who vote for Tea Party candidates get some form of federal support? What is the % of small business people who support Dems?

So, do tell, on what evidence to you base your assertion of correlation between economic conditions and Tea Party support? Apparently you think there is something dramatic. I say that the demographics of those who identify with the Tea Party are pretty similar to the demographics of everyone else, except to primarily in race/ethnicity, and to a lesser extent, age (they are slightly older and hence more likely to be on Medicare), and gender (more likely to be male).

But I'm open to evidence otherwise. And specifically demographics related to economic status.

October 15, 2013 | Unregistered CommenterJoshua

"Grades are the engine that drive motivation within our educational paradigm - "explaining" to them that they shouldn't be motivated by grades doesn't work if grades are the mechanism by which they are evaluated."

You are confusing "purpose" with "motivation".

Kids are undoubtedly *motivated* to get higher grades - both for their immediate benefits, and for the longer term ones. But the *purpose* in giving grades is to get them to learn by so motivating them.

In exactly the same way, the purpose of paid employment is to get people to create the wealth by which all our lives are collectively made better. But few people work *in order to* create wealth. (Some do, but generally those who care the least about money.) They work instead for personal advantage - for money, or the things money can buy. The purpose of money is to motivate people to create wealth. Paying them money without creating wealth defeats the entire object, like giving high grades without the student having learned anything.

But as I said, this is a matter of ideological viewpoint, on which I'm frankly not expecting us to ever agree. Obviously, leaning as you do to the left, you're going to think I'm *wrong*. My only point was that it's not the sort of difference that should lead one to think believers in it are stupid, or any less intelligent/educated/critical-thinking/reflective. It's just a different way of looking at the things.

October 15, 2013 | Unregistered CommenterNiV

"But the *purpose* in giving grades is to get them to learn by so motivating them."</>

The purpose in giving them grades is to motivated them to get good grades. It works against their development of intrinsic motivation to learn, because the activity of getting good grades displaces the activity of learning.

"In exactly the same way, the purpose of paid employment is to get people to create the wealth by which all our lives are collectively made better."

Interesting that you'd go there - because our predominant educational paradigm is based on the hierarchical structure of the workplace, where objective is to be rewarded for doing what you are told. It was a good system for factories in decades past. For educational institutions and the modern workplace, not so much.

And it is interesting because there is a common misconception about motivation in the workplace that is similar in nature. There is a lot of evidence out there about what motivates workers most strongly. Is it financial reward, or is it more intrinsic returns, such as autonomy, influence, respect, skill-development and expertise (not that I'm arguing that those rewards need be mutually exclusive)?

"Obviously, leaning as you do to the left, you're going to think I'm *wrong*."

Well, it isn't only that I think that you are wrong. It is that I think you are stating viewpoints for which you have no evidence, and certainly no empirical evidence. That's why I keep asking you for such. Maybe you aren't wrong. So where is the evidence on which you base your conclusions? Where is the evidence that quality learning takes place because of students being handed grades, as opposed to what I am arguing, that quality learning takes place because of intrinsic and not extrinsic reward.

I happen to know that there is a wide body of literature to support my viewpoint. I don't know of one to support yours. So maybe you could point me to some? It would be interesting reading.

You might be interested in reading what J. Scott Armstrong has to say about some of the issues we're discussing. He's a hardcore libertarian and climate "skeptic." He has written some interesting stuff about our predominant educational paradigm. It has some fairly extensively validated empirical evidence (although, not surprisingly, I disagree strongly with some of his main conclusions - primarily about the return on societal investment in education). Maybe after reading what he's written, you will question your attribution of our differences on this issue to our political orientation. Maybe then you will have to question whether our differences are based on experience and familiarity with the evidence. That would be kind of ironic, wouldn't it?

October 15, 2013 | Unregistered CommenterJoshua

"because our predominant educational paradigm is based on the hierarchical structure of the workplace, where objective is to be rewarded for doing what you are told."

That's not the case where I work. And it wasn't the case when I was educated, either.

But I guess different educational systems (and educators) work differently. It takes all sorts.

"So where is the evidence on which you base your conclusions?"

I already told you where I got my information from. But you're missing the point again. What do you think the purpose of valuing and rewarding higher grades is, if not to get students to learn?

October 15, 2013 | Unregistered CommenterNiV

But I guess different educational systems (and educators) work differently. It takes all sorts.
'

I'm talking about the predominant educational paradigm, the history of which is rather easily traceable - and tied to the history of the hierarchical work environment as developed during the industrial age.

What do you think the purpose of valuing and rewarding higher grades is, if not to get students to learn?

I think that the intent, ostensibly, is to help students learn. The question, then, is how well do they work towards that goal as opposed to other educational methodologies, and I'd say not very well (look at the studies that show how over the course of their academic careers, on average students become more passive in their attitudes towards their education). And the question is what do they do, functionally - which I think clearly includes another aspect of the dominant educational paradigm, whereby schools serve as social sorting mechanism to perpetuate status quo. So "purpose" gets a bit complicated.

It isn't coincidence that grades (and educational achievement) are so well correlated with SES. Of course, the explanatory mechanism for that correlation is very complex, but I have worked in schools where you could basically identify those who received poorer grades on the basis of their SES.

The model is not that different from a antiquated (although still commonly found) model of an industrial workplace, where it was important to maintain a pyramid-like status structure. So since our schools are largely based on a system aimed at preparing students to fulfill a role in that type of structure, it isn't surprising that they would manifest and maintain a similar structural outcome. Students largely succeed on the basis of how well they do what they are told to do, how well they follow instructions. Divergent thinking - a key element in "critical thinking," is not prioritized.

I'm not trying to be conspiratorial here, or suggest ill-will on educational professionals, or to make a uniform characterization of schools and teachers, but consider the following as an example of what I think is all too common.

I was once working with a high school student who had substance abuse and behavioral problems. He was a very "bright" kid, who would read esoteric philosophy and write poetry on his own time out of an intrinsic motivation. For one class he was given an assignment to write on the same topic as every other student in his class. He wrote (what I considered to be) a brilliant essay on a related but different topic. The teacher gave him a failing grade, because he didn't follow directions and because, the thinking was, if he wasn't held accountable for following directions, how would he learn what he needed to learn to be prepared for higher education?

Ignoring a potential mistake in priority of goals for that student (I would argue that one more poor grade, in a school career-long history of getting poor grades for failing to follow directions and for instead following his intrinsic intellectual interests, would certainly have no beneficial impact on him whereas positive feedback on the product of his intellectual engagement might, potentially, make a small bit of a positive difference), this teacher was also focused on her role within social sorting mechanism. Part of her rationale was that if she rewarded that student for failing to follow instructions, that would be a discouragement to the other students who were rewarded for following instructions. She felt that if he didn't learn how to follow directions in her class, he wouldn't be properly prepared for life beyond.

Her motivation was to motivate students to learn, but she was functionally sorting students out not on the basis of their individual intellectual development, but on the basis of how well they fit into a defined role where individuality was, in fact, a negative factor. She actually felt that her role was to sort out the "good" students (those who matched some idealized and generic model) from the "bad" students, and if that didn't happen, essentially, chaos would follow. Any failure was his failure, she felt, and he needed to be held accountable. She couldn't conceive that part of the failure was to adapt the educational environment to one that would maximize his intellectual development. And grades, a unilateral exercise of power, and which were essentially based not on anything specific about that student himself, but on comparing that student to others and rewarding or punishing him on the basis of how well he conformed to the "norm," were the tool that she used to enforce a stifling and limiting social standard.

Compare giving a student a grade, most likely a student who has a long history of getting essentially the same level of grades, with giving a student criterion referenced feedback about what material they have successfully mastered and what material they need to master next in order to advance their intellectual development. Compare giving a grade (one which is probably the same level of grade that they have always gotten in their educational career) to a student for following instructions well to telling that student that they nee to spend time thinking about and developing strategies for how to learn something, evaluating the strategies they used, modifying those strategies to correct for deficiencies, and then sitting down with you to discuss the strengths and weaknesses of their approach.

But seriously, since you think my views can be attributed to my different political outlook, if you have some time read what J. Scott Armstrong has to say. Let me know what you think and if you can see a way to differentiate my perspective from his and explain how you agree with him because of political outlook but disagree with me because of political outlook.

Or, you could try giving me some evidence, something empirical, that supports your argument.

October 15, 2013 | Unregistered CommenterJoshua

Here you go, NiV, in case you're interested:

http://marketing.wharton.upenn.edu/files/?whdmsaction=public:main.file&fileID=3459

I mean there's much better stuff out there (he doesn't seem to have much background in pedagogy), but it'll due as an overview. I highly recommend the work of Alfie Kohn (who Armstrong references quite often) w/r/t the effectiveness of grades (and a competitive paradigm of education). Alfie is a friend from a while back that I've fallen out of touch with. Can you predict Alfie's politics? : - )

October 15, 2013 | Unregistered CommenterJoshua

"I think that the intent, ostensibly, is to help students learn."

Thanks. That's all I was after.

" The teacher gave him a failing grade, because he didn't follow directions and because, the thinking was, if he wasn't held accountable for following directions, how would he learn what he needed to learn to be prepared for higher education?"

I would have done the same. It's not just a preparation for higher education, but for life. A brilliant answer to the wrong question doesn't get you paid. However, I would have also made it clear to the student that the essay was very good, would have succeeded if that had been the question, and told them to hold on to it for next week when I'd make sure the question fit what they were writing about. I'd also spend some time explaining to them about *why* it was necessary to answer the question asked.

I'll have a look at Armstrong.

October 16, 2013 | Unregistered CommenterNiV

"I fully expected I'd be shown a modest negative correlation between identifying with the Tea Party and science comprehension."

I would have expected that too.

Interesting. Why do you think you and I had that prejudice? Perhaps it can be explained by the cultural cognition theory. Perhaps those of us immersed in a liberal university background are predisposed to regard Tea Party members as a bunch of thickos.

October 16, 2013 | Unregistered CommenterPaul Matthews

Good job being open-minded, professor. A NYTimes article reported a poll in 2010 that showed tea party supporters are more college educated than the general population: http://www.nytimes.com/2010/04/15/us/politics/15poll.html?_r=0

October 16, 2013 | Unregistered CommenterKyle Becker

"I would have done the same. It's not just a preparation for higher education, but for life."

Interesting. Isn't engaging the student with their intellectual development, and rewarding them for exploring their intellectual interests, "preparation for life?" You have created a false dichotomy. Handing out grades may, in some contexts, be useful for "teaching lessons about life," but that context is important. It doesn't work that way in a blanket fashion, and often their is either no significant impact (and opportunity cost if it becomes the main vehicle for feedback and if it becomes the "currency" of in your educational paradigm), or counterproductive.

Anyway, perhaps you think the job of a teacher is to teach a student about life, and that you can do so by giving them a grade, for 12 years day after day, and that one grade in 12 years will make something of a difference. Maybe you think that the way to prepare a student for life is not to prioritize their intellectual development, but to show them that to succeed the most, they need to prioritize following instructions over exploring their creativity.

I disagree, fundamentally. I consider the job of a teacher is to help a student become aware of themselves as learners, to be activated in managing their own learning process, and knowledgeable about how to do that so as to maximize their strengths and correct for their weaknesses (be meta-cognitive, if you will). To, regardless of their ability or desire to succeed by virtue of following directions, know how to engage with the world intellectually and to develop skills in critical thinking (so as to better be able to decide when they want to follow directions and when they want to move in other directions).

But even if we accept those different orientations, the question is do you really think that you're going to teach a student in such a situation that following directions is how to be "prepared" for life? Do you really think that a student who has a history of drug problems and behavioral problems is going to receive the message that you want to give them with a failing grade? That in any measure they're going to say, "Oh, another failing grade in a long line of failing grades, because I didn't follow directions, and instead explored a topic based on what interested me as an individual, and because of that one failing grade, I now know that I won't get anywhere unless I start following drections, and investing my time and energy in activities that are of no intrinsic value to me?." Do you think that the other students, who have a history of getting good grades from following directions is going to see another student who gets good grades for engaging their interests and intellect, and then say to themselves: "OK, now I know that following directions is unimportant, and I'll try to succeed in life by not following directions?"

And what is really interesting about your position is that now you seem to be saying that it isn't quite that motivating the student to learn that is the focus of your grading, but that the grade is a vehicle for teaching the student that following directions is how he will succeed in life. The point is to follow directions, independent of their learning how to manage their own intellectual development, and then to be judged on the basis of how well they followed directions. Does a student "learn" more if they follow directions and write on a topic that they are uninterested in, and not engaged in, than if they explore a work deeply that they care about? You might find it interesting to see what Armstrong has to say about that.

In short, the message to the student is that the point of grading is to follow directions so that he can get a good grade. And unfortunately, all too much that is what students care about, and some of the worst in that regard are the students who over the course of their academic career have gotten good grades. Keep in mind that the context that Armstrong has used for evaluating methodology is teaching at Wharton (I have also worked there, but I suspect that I have a much wider experiential context for evaluating methodology than does Armstrong). Wharton students are, I would say in a certain sense, among the worst in the sense of prioritizing the grade itself over the learning that takes place in the educational environment. I did say that the "brightest" students tend to be those who care about grades the least, and many Wharton students are certainly very "bright." But it is a special case, because many students see Wharton as primarily a place to network and to gain credentials for the sake of gaining credentials - because the real learning related to their field of study takes place in the field and not in the classroom.

October 16, 2013 | Unregistered CommenterJoshua

On the composition of the Tea Party: I think that depends on several more subtle things.

One, the NYT article cited above is clearly biased towards retired people. They hate government, but they love their Social Security and Medicare. My bet is that this was a poll conducted using telephone land lines. Because if you do that, that's what you get. Mitt Romney didn't figure out that bias in time.

Second, such polls limit themselves to finding people who actually identify themselves as "Tea Party". I think that a different, but more valid analysis could be made based on: Who is it who votes for political candidates that support Tea Party policies? In that case you get a map that looks like a map of the formerly Confederate states plus rural areas of the Midwest and Far West. For that, you get a map something like this one: http://www.newyorker.com/online/blogs/comment/2013/09/meadows-boehner-defund-obamacare-suicide-caucus-geography.html?currentPage=all&mobify=0

Then, I think that you need to analyze educational outcomes in this country. The map above is also a map of areas in which educational outcomes are lower than some. But so are Democratic Party strongholds in the urban areas of cities. So if you look at education only in terms of a party divide, effects are conflated by other issues.

You could measure Tea Party support by dollars invested in support of it, in which case you'd find only a handful of people, including two with the last name Koch. But the Kochs have such outsized wealth relative to most of us that this is an obviously biased way to measure commitment., even if it does say much about influence.

You might look at audience. Who watches Fox News? Who listens to Rush Limbaugh on talk radio? But who watches TV has do do with both money and time. Retired folks for example. Or in the case of talk radio, also people able to play the radio while working. Truck drivers, farmers, auto repair shops yes, teachers, office workers medical professionals not so much.

In a broader sense, I have long argued with Chris Mooney, on his blogs, that identifying Americans by the labels "Democratic" or "Republican" did not really take into account the breadth of opinions and diversity of people beneath those labels. Or the large numbers who register as independents, who might be in between, but easily could be further to the left or right than other parties. This is especially true since the parties are so embedded. We do not have a system that easily allows new parties to arise with the times. Both parties link rich and poor constituencies in ways that may have a historical basis, but still might seem incompatible in present circumstances. And new issues that arise cross cut these constituencies. Privacy and the NSA, for example, is much more likely to be a concern of those at either end of the political spectrum, and less so in the "silent majority" part of the middle.

On Montessori: Joshua, I think we are largely in agreement. Both of my offspring attended Montessori preschools, although one also attended a preschool accredited by the National Association for the Education of Young Children, which I also thought was very nice. Using such preschools as opposed to chain outfits like Kindercare, or a patchwork of family and neighborhood facilities, is an obvious measure of privilege. I'd also note that regulations are different in different states. This make a huge difference on significant things such as the numbers of children per adult. Those options, or the option to stay home and devote time to one's child are means of giving that child a forward boost that is not necessarily taken into account when measuring outcomes of school systems, although programs like Head Start are an attempt to help overcome this barrier.

On grading: I think that these are a shortcut to more personalized forms of feedback, but in the real world, shortcuts are sometimes necessary. I think that information needs to be generated that demonstrates differences in educational outcomes. Because how else can improvements be made? But somehow these evaluations need to be undertaken in manners that create as true a measure as possible, and ones that inform rather than stigmatize.

October 16, 2013 | Unregistered CommenterGaythia Weis

"Maybe you think that the way to prepare a student for life is not to prioritize their intellectual development, but to show them that to succeed the most, they need to prioritize following instructions over exploring their creativity."

It's not about "following instructions", it's about satisfying the customer's requirement. If the customer wants you to be creative, they would have told you to be creative, and being creative would be great. But if the customer asked for something specific, then that's what you need to deliver to get them to pay you in return.

"Do you really think that a student who has a history of drug problems and behavioral problems is going to receive the message that you want to give them with a failing grade?"

If you just given them the grade with no explanation, of course not. But I would expect the teacher to explain why, to give positive feedback and encouragement about the positive bits, and go through what the student could do about it.

You explain it like this. Working for a living is basically about doing deals. You already know how that works. You go to your drug dealer, give them money, and they give you the drugs. You do not give your dealer poetry you have composed in their honour. You don't give them a hug. You may argue that such things have value infinitely greater than tawdry cash, but this is not a strategy that will succeed. Same with a grocery store, or car dealer, or burger bar. You survive in this world by doing deals with other people, in which you give them what they want so they will give you what you want, and everybody gains. But it means you have to drop the self-indulgent crap and pay attention to what the other person wants, and make sure you provide it. If you want the drug, you have to deliver the goods. The same applies to school grades, wages, promotion, praise, profits, fast cars and wild women.

You offer the customer what they want at a price they want to pay. It is the secret of success.

There are places in this world for people who don't conform. If you've got the skills, there are employers who will make allowances. They'll let you go off and play, they'll listen to your ideas, and if they think they can make money out of some of them, they'll handle all the tedious conformity stuff for you. (All at a price, of course.) But when they need something specific, you have to deliver what they asked for, or they won't get the money they need to be able to pay you.

And you have to have something they want - skills, creativity, enthusiasm, whatever - and that's what school is about. School is where you load up on valuable skills that you can later sell. And sure, if you think you can afford it, you can load up on a skill or two purely for your own pleasure. but fundamentally it's not about what you want or like, or even what your teachers want, it's about what the customers (i.e. employers) want, because you have to give them what they want so they will give you what you want.

Exactly the same reasoning applies if the profession you end up in is "professional burglar" being effectively employed by your drug dealer. It's just another very high risk job for not enough pay. But the same rule applies - you give your employer what he wants so he will give you what you want. You can't escape from the system by dropping out.

So you tell them they got a fail this time, but only because it wasn't what was asked for, not because it wasn't any good. And we can make sure that next time it will be what was asked for, because you intend to ask for it. Next time, they'll get an 'A', and they'll deserve it. Deals can be made. By being clever and thinking about more than just the immediate task, they can do a lot better for themselves than if they stumble along blindly.

People learn when they can see the personal advantage in knowing it. That advantage might be their own pleasure, or it might be strictly business. While I would agree that knowing anything is better than knowing nothing, and if the only work you can get them to do is what they're personally interested in, then that's probably the best you're going to achieve. But there are too many kids coming out of school who have skipped all the difficult, boring bits, and expect to be entertained. They walk out with virtually no usable skills, and moan about how there are "no jobs", and it's all too difficult for them. It used to be about instilling the "work ethic", but now its all about "exploring your feelings" and "personal development". Which is all very nice, but it doesn't buy the goods.

When kids emerge from school with no hope like this, the teacher ought to get a 'fail' too. But they're a product of the same system, often having left school by one door only to immediately re-enter it by another door as an employee, and often don't understand how the outside world works either. It's a terrible mess.

--

I've just had a quick look at Armstrong. Yes, it works better when students take responsibility for their learning, but this is because people learn when they can see the personal advantage in doing so, and meeting their own targets and goals works as such an advantage. That doesn't mean that grades and teacher tuition are intrinsically a bad thing, it's that the students have to use them to meet their own aims, not just because they've been told to. Good grades only work if there's an obvious advantage in getting them.

And it would be less good if students were to plan their learning and 'take responsibility' merely because they had been told to take responsibility. It's all about the motivation.

October 16, 2013 | Unregistered CommenterNiV

Hey Gaythia -

"but in the real world, shortcuts are sometimes necessary."

Sure - the list of tasks on a teacher's schedule imposes certain practical limitations...

"Because how else can improvements be made?"

As I'm sure you know, many people using portfolio assessments with large student populations. Do you not think that can be a viable way to assess students work and give them feedback? Norm-referenced grading and testing - based on a competitive model - are inherently anti-intellectual, IMO,and much more likely counterproductive than helpful (although they aren't necessarily counteproductive). Even as a shortcut, on that can be supplemented with other forms of assessment and feedback, I'd say that they are negative in balance, and certainly if you consider the "opportunity cost" of other forms of assessment (like portfolio assessment). Criterion referenced assessments are the way to go,IMO. Look at the success that Finland has had through employing a more progressive model.

---------------------

NiV -

"That doesn't mean that grades and teacher tuition are intrinsically a bad thing, "

The bottom line is that if you're relying on grades to motivate students to learn, you've basically failed as a teacher because you lack the knowledge, imagination and skill to find the way to uncover and exploit students' intrinsic motivation. For sure, not all students are ready to become active learners. In fact, in my experience (even working at "elite" colleges and universities) the majority of students aren't. But a good teacher, IMO, keeps a focus on helping students to become active learners as their number one goal. That needs to be the main message. If you only take a student through steps 4 through 4.5 of the 1,000 steps s/he has to take to become an active learner, because they are not ready to advance further, at least you have kept the right focus rather than succumbed to an antiquated concept of teaching whereby learning is doing what someone tells you do to, and your work is graded on the basis of how well you did what you wre told to do as compared to the student next to you. The longer you try to rely on empty substitutes, such as grades and a competitive model, the longer you're delaying them developing the the most important skills of a good learner - skill that pay life-long dividends far more valuable than leaning to conform to someone's instructions. An active learner has all the skills necessary to succeed in a workplace where they have to meet a customer's or boss's expectations. There is NOTHING about being an active learner that is in any way an obstacle for understanding how to fulfill such tasks.

...and if the only work you can get them to do is what they're personally interested in, then that's probably the best you're going to achieve.

That reflects a misunderstanding. Students who are intrinsically motivated can certainly undertake tasks that they aren't interested in, because they understand the need to do so. You don't seem to be addressing the fact that our schools are graduating passive students despite that they rely so heavily on a competitive model and grading.

"But I would expect the teacher to explain why, to give positive feedback and encouragement about the positive bits, and go through what the student could do about it."

Think about it. Do you really believe that with the kind of student I was referencing, they haven't heard your sort of armchair philsophizing hundreds of times from dozens of teachers? Do you really think that what you're describing about what a student should be told is something in any way different from the typical student/teacher interaction that has failed that student (and others like him), many times in the past? The grade in that system is the bottom line, It's were you deliver to the student the real message behind your view of education. "Yes, your exploration of a complex topic and your intellectual development is exactly of the sort that scholars in your field undertake. And you show great skill in your work. But because you didn't follow directions I'm going to fail you, and you will have to take this course again and keep doing it, no matter how brilliant your work is, until you learn to follow my instructions. And as long as you continue to fail this course because you don't follow instructions, you will not graduate. Of course, you can drop out if you want, but understand that is your fault because you are choosing to not follow directions."

All I can say is good luck with that!

And what's really interesting about delivering that kind of message to a student is that in some ways wouldn't be bad for the student, as they are at least getting positive feedback for the more important aspects of their efforts. What is really bad about that kind of absurd hypothetical is the message that it delivers to the student about how the teacher involved views the educational process, and what the teacher is trying to say to the student about what they need to do to succeed in life. Fortunately, at least some students see right through that kind of view, because they can look around and realize that the people who most succeed are the ones who know how to pick and choose when to do what someone asks them to do, and to not just do essentially meaningless tasks just because someone is telling them to do them and judging them on how well they follow instructions. Read what Armstrong (and other research) has to say about the relationship between grades in school and long term job performance. Read what he has to say about the effect on motivation of extrinsic rewards.

"But they're a product of the same system, often having left school by one door only to immediately re-enter it by another door as an employee, and often don't understand how the outside world works either."

Maybe you should write a letter to Armstrong to explain to him how he doesn't understand how the real world works - and if he did, he'd understand how to talk to students to explain to them how important it is that jumping through hoops is a desirable form of education?

" It used to be about instilling the "work ethic", but now its all about "exploring your feelings" and "personal development". ."

Very funny. So you think that is what Armstrong is advocating? Did you even bother to think about the context in which Armstrong teaches, and who the students are that he was describing?

Anyway, IMO it's quite unfortunate that people have such an antiquated and bass ackwards view of education. I do hope that you at least will take forward that a more progressive view of education does not necessarily come from "leaning left," but for some at least, it comes from studying education and reviewing the literature and from gaining experience in walking the line between educational theory and practice. That's why I keep asking you for some kind of empirical evidence to support your seat-of-the-pants philosophizing about what to tell students so that they''ll "understand." There is some out there, and if you would refer to it the discussion could deepen. But why won't you back up some of your educational philosophy with some careful research? Undoubtedly, educational research is incredibly complicated (so many hard to control variables), but surely you can some up with something that you have relied on to develop your understanding of pedagogy other than just anecdotal wisdom?

October 16, 2013 | Unregistered CommenterJoshua

Hah, I love all the respondents whose immediate reply to your scientific finding is "you must have done something wrong! There's no way reality could fail to match up with my uninformed prejudices!"

You guys aren't actually interested in science. You just want something with Authority that tells you you're right about everything.

October 17, 2013 | Unregistered CommenterThirteenthLetter

In my observation, the main reasons this is 'surprising' to people is a confusion between different types of 'right-wingers'. Many people seem to equate the 'Tea Party' with the 'religious right' when they are two very different groups of people. The Tea Partiers are fiscal conservatives, concerned about high levels of government spending and taxes, are usually not concerned much with social issues, and oppose laws restricting 'social ills' like gambling. The religious right, the people out protesting about abortion and complaining about evolution (the Rick Santorum crowd) tend to be less interested in fiscal issues and more likely to support welfare programs like TANF and WIC. The religious right probably scores low and drags down the overall average for 'conservatives'.

October 17, 2013 | Unregistered CommenterLibertarian

"Many people seem to equate the 'Tea Party' with the 'religious right' when they are two very different groups of people."

Well, the demographic evidence shows quite a bit of overlap. 'So, no, they aren't two very different groups of people.

For example:

Nearly two-thirds consider themselves "pro-life" on the abortion issue, compared with 46% of all national adults.

http://www.gallup.com/poll/127181/tea-partiers-fairly-mainstream-demographics.aspx

And there are certainly some policy stances associated with the Tea Party that could be argued are not "fiscally conservative."

October 17, 2013 | Unregistered CommenterJoshua

"Do you really believe that with the kind of student I was referencing, they haven't heard your sort of armchair philsophizing hundreds of times from dozens of teachers?"

If they have, and they have made their decision in full knowledge of the consequences, then you let them. They are a free human being. They don't have to fit in to our society if they don't want to. Let them go.

Boot them out. Stop wasting taxpayer money on them.

"What is really bad about that kind of absurd hypothetical is the message that it delivers to the student about how the teacher involved views the educational process, and what the teacher is trying to say to the student about what they need to do to succeed in life."

It's an accurate message, though.

I recall one elective statistics course that I did the marking for, where the students were basically useless. The lecturer put pressure on us to mark them high, so as not to discourage them. Yes, they weren't mathematicians, we shouldn't judge them by our higher standards. We argued, but eventually with a shrug, we complied. Customer requirement.

At the end of the course, the lecturer got to mark the exam. Afterwards, he came round to apologise to us. (With chocolate cake!) They'd gone through an entire ten-week course and were still making the same basic mistakes and misunderstandings at the end as at the beginning. And why should they not, when their grades were telling them everything was fine?

We lied to them. We told them they were good. We told them the course was easy. We told them they could succeed with no effort. And at the end, we were faced with the choice of passing them and degrading the course reputation as a worthless qualification, or failing them all and kicking them out with nothing for their time.

And none of them would be able to do the job, or earn a living with what little they had learned, whether they got the qualification or not.

That's what's bad about it.

October 17, 2013 | Unregistered CommenterNiV

"If they have, and they have made their decision in full knowledge of the consequences, then you let them. They are a free human being. They don't have to fit in to our society if they don't want to. Let them go."

Who's saying that they shouldn't be given the freedom to reject your advice? Certainly not I. Why would you make that assumption. It is completely inconsistent with what I've been arguing here.

"Boot them out. Stop wasting taxpayer money on them."

Interesting it seems that the only two alternatives you can think of are giving them your advise and booting them out of they don't take it.

In fact, there are other ways to go. That is my whole point.

We both know that the current system doesn't work very well for a lot of students. We both know that the way that it works produces a kind of social sorting system, that perpetuates social class problems. And rather than consider pedagogical options to create different results results that would be a more efficient use of tax dollars (in fact, rather than even study, empirically, the results of different options),, you'd rather just actually double-down to likely make the results even more skewed by SES status? Have you read the economic studies that explore the negative economic effect of educational failure among a large % of our youth?

But actually, in a way, I am sympathetic to your viewpoint - as these are problems that are in some ways related to mandatory educational requirements. For many kids, the only reasons why they're in school is because they have to be (because people think that they'll benefit from being told how they'll benefit from following directions). In fact, they'd probably be happier and more productive long term if they were doing something like apprenticing at an automobile repair shop.

You want to know the irony of your prescription for that kid? He was actually able to hold a job an an automobile repair shop. He was fairly skilled. You see, it wasn't that he just didn't know how to follow directions, and that he could learn how to do so if someone like you explained to him how important it is. He just didn't see the point in following directions that he saw no benefit or purpose to. But he did see the purpose in exploring his intellectual development. The problem is that the educational process in the school didn't really incorporate a complete picture of his intellectual development. The problem, if you want to call it that, is that he didn't prioritize the attainment of a diploma over his intellectual development.


"where the students were basically useless. "

Classic. Really. A teacher who calls the students in the class "basically useless."

"We lied to them. We told them they were good."

It is interesting how often you come back to the same misconception, the same uninformed caricature, over and over. Nothing that I'm arguing here suggests, in any way, that you "lie" to students and tell them that they have accomplished a goal when in fact they haven't. What you're describing is completely antithetical to outcomes based assessment. Get outside of your binary limitations that there is no alternative to the kind of pedagogy that you're arguing in support of other than not holding students accountable for their learning an intellectual development.

October 17, 2013 | Unregistered CommenterJoshua

NiV -

An interesting facet to this discussion is that when I read Armstrong's viewpoints, and consider the empirical support that he offers for his views, it strikes me that his views and the ways that he presents them are entirely consistent with a libertarian ideology and a "hard science" approach, and that your arguments are much more consistent with what I would think of as a "statist" or "elistist" viewpoint and a not-hard-science approach (what is the opposite of "hard science?" certainly not "soft science." Perhaps "anecdotal philosophizing?"

October 17, 2013 | Unregistered CommenterJoshua

Almost all minorities are part of the liberal democratic base, yet according to the author the liberal democrats are once again far more intelligent than Conservative Republicans. I've often wondered how this can be, do the liberals exclude their own base? Are minorities just written off and not included in these tests, which are designed of course to show the lefts superiority ?

October 17, 2013 | Unregistered CommenterWondering

Almost all minorities are part of the liberal democratic base, yet according to the author the liberal democrats are once again far more intelligent than Conservative Republicans. I've often wondered how this can be, do the liberals exclude their own base? Are minorities just written off and not included in these tests, which are designed of course to show the lefts superiority ?

October 17, 2013 | Unregistered CommenterWondering

Almost all minorities are part of the liberal democratic base, yet according to the author the liberal democrats are once again far more intelligent than Conservative Republicans. I've often wondered how this can be, do the liberals exclude their own base? Are minorities just written off and not included in these tests, which are designed of course to show the lefts superiority ?

October 17, 2013 | Unregistered CommenterWondering

The foundation of the Tea Party movement is a logical recognition of the inevitable long-term consequences of unsustainable deficits. This is pure application of reason, and it challenges the political ideologues who are Deniers of that reality. The predictable response by ideologues whose delusion is challenged is rage, personal attacks, and demonization, which is exactly what has been done toward Tea Partiers.

October 17, 2013 | Unregistered CommenterBrad Gates

Dan,

Might not the Tea Party effect be readily explained by the fact that they are, on average, more educated and wealthier than the average person? http://www.nytimes.com/2010/04/15/us/politics/15poll.html?_r=0&pagewanted=all

October 17, 2013 | Unregistered CommenterJosh Rosenau

"Who's saying that they shouldn't be given the freedom to reject your advice?"

I have no idea. Who's saying they should?

"Interesting it seems that the only two alternatives you can think of are giving them your advise and booting them out of they don't take it."

No, I'm saying that if they don't want to learn, and understand the consequences of that decision, then you boot them out. If they're not going to learn, then it's a waste of everyone's time for them to be sat there - theirs included.

"We both know that the way that it works produces a kind of social sorting system, that perpetuates social class problems."

I don't agree. It's class culture that creates and perpetuates the problems. But changing people's culture can only be by their own choice.

"you'd rather just actually double-down to likely make the results even more skewed by SES status?"

The skew is not caused by SES status. It's caused by cultural choices that are strongly correlated with SES status. It's perfectly possible for poor kids to get the right attitude and work their way out of poverty. It's equally possible for rich kids to have a bad attitude, waste their education, and drop into it. It's all about the choices people make. And it would help if people stopped making excuses for them, telling them they're not responsible for the results.

"Have you read the economic studies that explore the negative economic effect of educational failure among a large % of our youth?"

Yes.

"But actually, in a way, I am sympathetic to your viewpoint - as these are problems that are in some ways related to mandatory educational requirements."

Indeed. That was my thinking.

"You want to know the irony of your prescription for that kid? He was actually able to hold a job an an automobile repair shop. He was fairly skilled."

So what's the problem? You drop him from classes he has no interest in learning, and enrol him full time in those he is.

"He just didn't see the point in following directions that he saw no benefit or purpose to."

He may well have been right.

"But he did see the purpose in exploring his intellectual development. The problem is that the educational process in the school didn't really incorporate a complete picture of his intellectual development."

Exploring your intellectual development is a fine goal, but it's not what grades and qualifications measure. So long as you understand that, then you understand that the failing grades don't actually matter, and you can carry on regardless.

" The problem, if you want to call it that, is that he didn't prioritize the attainment of a diploma over his intellectual development."

It's not the diploma he needs, it's the skills. The diploma is just to certify that the skills have been acquired.

"Classic. Really. A teacher who calls the students in the class "basically useless." "

They were.

If teachers were a bit more honest about this sort of stuff, we'd have less problems.

" Nothing that I'm arguing here suggests, in any way, that you "lie" to students and tell them that they have accomplished a goal when in fact they haven't."

Then I'm completely confused as to what you're talking about. The student is given a question which they didn't answer, giving an answer to a different question that wasn't asked, but which they found more interesting. The goal was to answer the question asked. They didn't accomplish it. That's a fail. Anything else is a lie.

" it strikes me that his views and the ways that he presents them are entirely consistent with a libertarian ideology and a "hard science" approach, and that your arguments are much more consistent with what I would think of as a "statist" or "elistist" viewpoint and a not-hard-science approach"

Armstrong's approach is consistent with my approach - except that I'd offer the choice.

My approach is based on "the freedom to take the consequences". Schools offer the opportunity to learn new skills, but you should have the freedom not to take it. If you choose not to take it, understanding the consequences, then I would let you. It's your choice. Grades are (should be) a guide to how well you are doing learning the skills employers are looking for - including answering the question asked. You have the freedom to pursue them or ignore them as you choose. If you want to pursue them, this is what you have to do to get them. If you choose not to, then you shouldn't have a problem with not getting them, and a long string of 'fail's sends no particular message. You know the consequences of not getting good grades. It's your choice.

The teacher is responsible for making sure they understand the consequences of their choices, but the student is responsible for choosing. They have to take responsibility for their own education, one way or the other.

It's the same in life after school. You can choose to pursue money, or you can choose not to. You know what the consequences of not having any money are. But if you chose that life, then don't complain, and don't beg for handouts from those who chose differently. Freedom has its ups and downs.

For the people who choose to pursue it, the pursuit motivates the generation of new wealth on which we all live. And as with school grades, subverting the signals it sends subverts that process, and cripples our source of prosperity. That's wouldn't be my choice.

October 17, 2013 | Unregistered CommenterNiV

You're a bigot, Dan Kahan.

October 17, 2013 | Unregistered CommenterJP

"I don't know a single person who identifies with the Tea Party. All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico)."

So Tea Party stereotypes are based on misguided talking points (read 'lies')?

Maybe a smart Yale law professor like yourself should find out more about these people on your own, instead of letting some media source that is running an agenda tell you what these "enemies" are about?

October 17, 2013 | Unregistered CommenterCarlos Danger

I'm a Tea Party member. I have BS in Information Technology Cyber Security. I love science and am also a practicing Christian. Why am I a member of Tea Party? I think many people have the wrong impression of our goals. We don't want to shut down government. Quite the contrary. We recognize that government is necessary in any society. What concerns us is government attempting to do all things for all people. That is not their job. We think the federal government should adhere to the powers granted them in the Constitution. They are enumerated and clear. The 10th amendment also makes it clear that if the power is not listed then it belongs to the states and the people.
While we believe in helping our fellow man, we don't believe that the Constitution gives the federal government the right to use one citizens money to provide charity to another citizen.

October 17, 2013 | Unregistered CommenterJohn

I read the article and proceded to read the comments. It is amazing that these so called "scientists" create a study to prove that liberals are smarter than conservatives and when the results do not come out as planned, they write paragraph after paragraph about how the study was flawed somehow, or they need to use a different dataset. What a load of BS. This is the same mindset that pushes global warming and other "religious" beliefs on the populous. When the science doesn't match their outlook, they change the dataset and run it again and again until it does. That's not science and not honest.
Signed - a proud Tea Party member

October 17, 2013 | Unregistered CommenterT-party

Not really surprising to me. The core of the actual tea party worldview might be simply described as follows:
(1) Fiscal responsibility - taxes are too high compared to benefits, and the government is spending far more than it takes in. Something that cannot go up forever -- won't. The higher it goes, the more painful the crash.
(2) The government, specifically Federal government, is too large, inefficient, and wasteful. Government should be first local, then state, and only last Federal; and as little should be federal as possible.
(3) The free market, while imperfect, works better than any other system (so far). Increasing government regulations are crippling our economy.
Notice there is nothing in there about religion, abortion, gays, or any other so called "conservative" strawmen. That is because the tea party movement is focused on fiscal responsibility, reducing the power and size of the Federal government, and reducing the size of the regulatory thicket. And that is it.

People who hold these views may be quite intelligent, rational, and scientifically knowledgeable.

October 17, 2013 | Unregistered CommenterSunSwordTiger

But then again, I don't know a single person who identifies with the Tea Party. All my impressions come from watching cable tv -- & I don't watch Fox News very often -- and reading the "paper" (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).

This might be the most significant statement of the study and, in my opinion, explains a tremendous amount about the conflict we are seeing. I suspect both right and left sides are equally homogeneous in their daily lives, such that each now comfortably resides in an echo chamber, becoming more and more convinced their position is correct because it is repeated much like a feedback loop. Not only is their confidence in their opinion strengthened, but also their genuine ignorance of the other side also grows, because (1) they only hear about it from their side, which has a vested interest in degrading the other, and (2) since they don't regularly encounter a different view point, they presume only a bizarrely foreign person could harbor such a view. In truth, the opposition is no more bizarre or foreign than a neighbor or colleague.

Perhaps (I don't have any evidence either way) this why there are no more great compromises, and everything is has become all or nothing. Each side brands the other in such pejorative terms that to make even the most minor concession is seen as defeat and capitulation by the other side.

October 17, 2013 | Unregistered CommenterGD

Dan you said it yourself, you live a liberal bubble. You only interact with liberals and consume liberal media. Since 95% of the media and academia is liberal, there is an echo chamber of people telling each other how dumb people opposed to their politics are and soon becomes their reality. Just because I don't agree with reckless spending and saddling my children with this debt burden doesn't make me an idiot.

October 17, 2013 | Unregistered CommenterCK

It appears the author deems his viewpoint superior so will just dismiss any other opinion. Did he shut his eyes and plug his ears when the news showed feral humans plummeting Walmart after EBT system crashed? Hey Professor, YOU DID BUILD THAT.

October 17, 2013 | Unregistered CommenterTessa

I have a MS and PhD in microbiology, spent almost five years in postdoctoral training in biochemistry, am a researcher in vaccine development and structural biology, and serve as a reviewer for a major biochemistry and biophysics journal. I am an academic, make a decent salary, and have a wide range of friends, including liberals. I am also a Tea Party guy, if that is the label. The reason for this and my concerns are that these deficits are absolutely unsustainable and they will eventually collapse the economy of this country. The government has grown too much under Obama and our freedoms are rapidly vanishing. No arguing these points. Once the Hannibal Lecter mask comes off liberals are going to understand what it means to be careful what one wishes for.

October 17, 2013 | Unregistered CommenterRichard

While I find your tea party information very interesting, I was wondering if you could do a different run of it. You have the left-right political outlook measure. Can you run the Tea Party vs other conservatives and then run the tea party vs the liberals? I think that some of the data gets skewed (ie a shift in the data output) a little when you put the tea party vs the rest of the data set. Now I could be very wrong on this, but would like to see how tea party stands against only the right and only the left and then see how the data looks with the left compared to the right minus the tea party.

October 17, 2013 | Unregistered CommenterBrent

If you want to read an engaging, heavily-footnoted airtight explanation of the factual legal and historical reasons why the so-called 'tea party' position on money, central banking, and progressive income taxation (a.k.a. Communism -- citation: The Communist Manifesto, Central Platform, Planks 2 and 5) is correct, I suggest you get down to the University library and check out Pieces of Eight: The Monetary Powers and Disabilities of the United States Constitution, 2nd ed., by Dr. Edwin Vieira, Jr. Over 6000 footnotes for you there.

Oh, and Dr. Vieira's B.S., M.S., Ph.D. and J.D. are all from Harvard, but don't discount it on that account.

And Hard Science literacy? The "Dr." is for his Ph.D. -- in Chemistry.

October 17, 2013 | Unregistered CommenterDB

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>