A "CCP journal club!" report from D. Evans:
"Aporia" is a mode of reasoning that shows the author comprehends “an issue’s intractable complexity.”
Too often, judicial opinions addressing complex value questions are anything but aporetic. While the public is deeply divided over the issue, judicial opinions often “effect a posture of unqualified, untroubled confidence” in the outcome. This “[h]yperbolic certitude” might undermine the legitimacy of the opinion with the losing side, making it seem as though the decisionmaker was biased or unwilling to recognize the strength of arguments supporting the losing side’s position.
In addressing how courts can assure citizens of the law's neutrality, my CCP colleagues and I have conjectured that judicial decisions might reduce cultural polarization and garner acceptance from the losing side by abandoning the norm of reasoning as if the answer is obvious, indisputable, and certain.
Instead, if a court were to recognize (a) the difficulty (even intractability) of the problem, and (b) the strength of the losing side’s case, perhaps the losers would be more likely to perceive the opinion as a legitimate one; one that took their concerns and arguments deeply into account. If the losing side sees its concerns and arguments were thoroughly considered in the decision, it might also be more open to accepting the arguments that prevailed in the outcome. I have long thought about testing this hypothesis that aporetic reasoning would reduce cultural polarization over a controversial ruling.
So I was really excited to read Rob Robinson’s empirical study on exactly this point: It’s How You Say It – Ameliorating Cultural Cognition of Judicial Rulings Through Aporetic Reasoning.
Robinson's study follows a few others with promising implications for the aporia hypothesis: Tom Tyler's research, described here, finds that public views about the legitimacy of legal authority are influenced by the procedural justice and by the distributive justice of the outcomes, but less affected by the favorability of the outcome. Dan Simon and Nicholas Scurich, Lay Judgmentsof Judicial Decision-Making, have found that people tend to agree more with decisions recognizing good reasons support either side of the case than decisions that only recognize the value of one side's position. They also find that an opinion giving no reasons is more persuasive than one including a single, curt reason. (Simon and Schurich's findings rebuffed a preexisting hypothesis called ‘placebic reasoning’ – that people are more likely to credit decisions or actions when backed by reasons, even if those reasons are entirely redundant (i.e., asking to cut in line for a copy machine was less credible than asking to cut in line for a copy machine and providing a redundant reason, “because I have to make copies.”)).
While these studies support the aporia hypothesis, Robinson is the first (to my knowledge) to frame his testing in terms of aporia, specifically.
Robinson conducted an experiment designed to test how members of the public would react to more and less aporetic versions of a judicial decision contrary to their own position on gay marriage.
The study subjects, 619 individuals representing a mix of university students Amazon’s MTurk workers, were assigned to one of three mirror-image conditions.
In the “control” condition, subjects read a newspaper article describing a judicial decision that examined whether homosexuality should be recognized as an “immutable” (i.e., unchosen, and unalterable) trait. The story reported the court’s conclusion—either “no,” if subjects said they supported gay marriage; or “yes,” if they said they did—and nothing more.
In the “monolithic” condition, the article includes a quote from the court’s opinion in which the court defends its reasoning by remarking that that an “objective reading of the evidence leads to no other conclusion.” The court explains that it is obliged to reject the position supported by the study subject—either that homosexuality is “ immutable,” in the version of the article shown to gay-marriage supporters; or that it is not, in the version shown to gay-marriage opponents—on the ground that there is “no clear scientific consensus” in favor of that view.
In the “aporetic” condition, the news story quotes language from the opinion evincing a more nuanced stance. The quoted language chides one side or the other—either “those who believe homosexuality is a choice” for “often ignor[ing] evidence [to the contrary]” or “those who argue sexual orientation is fixed or unchanging” for “often overstat[ing] their case.” The court nevertheless justifies a ruling in favor of the scolded side on the ground that a court is powerless to deem matters otherwise in the face of uncertain evidence.
Robinson reports that subjects found the court’s reasoning more persuasive in both the “monolithic” and “aporetic” conditions than in the control. In other words, the subjects were least disappointed by the decision when they were told the court had given an explanation for rejecting their position.
In the view of the subjects who oppose gay marriage, the aporetic opinion was even more persuasive than the monolithic one.
But for those who support gay marriage, the persuasiveness of the decision did not differ significantly among those assigned to “aporetic” and “monolithic” conditions, respectively.
The mean opponents of same-sex marriage rated their disagreement with three forms of the pro-same-sex marriage decision on a scale of 1 ("extremely agree") to 6 ("extremely disagree"): Control 4.16; Monolothic 4.10; Aporetic 3.53. For opponents of same sex marriage, the monolithic opinion was about .06 less disagreeable than the control, the aporetic one was .6 less disagreeable. The mean supporters of same-sex marriage rated their three forms of the anti-same-sex marriage decision follows: Control 4.58; Monolithic 4.46; Aporetic 4.36. Among supporters of same-sex marriage, the monolithic opinion was about .02 less disagreeable than the control, and the aporetic one was .12 less disagreeable than the control.
This is a super valuable study!
I particularly liked the way in which Robinson distilled the aporetic reasoning into a few quotes set within the framework of a newspaper article. There is much innovative about his deisgn, and his study makes me eager to design a follow up study along these lines. In thinking about how to do so, I have been pondering several questions about the design of this study:
- One puzzling aspect of his findings is that supporters of same-sex marriage were overall more negative about all three forms of the opinion ruling against it, and they found the aporetic version only slightly less disagreeable. While the aporetic opinion significantly reduced the extent that opponents of same-sex marriage disagreed with a pro-same-sex marriage decision. (The effect of the aporetic treatment on anti-same sex marriage group's disagreement was -0.592, while the effect of the aporetic treatment on pro-same-sex marriage group's disagreement was only -0.150.)
Why were supporters of same-sex marriage overall more resistant to crediting the contrary opinion, and why was their disagreement less mitigated by aporia? Robinson states this might be caused by the sample of those who favor same-sex marriage being larger (N pro-same-sex marriage=496, N anti-same-sex= 161). (But the larger sample should supply the more significant result if the phenomenon exists, not the less significant one.) He also posits that the difference in reaction may result from "those who favor gay marriage simply having a stronger reaction to empirical claims regarding immutability than those who are opposed." P. 18.
It could be the case that supporters of same-sex marriage are categorically more rigid in their position, and less willing to credit a contrary ruling regardless of its reasoning.
But I'd posit another possible explanation. Perhaps the pro-same-sex marriage group's rigid disagreement relates to their views on the relevance of whether homosexuality is immutable, as opposed to an extra-strong belief that same-sex marriage should be allowed. It seems that there may be many egalitarian individuals like me who think that same-sex marriage should be allowed regardless of whether it is immutable. I think any constitutionally protected individual liberty should be an impermissible basis for discrimination, regardless of whether it is immutable. (Indeed I'm offended by the notion that protection is limited to traits that are predetermined rather than chosen pursuant to constitutionally guaranteed autonomy.). I would be much more persuaded to support regulation of same-sex relationships if it were shown that they caused harm to public welfare: the stability of marriage or childrearing.
Hence, I wonder whether the extra-strong disagreement with the opinion finding homosexuals are not a protected class may represent disdain of the idea that immutability determines the degree of constitutional protection. This is frustration with the legal standard as opposed to ideology-based cognitive rigidity. For this reason, one of my overarching questions about Robinson's study is whether immutability is the best empirical issue for measuring cultural effects in the same sex marriage debate. I would be inclined to focus on welfare-related empirical questions, such as how same-sex marriage impacts childrearing, a question on which strong cultural effects have been observed.
Furthermore, because these welfare concerns seem to be more often cited in the public debate as a reason for prohibiting same sex marriage, it seems cultural identity may be more strongly tied to one’s beliefs about these questions than one’s belief about immutability. (While certainly part of the debate about the morality of homosexuality, immutability seems to be cited less often as the public reason for prohibiting same sex marriage.) It seems some might oppose same sex marriage for purported public welfare consequences, regardless of whether sexual orientation is immutable. And as I have described above, some proponents of same-sex marriage might be particularly resentful of a decision based on immutability, as they do not believe this should be a relevant factor. This group might also, while cognitively motivated to support a pro-same-sex marriage ruling, be disinclined to support a ruling that homosexuality is immutable.
My other questions pertain to specific elements of the study's design:
- Asking for views about same-sex marriage: I wonder whether first asking subjects about their stance on same-sex marriage makes them less susceptible to being persuaded by the aporetic reasoning we are testing. Because people don’t want to be inconsistent—either internally or be perceived as such by those conducting the survey—they might resist crediting the ruling after reporting disagreement with its conclusion at the outset of the study, regardless of whether the find the aporetic or monolithic reasoning persuasive. It seems the cultural measures provide enough information to predict a subject’s likely orientation on same-sex marriage, and it is unnecessary to ask subjects about the issue being studied.
- Assignment to conditions with which subjects are inclined to disagree: I also question the decision to only show subjects opinions with which they are inclined to disagree. It seems to me that a study of this nature should measure the reasoning’s persuasiveness to both those inclined to disagree with it and those inclined to agree with it. It may be that an aporetic opinion is more persuasive to those inclined to disagree, it is less persuasive to those inclined to agree. It seems this, too, would be a noteworthy finding. The question should be whether opposing cultural groups converge on the persuasiveness of an aporetic opinion more than they do on a monolithic one.
- Focus on whether the opinion is persuasive rather than correct: I would not focus on asking subjects whether the court’s conclusion is correct or accurately reflects scientific findings, but whether they find the opinion persuasive. Subjects might agree with the court’s conclusion or believe that it accurately states scientific research, but find its reasoning unpersuasive. Or to the contrary, they might disagree with the court’s scientific conclusion, but find the reasoning persuasive.
- More detailed reasoning: I might consider including a few more sentences so that the court’s reasoning more clearly pronounces three elements that I associate with aporia: (a) noting that this is a difficult, perhaps intractable, question, on which there may be no correct answer; (b) saying the evidence is unclear, and presents the strongest points in favor of each side; and (c) gives reasons for crediting one side’s position despite this empirical uncertainty. (I think this last point is the most contentious aspect of aporia – a court must justify its conclusion after admitting that it is uncertain as to the evidence – and it would be particularly interesting to test.). The monolithic condition would do the opposite--e.g., (a) state that the question is simple with a clear right answer; (b) say the evidence is clear or unequivocal; and (c) hold that there's no way one could reach a different result based on the evidence before the court.
- Singling out one side: The aporetic versions in Robinson's study single out one side (The unprotected class version begins: “Those who believe homosexuality is a choice often ignore evidence [to the contrary]”; and protected class begins: “Those who argue sexual orientation is fixed or unchanging often overstate their case.”). In contrast, the monolithic condition does not single out one side in this way, but states: “There is no scientific consensus. . . .” I wonder whether statements that the winning parties “overstate” their case or “ignore” evidence are necessary to the aporetic reasoning. It seems that, for the sake of maintaining the highest degree of similarity between conditions, the aporetic opinion should simply say “The evidence is uncertain as to whether. . . .” Aside from uniformity, I am concerned is that these words might be read as accusing the prevailing side of being disingenuous. One party overstating its case has nothing to do with the court’s aporetic reasoning, but it could heighten the losing side’s suspicion for the winning side’s claims.
- Explaining what’s at stake before the aporia manipulation: The prompt in this survey tells subjects that immutability determines the degree of constitutional protection afforded same-sex couples, but it does not explicitly say that the degree of constitutional protection determines whether laws prohibiting same-sex marriage are constitutional. It seems this connection—immutability effectively determines the constitutionality of laws prohibiting same-sex marriage—should be made explicit before the aporetic statement about immutability. It seems that priming readers with the cultural significance of the court’s reasoning about immutability would enhance the tendency to engage in motivated reasoning, and this would increase the effects we’d expect to see.
In raising these questions, I do not mean to undermine the value of Robinson’s study. To the contrary, I find it very valuable. Not only is it encouraging in that it suggests this question is worth studying further, it also supplies an inspiring baseline for designing another study on this subject.