follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« How should I be updating views on impact of fake news based on new evidence? | Main | Are you curious to see what Financial Times says about curiosity? »

97% (p < 0.01) of social scientists don't agree on p-value threshold

This pre-print responds to the recent Nature Human Behvior article/manifesto (pre-print here) that recommended a "change to P< 0.005" be implemented in "fields where the threshold for defining statistical significance for new discoveries is [now] P < 0.05":

Notwithstanding the conservative, lawyerly tone of the piece ("insufficient evidence ... not strong enough ... evaluated before large-scale changes"), the radical bottom line is in the bottom line: there shouldn't be any single standard for "significance"; rather, researchers should use their reason to identify and explain whatever statistical test they use to guard against type 1 error.

Indeed, if one wants to see a defense of replacing p-values with Bayesian "weight of the evidence" statistics, one should read (or re-read) the Nature Human Behaviour piece, which pictures the p < 0.005 standard as a self-punishing, "the worse the better" historical segue to Bayes Factors.  

So embracing Bayes was the cost of getting 72 scholars to agree to continuing the tyranny of p-values, while disclaiming Bayes was the cost of getting another 88 to agree that p-values shouldn't be treated as a threshold screen for publication.




PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>