Episode
P-Values: Are we using a flawed statistical tool?
- Published
- Sep 22, 2025
- Duration seconds
- 4406
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/normal-curves-sexy-science-serious-statistics-7212156/episodes/p-values-are-we-using-a-flawed-statistical-tool/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/normal-curves-sexy-science-serious-statistics-7212156/p-values-are-we-using-a-flawed-statistical-tool.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
P-values show up in almost every scientific paper, yet they’re one of the most misunderstood ideas in statistics. In this episode, we break from our usual journal-club format to unpack what a p-value really is, why researchers have fought about it for a century, and how that famous 0.05 cutoff became enshrined in science. Along the way, we share stories from our own papers—from a Nature feature that helped reshape the debate to a statistical sleuthing project that uncovered a faulty method in sports science. The result: a behind-the-scenes look at how one statistical tool has shaped the culture of science itself. Statistical topics Bayesian statistics Confidence intervals Effect size vs. statistical significance Fisher’s conception of p-values Frequentist perspective Magnitude-Based Inference (MBI) Multiple testing / multiple comparisons Neyman-Pearson hypothesis testing framework P-hacking Posterior probabilities Preregistration and registered reports Prior probabilities P-values Researcher degrees of freedom Significance thresholds (p < 0.05) Simulation-based inference Statistical power Statistical significance Transparency in research Type I error (false positive) Type II error (false negative) Winner’s Curse Methodological morals “If p-values tell us the probability the null is true, then octopuses are psychic.” “Statistical tools don't fool us, blind faith in them does.” References Nuzzo R. Scientific method: statistical errors. Nature. 2014 Feb 13;506(7487):150-2. doi: 10.1038/506150a. Nuzzo, R., 2015. Scientists perturbed by loss of stat tools to sift research fudge from fact . Scientific American , pp.16-18. Nuzzo RL. The inverse fallacy and interpreting P values . PM&R. 2015 Mar;7(3):311-4. doi: 10.1016/j.pmrj.2015.02.011. Epub 2015 Feb 25. Nuzzo, R.,…