Econometrica

Journal Of The Econometric Society

An International Society for the Advancement of Economic
Theory in its Relation to Statistics and Mathematics

Edited by: Guido W. Imbens • Print ISSN: 0012-9682 • Online ISSN: 1468-0262

Econometrica: Mar, 2022, Volume 90, Issue 2

Detecting p-Hacking

https://doi.org/10.3982/ECTA18583
p. 887-906

Graham Elliott, Nikolay Kudrin, Kaspar Wüthrich

We theoretically analyze the problem of testing for p‐hacking based on distributions of p‐values across multiple studies. We provide general results for when such distributions have testable restrictions (are non‐increasing) under the null of no p‐hacking. We find novel additional testable restrictions for p‐values based on t‐tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of p‐values. These testable restrictions result in more powerful tests for the null hypothesis of no p‐hacking. When there is also publication bias, our tests are joint tests for p‐hacking and publication bias. A reanalysis of two prominent data sets shows the usefulness of our new tests.


Log In To View Full Content

Supplemental Material

Supplement to "Detecting p‐Hacking"

Graham Elliott, Nikolay Kudrin, and Kaspar Wüthrich

This zip file contains the replication files for the manuscript.