Econometrica: Mar 2022, Volume 90, Issue 2
Detecting p-Hacking
https://doi.org/10.3982/ECTA18583
p.
887-906
Graham Elliott, Nikolay Kudrin, Kaspar Wüthrich
We theoretically analyze the problem of testing for p‐hacking based on distributions of p‐values across multiple studies. We provide general results for when such distributions have testable restrictions (are non‐increasing) under the null of no p‐hacking. We find novel additional testable restrictions for p‐values based on t‐tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of p‐values. These testable restrictions result in more powerful tests for the null hypothesis of no p‐hacking. When there is also publication bias, our tests are joint tests for p‐hacking and publication bias. A reanalysis of two prominent data sets shows the usefulness of our new tests.
Log In To View Full Content
Supplemental Material
Supplement to "Detecting p‐Hacking"
This zip file contains the replication files for the manuscript.