domains / families / research / 06-phack-detection

06-phack-detection

separate

Test whether a body of reported results shows p-hacking, selective reporting or selection between stages, using the p-curve battery and the threshold and across-stages tests of Adda, Decker and Ottaviani (2020). Implements the binomial, Fisher, Stouffer and least-concave-majorant monotonicity tests of Elliott, Kudrin and Wuethrich (2022); threshold-bunching tests calibrated against a smooth counterfactual density; a Cattaneo-Jansson-Ma density-jump test and a right-side spike test that tell a level shift (results withheld) from a spike (results pushed across); the phase II vs phase III comparison and the selective-continuation decomposition; and Simonsohn's p-curve power estimate. Use when asked whether a literature, a journal, a research design, an author, a registry or a set of agent runs shows evidence of p-hacking, publication bias or selective continuation.

When to use

• detecting p-hacking
• detecting selective reporting
• detecting selection between stages
• analyzing p-curve shape
• testing threshold bunching
• estimating p-curve power
• comparing phase II vs phase III results

When not to use

• establishing that any individual paper was p-hacked

Skill page

Source