Conservative Tests under Satisficing Models of Publication Bias

Published on Feb 22, 2016in PLOS ONE2.78
· DOI :10.1371/journal.pone.0149590
Justin McCrary15
Estimated H-index: 15
(University of California, Berkeley),
Garret Christensen4
Estimated H-index: 4
(University of California, Berkeley),
Daniele Fanelli19
Estimated H-index: 19
(Stanford University)
View in Source
Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. We calculate critical values for statistical significance that could help to adjust after the fact for the distortions created by this selection effect, assuming that the only source of publication bias is file drawer bias. These adjusted critical values are easy to calculate and differ from unadjusted critical values by approximately 50%—rather than rejecting a null hypothesis when the t-ratio exceeds 2, the analysis suggests rejecting a null hypothesis when the t-ratio exceeds 3. Samples of published social science research indicate that on average, across research fields, approximately 30% of published t-statistics fall between the standard and adjusted cutoffs.
  • References (53)
  • Citations (4)
#1Eva VivaltH-Index: 1
17 Citations
#1Abel Brodeur (U of O: University of Ottawa)H-Index: 4
#2Mathias Lé (PSE: Paris School of Economics)H-Index: 2
Last.Yanos ZylberbergH-Index: 4
view all 4 authors...
98 CitationsSource
#1Michal Krawczyk (University of Warsaw)H-Index: 10
13 CitationsSource
#1Alexander A. AartsH-Index: 1
#2Joanna E. Anderson (DRDC: Defence Research and Development Canada)H-Index: 3
Last.Kellylynn Zuni (Adams State University)H-Index: 2
view all 270 authors...
2,235 CitationsSource
#1Gregory Francis (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 21
#2Jay TanzmanH-Index: 1
Last.William J. Matthews (University of Cambridge)H-Index: 22
view all 3 authors...
30 CitationsSource
#1Katrin Auspurg (Goethe University Frankfurt)H-Index: 12
#2Thomas HinzH-Index: 15
Last.Andreas Schneck (Goethe University Frankfurt)H-Index: 1
view all 3 authors...
5 CitationsSource
46 CitationsSource
#1Annie Franco (Stanford University)H-Index: 4
#2Neil Malhotra (Stanford University)H-Index: 26
Last.Gabor Simonovits (Stanford University)H-Index: 6
view all 3 authors...
342 CitationsSource
#1Tommy Stanley (Hendrix College)H-Index: 32
#2Hristos Doucouliagos (Deakin University)H-Index: 30
191 CitationsSource
#1Uri Simonsohn (UPenn: University of Pennsylvania)H-Index: 26
#2Leif D. Nelson (University of California, Berkeley)H-Index: 26
Last.Joseph P. Simmons (UPenn: University of Pennsylvania)H-Index: 24
view all 3 authors...
457 CitationsSource
Cited By4
18 CitationsSource
#1Christopher Winship (Harvard University)H-Index: 33
#2Xiaolin Zhuo (Harvard University)
View next paperEstimating the proportion of studies missing for meta-analysis due to publication bias