Branding/Logomark minus Citation Combined Shape Icon/Bookmark-empty Icon/Copy Icon/Collection Icon/Close Copy 7 no author result Created with Sketch. Icon/Back Created with Sketch. Match!

Correlational Effect Size Benchmarks

Published on Jan 1, 2015in Journal of Applied Psychology 4.64
· DOI :10.1037/a0038047
Frank A. Bosco11
Estimated H-index: 11
(VCU: Virginia Commonwealth University),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington)
+ 2 AuthorsCharles A. Pierce30
Estimated H-index: 30
(U of M: University of Memphis)
Abstract
Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We extracted 147,328 correlations and developed a hierarchical taxonomy of variables reported in Journal of Applied Psychology and Personnel Psychology from 1980 to 2010 to produce empirical effect size benchmarks at the omnibus level, for 20 common research domains, and for an even finer grained level of generality. Results indicate that the usual interpretation and classification of effect sizes as small, medium, and large bear almost no resemblance to findings in the field, because distributions of effect sizes exhibit tertile partitions at values approximately one-half to one-third those intuited by Cohen (1988). Our results offer information that can be used for research planning and design purposes, such as producing better informed non-nil hypotheses and estimating statistical power and planning sample size accordingly. We also offer information useful for understanding the relative importance of the effect sizes found in a particular study in relationship to others and which research domains have advanced more or less, given that larger effect sizes indicate a better understanding of a phenomenon. Also, our study offers information about research domains for which the investigation of moderating effects may be more fruitful and provide information that is likely to facilitate the implementation of Bayesian analysis. Finally, our study offers information that practitioners can use to evaluate the relative effectiveness of various types of interventions.
  • References (58)
  • Citations (144)
Cite
References58
Newest
Published on Aug 1, 2014in Organizational psychology review 3.07
Jeffrey R. Edwards46
Estimated H-index: 46
(UNC: University of North Carolina at Chapel Hill),
Michael S. Christian16
Estimated H-index: 16
(UNC: University of North Carolina at Chapel Hill)
In organizational psychology research, most theories put forth directional predictions, such as stating that an increase in one construct will result in an increase or decrease in another construct. Such predictions are imprecise, given that they can be confirmed by a wide range of values, and theories that rely on such predictions bear little risk of falsification. In this article, we present an approach for increasing theoretical precision by using results from meta-analyses to calibrate the p...
3 Citations Source Cite
Published on Jan 1, 2014in Journal of Applied Psychology 4.64
Margaret E. Brooks7
Estimated H-index: 7
,
Dev K. Dalal9
Estimated H-index: 9
,
Kevin P. Nolan3
Estimated H-index: 3
Communicating the results of research to nonscientists presents many challenges. Among these challenges is communicating the effectiveness of an intervention in a way that people untrained in statistics can understand. Use of traditional effect size metrics (e.g., r, r²) has been criticized as being confusing to general audiences. In response, researchers have developed nontraditional effect size indicators (e.g., binomial effect size display, common language effect size indicator) with the goal...
14 Citations Source Cite
Published on Oct 1, 2012in Organizational Research Methods 4.92
John K. Kruschke33
Estimated H-index: 33
(IU: Indiana University Bloomington),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington),
Harry Joo11
Estimated H-index: 11
(IU: Indiana University Bloomington)
The use of Bayesian methods for data analysis is creating a revolution in fields ranging from genetics to marketing. Yet, results of our literature review, including more than 10,000 articles published in 15 journals from January 2001 and December 2010, indicate that Bayesian approaches are essentially absent from the organizational sciences. Our article introduces organizational science researchers to Bayesian methods and describes why and how they should be used. We use multiple linear regress...
197 Citations Source Cite
Published on Jan 1, 2012in Journal of Applied Psychology 4.64
John E. Mathieu57
Estimated H-index: 57
(UConn: University of Connecticut),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington)
+ 1 AuthorsGilad Chen35
Estimated H-index: 35
(UMD: University of Maryland, College Park)
Cross-level interaction effects lie at the heart of multilevel contingency and interactionism theories. Researchers have often lamented the difficulty of finding hypothesized cross-level interactions, and to date there has been no means by which the statistical power of such tests can be evaluated. We develop such a method and report results of a large-scale simulation study, verify its accuracy, and provide evidence regarding the relative importance of factors that affect the power to detect cr...
161 Citations Source Cite
Published on Jan 1, 2012in Psychological Methods 6.49
Ken Kelley25
Estimated H-index: 25
(Mendoza College of Business),
Kristopher J. Preacher21
Estimated H-index: 21
(Vandy: Vanderbilt University)
The call for researchers to report and interpret effect sizes and their corresponding confidence intervals has never been stronger. However, there is confusion in the literature on the definition of effect size, and consequently the term is used inconsistently. We propose a definition for effect size, discuss 3 facets of effect size (dimension, measure/index, and value), outline 10 corollaries that follow from our definition, and review ideal qualities of effect sizes. Our definition of effect s...
284 Citations Source Cite
Published on Jan 1, 2012in Occupational Therapy in Health Care
Kate E. DeCleene1
Estimated H-index: 1
(College of Health Sciences, Bahrain),
Jennifer L. Fogo2
Estimated H-index: 2
(College of Health Sciences, Bahrain)
1,093 Citations Source Cite
Published on Jan 1, 2012in Organizational Research Methods 4.92
Kevin D. Carlson11
Estimated H-index: 11
,
Andrew O. Herdman7
Estimated H-index: 7
Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findi...
78 Citations Source Cite
Published on Jul 12, 2011
Geoff Cumming30
Estimated H-index: 30
Preface. About this Book 1. Introduction to The New Statistics 2. From Null Hypothesis Significance Testing to Effect Sizes 3. Confidence Intervals 4. Confidence Intervals, Error Bars, and p Values 5. Replication 6. Two Simple Designs 7. Meta-Analysis 1: Introduction and Forest Plots 8. Meta-Analysis 2: Models 9. Meta-Analysis 3: Large-Scale Analyses 10. The Noncentral t Distribution 11. Cohen's d 12. Power 13. Precision for Planning 14. Correlations, Proportions, and Further Effect Size Measure...
787 Citations
Published on Apr 1, 2011in Organizational Research Methods 4.92
Jose M. Cortina28
Estimated H-index: 28
,
Ronald S. Landis21
Estimated H-index: 21
Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This article examines the myth that uncritical replacement of NHST with EST will improve our science. The use of NHST and EST is described along with a summary of the arguments offered in support and against both. After addressing the veracity of these assertions, the article describes the concept of the tra...
33 Citations Source Cite
Published on Jan 1, 2011in Journal of Management 8.08
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington),
Dan R. Dalton59
Estimated H-index: 59
(IU: Indiana University Bloomington)
+ 2 AuthorsCatherine M. Dalton16
Estimated H-index: 16
(IU: Indiana University Bloomington)
The authors content analyzed 196 meta-analyses including 5,581 effect-size estimates published in Academy of Management Journal, Journal of Applied Psychology, Journal of Management, Personnel Psychology, and Strategic Management Journal from January 1982 through August 2009 to assess the presumed effects of each of 21 methodological choices and judgment calls on substantive conclusions. Results indicate that, overall, the various meta-analytic methodological choices available and judgment calls...
111 Citations Source Cite
Cited By144
Newest
Published on May 24, 2019in Ergonomics 2.02
Olga Kombeiz (Lboro: Loughborough University), Erik Dietl3
Estimated H-index: 3
(Lboro: Loughborough University)
AbstractWorkplace illumination is known to impact mood, performance and decision making. Based on the idea that positive feelings associated with light might influence social judgments in workplaces, we propose that satisfaction with light as a specific affective response to light would lead to positive judgments of other individuals. In a laboratory experiment (N = 164), participants assessed their satisfaction with light and rated other person’s faces on warmth and competence. Results showed t...
Source Cite
Published on Jun 22, 2018in Journal of Business and Psychology 2.58
Wayne S. Crawford3
Estimated H-index: 3
(UTA: University of Texas at Arlington),
K. Michele Kacmar9
Estimated H-index: 9
(Texas State University),
Kenneth J. Harris29
Estimated H-index: 29
(Indiana University Southeast)
Based on the idea that both actor and audience member are present in impression management (IM), we argue that the effectiveness of IM usage can only be determined when ratings from both the actor and the audience are considered. Further, we use self-verification theory to explain how IM incongruence may impact workplace outcomes. To test our arguments, we employed congruence analysis (Cheung in Organizational Research Methods 12, 6–33, 2009a). Our approach differs from the majority of extant IM...
Source Cite
Published on Mar 15, 2018in Organizational Research Methods 4.92
Christopher D. Nye14
Estimated H-index: 14
(MSU: Michigan State University),
Jacob Bradburn2
Estimated H-index: 2
(MSU: Michigan State University)
+ 2 AuthorsFritz Drasgow43
Estimated H-index: 43
(UIUC: University of Illinois at Urbana–Champaign)
Recently, an effect size measure, known as dMACS, was developed for confirmatory factor analytic (CFA) studies of measurement equivalence. Although this index has several advantages over traditional methods of identifying nonequivalence, the scale and interpretation of this effect size are still unclear. As a result, the interpretation of the effect size is left to the subjective judgment of the researcher. To remedy this issue for other effect sizes, some have proposed guidelines for evaluating...
4 Citations Source Cite
Published on Jun 1, 2019in Personnel Psychology 5.52
Nicolas Roulin1
Estimated H-index: 1
(Saint Mary's University),
Julia Levashina9
Estimated H-index: 9
(College of Business Administration)
Source Cite