Correlational Effect Size Benchmarks

Published on Jan 1, 2015in Journal of Applied Psychology5.07
· DOI :10.1037/a0038047
Frank A. Bosco11
Estimated H-index: 11
(VCU: Virginia Commonwealth University),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington)
+ 2 AuthorsCharles A. Pierce30
Estimated H-index: 30
(U of M: University of Memphis)
Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We extracted 147,328 correlations and developed a hierarchical taxonomy of variables reported in Journal of Applied Psychology and Personnel Psychology from 1980 to 2010 to produce empirical effect size benchmarks at the omnibus level, for 20 common research domains, and for an even finer grained level of generality. Results indicate that the usual interpretation and classification of effect sizes as small, medium, and large bear almost no resemblance to findings in the field, because distributions of effect sizes exhibit tertile partitions at values approximately one-half to one-third those intuited by Cohen (1988). Our results offer information that can be used for research planning and design purposes, such as producing better informed non-nil hypotheses and estimating statistical power and planning sample size accordingly. We also offer information useful for understanding the relative importance of the effect sizes found in a particular study in relationship to others and which research domains have advanced more or less, given that larger effect sizes indicate a better understanding of a phenomenon. Also, our study offers information about research domains for which the investigation of moderating effects may be more fruitful and provide information that is likely to facilitate the implementation of Bayesian analysis. Finally, our study offers information that practitioners can use to evaluate the relative effectiveness of various types of interventions.
  • References (58)
  • Citations (144)
Published on Aug 1, 2014in Organizational psychology review4.11
Jeffrey R. Edwards46
Estimated H-index: 46
(UNC: University of North Carolina at Chapel Hill),
Michael S. Christian16
Estimated H-index: 16
(UNC: University of North Carolina at Chapel Hill)
In organizational psychology research, most theories put forth directional predictions, such as stating that an increase in one construct will result in an increase or decrease in another construct. Such predictions are imprecise, given that they can be confirmed by a wide range of values, and theories that rely on such predictions bear little risk of falsification. In this article, we present an approach for increasing theoretical precision by using results from meta-analyses to calibrate the p...
Published on Jan 1, 2014in Journal of Applied Psychology5.07
Margaret E. Brooks7
Estimated H-index: 7
Dev K. Dalal9
Estimated H-index: 9
Kevin P. Nolan3
Estimated H-index: 3
Communicating the results of research to nonscientists presents many challenges. Among these challenges is communicating the effectiveness of an intervention in a way that people untrained in statistics can understand. Use of traditional effect size metrics (e.g., r, r²) has been criticized as being confusing to general audiences. In response, researchers have developed nontraditional effect size indicators (e.g., binomial effect size display, common language effect size indicator) with the goal...
Published on Oct 1, 2012in Organizational Research Methods6.55
John K. Kruschke33
Estimated H-index: 33
(IU: Indiana University Bloomington),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington),
Harry Joo11
Estimated H-index: 11
(IU: Indiana University Bloomington)
The use of Bayesian methods for data analysis is creating a revolution in fields ranging from genetics to marketing. Yet, results of our literature review, including more than 10,000 articles published in 15 journals from January 2001 and December 2010, indicate that Bayesian approaches are essentially absent from the organizational sciences. Our article introduces organizational science researchers to Bayesian methods and describes why and how they should be used. We use multiple linear regress...
Published on Jan 1, 2012in Journal of Applied Psychology5.07
John E. Mathieu57
Estimated H-index: 57
(UConn: University of Connecticut),
Herman Aguinis54
Estimated H-index: 54
(IU: Indiana University Bloomington)
+ 1 AuthorsGilad Chen35
Estimated H-index: 35
(UMD: University of Maryland, College Park)
Cross-level interaction effects lie at the heart of multilevel contingency and interactionism theories. Researchers have often lamented the difficulty of finding hypothesized cross-level interactions, and to date there has been no means by which the statistical power of such tests can be evaluated. We develop such a method and report results of a large-scale simulation study, verify its accuracy, and provide evidence regarding the relative importance of factors that affect the power to detect cr...
Published on Jan 1, 2012in Psychological Methods8.19
Ken Kelley25
Estimated H-index: 25
(Mendoza College of Business),
Kristopher J. Preacher21
Estimated H-index: 21
(Vandy: Vanderbilt University)
The call for researchers to report and interpret effect sizes and their corresponding confidence intervals has never been stronger. However, there is confusion in the literature on the definition of effect size, and consequently the term is used inconsistently. We propose a definition for effect size, discuss 3 facets of effect size (dimension, measure/index, and value), outline 10 corollaries that follow from our definition, and review ideal qualities of effect sizes. Our definition of effect s...
Published on Jan 1, 2012in Occupational Therapy in Health Care
Kate E. DeCleene1
Estimated H-index: 1
(College of Health Sciences, Bahrain),
Jennifer L. Fogo2
Estimated H-index: 2
(College of Health Sciences, Bahrain)
Published on Jan 1, 2012in Organizational Research Methods6.55
Kevin D. Carlson11
Estimated H-index: 11
Andrew O. Herdman7
Estimated H-index: 7
Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findi...
Published on Jul 12, 2011
Geoff Cumming30
Estimated H-index: 30
Preface. About this Book 1. Introduction to The New Statistics 2. From Null Hypothesis Significance Testing to Effect Sizes 3. Confidence Intervals 4. Confidence Intervals, Error Bars, and p Values 5. Replication 6. Two Simple Designs 7. Meta-Analysis 1: Introduction and Forest Plots 8. Meta-Analysis 2: Models 9. Meta-Analysis 3: Large-Scale Analyses 10. The Noncentral t Distribution 11. Cohen's d 12. Power 13. Precision for Planning 14. Correlations, Proportions, and Further Effect Size Measure...
Published on Apr 1, 2011in Organizational Research Methods6.55
Jose M. Cortina28
Estimated H-index: 28
Ronald S. Landis21
Estimated H-index: 21
Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This article examines the myth that uncritical replacement of NHST with EST will improve our science. The use of NHST and EST is described along with a summary of the arguments offered in support and against both. After addressing the veracity of these assertions, the article describes the concept of the tra...
Published on Jan 1, 2011in Journal of Applied Psychology5.07
Winny Shen13
Estimated H-index: 13
(UMN: University of Minnesota),
Thomas Kiger5
Estimated H-index: 5
(UMN: University of Minnesota)
+ 3 AuthorsDeniz S. Ones46
Estimated H-index: 46
This study examines sample characteristics of articles published in Journal of Applied Psychology (JAP )from 1995 to 2008. At the individual level, the overall median sample size over the period examined wasapproximately 173, which is generally adequate for detecting the average magnitude of effects of primaryinterest to researchers who publish in JAP . Samples using higher units of analyses (e.g., teams,departments/work units, and organizations) had lower median sample sizes ( Mdn 65), yet were...
Cited By144
Published on May 1, 2019in Psychology of Sport and Exercise2.71
Niklas K. Steffens16
Estimated H-index: 16
(UQ: University of Queensland),
Emma Slade (UQ: University of Queensland)+ 2 AuthorsTim Rees19
Estimated H-index: 19
(BU: Bournemouth University)
This research examined how identity leadership displayed by group exercise instructors is associated with exercisers’ class attendance and in-class effort. Group exercise participants assessed their instructors’ engagement in identity leadership at baseline before indicating their comfort in the exercise environment, identification with the exercise group, class attendance, and in-class effort four weeks later. Results indicated positive associations between instructors’ identity leadership and ...
Published on Oct 1, 2019in Cognition3.54
Christin Köber6
Estimated H-index: 6
(NYUAD: New York University Abu Dhabi),
Christopher Facompre1
Estimated H-index: 1
(SBU: Stony Brook University)
+ 1 AuthorsJeffry A. Simpson63
Estimated H-index: 63
(UMN: University of Minnesota)
Abstract Increasing evidence suggests that both attachment representations and autobiographical memories are moderately stable over time. Evidence examining the stability of attachment-related memories is scarce, although these memories of early caregiving are thought to underpin attachment representations. Connecting research on stability of autobiographical memories with research on attachment representation, the present study investigated the stability of attachment-related autobiographical m...
Published on 2019in Psychological Medicine5.64
Kirsty Lee5
Estimated H-index: 5
Tracy Vaillancourt38
Estimated H-index: 38
Published on May 18, 2019in Journal of the Academy of Marketing Science9.36
Jeremy S. Wolter4
Estimated H-index: 4
(AU: Auburn University),
Dora E. Bock5
Estimated H-index: 5
(AU: Auburn University)
+ 2 AuthorsJeffery S. Smith (VCU: Virginia Commonwealth University)
Does improving employee happiness affect customer outcomes? The current study attempts to answer this question by examining the impact of employee satisfaction trajectories (i.e., systematic changes in employee satisfaction) on customer outcomes. After accounting for employees’ initial satisfaction levels, the analyses demonstrate the importance of employee satisfaction trajectories for customer satisfaction and repatronage intentions, as well as identify customer-employee contact as a necessary...
Published on Jul 12, 2019in Journal of Social Issues2.42
Jenna A. Harder3
Estimated H-index: 3
(MSU: Michigan State University),
Victor N. Keller4
Estimated H-index: 4
(MSU: Michigan State University),
William J. Chopik14
Estimated H-index: 14
(MSU: Michigan State University)
Published on Sep 1, 2019in BMJ Open2.38
Jan B. Schmutz (NU: Northwestern University), Laurenz L. Meier27
Estimated H-index: 27
Tanja Manser27
Estimated H-index: 27
(University of Applied Science and Arts Northwestern Switzerland FHNW)
Objectives To investigate the relationship between teamwork and clinical performance and potential moderating variables of this relationship. Design Systematic review and meta-analysis. Data source PubMed was searched in June 2018 without a limit on the date of publication. Additional literature was selected through a manual backward search of relevant reviews, manual backward and forward search of studies included in the meta-analysis and contacting of selected authors via email. Eligibility cr...
Published on 2019in Marketing Letters1.62
Chao Miao8
Estimated H-index: 8
(SU: Salisbury University),
Michael J. Barone17
Estimated H-index: 17
(University of Louisville)
+ 1 AuthorsRonald H. Humphrey22
Estimated H-index: 22
(Lancaster University)
In an increasingly competitive market economy, retailers are seeking ways to manage customer perceptions of their service quality. Selecting employees who are high on emotional intelligence (EI), and training employees in emotional competencies, may be ways to improve service quality. This meta-analysis tests the claims that EI improves service quality. The findings indicate that EI is significantly and positively related to service quality and that this relationship is stronger (1) for cultures...
Published on May 24, 2019in Ergonomics2.18
Olga Kombeiz (Lboro: Loughborough University), Erik Dietl3
Estimated H-index: 3
(Lboro: Loughborough University)
AbstractWorkplace illumination is known to impact mood, performance and decision making. Based on the idea that positive feelings associated with light might influence social judgments in workplaces, we propose that satisfaction with light as a specific affective response to light would lead to positive judgments of other individuals. In a laboratory experiment (N = 164), participants assessed their satisfaction with light and rated other person’s faces on warmth and competence. Results showed t...
Published on Jun 22, 2018in Journal of Business and Psychology2.58
Wayne S. Crawford3
Estimated H-index: 3
(UTA: University of Texas at Arlington),
K. Michele Kacmar9
Estimated H-index: 9
(Texas State University),
Kenneth J. Harris29
Estimated H-index: 29
(Indiana University Southeast)
Based on the idea that both actor and audience member are present in impression management (IM), we argue that the effectiveness of IM usage can only be determined when ratings from both the actor and the audience are considered. Further, we use self-verification theory to explain how IM incongruence may impact workplace outcomes. To test our arguments, we employed congruence analysis (Cheung in Organizational Research Methods 12, 6–33, 2009a). Our approach differs from the majority of extant IM...
Published on Jul 19, 2019
Marc Brysbaert57
Estimated H-index: 57
(UGent: Ghent University)
Given that an effect size of d = .4 is a good first estimate of the smallest effect size of interest in psychological research, we already need over 50 participants for a simple comparison of two within-participants conditions if we want to run a study with 80% power. This is more than current practice. In addition, as soon as a between-groups variable or an interaction is involved, numbers of 100, 200, and even more participants are needed. As long as we do not accept these facts, we will keep ...
View next paperA power primer.