Branding/Logomark minus Citation Combined Shape Icon/Bookmark-empty Icon/Copy Icon/Collection Icon/Close Copy 7 no author result Created with Sketch. Icon/Back Created with Sketch. Match!

STUDY QUALITY IN SLA: An Assessment of Designs, Analyses, and Reporting Practices in Quantitative L2 Research

Published on Dec 1, 2013in Studies in Second Language Acquisition 2.60
· DOI :10.1017/s0272263113000399
Luke D Plonsky19
Estimated H-index: 19
(NAU: Northern Arizona University)
Cite
Abstract
This study assesses research and reporting practices in quantitative second language (L2) research. A sample of 606 primary studies, published from 1990 to 2010 in Language Learning and Studies in Second Language Acquisition , was collected and coded for designs, statistical analyses, reporting practices, and outcomes (i.e., effect sizes). The results point to several systematic strengths as well as many flaws, such as a lack of control in experimental designs, incomplete and inconsistent reporting practices, and low statistical power. I discuss these trends, strengths, and weaknesses in comparison with methodological reviews of L2 research (e.g., Plonsky & Gass, 2011 ) as well as reviews from other fields (e.g., education, Skidmore & Thompson, 2010 ). On the basis of the findings, I also make a number of suggestions for methodological reforms in applied linguistics.
  • References (92)
  • Citations (60)
Cite
References92
Newest
Published on Feb 21, 2012
Luke D Plonsky19
Estimated H-index: 19
(MSU: Michigan State University),
Frederick L. Oswald33
Estimated H-index: 33
(Rice University)
Published on Dec 1, 2011in Language Learning 2.00
Luke D Plonsky19
Estimated H-index: 19
(NAU: Northern Arizona University)
Research on the effects of second language strategy instruction (SI) has been extensive yet inconclusive. This meta-analysis, therefore, aims to provide a reliable, quantitative measure of the effect of SI as well as a description of the relationship between SI and the variables that moderate its effectiveness (i.e., different learning contexts, treatments, and outcome variables). A comprehensive search was conducted to collect the population of SI studies. Effect sizes were calculated for 61 pr...
Published on Nov 14, 2011
John M. Norris19
Estimated H-index: 19
,
Lourdes Ortega23
Estimated H-index: 23
Published on Nov 11, 2011
Alison Mackey36
Estimated H-index: 36
(Georgetown University),
Susan M. Gass47
Estimated H-index: 47
List of Contributors vii 1 Introduction 1 Alison Mackey and Susan M. Gass Part I Data Types 5 2 How to Use Foreign and Second Language Learner Corpora 7 Sylviane Granger 3 Formal Theory-Based Methodologies 30 Tania Ionin 4 Instructed Second Language Acquisition 53 Shawn Loewen and Jenefer Philp 5 How to Design and Analyze Surveys in Second Language Acquisition Research 74 Zoltan Dornyei and Kata Csizer 6 How to Carry Out Case Study Research 95 Patricia A. Duff 7 How to Use Psycholinguistic Metho...
Published on Jun 1, 2011in Language Learning 2.00
Luke D Plonsky19
Estimated H-index: 19
(MSU: Michigan State University),
Susan M. Gass47
Estimated H-index: 47
(MSU: Michigan State University)
This article constitutes the first empirical assessment of methodological quality in second language acquisition (SLA). We surveyed a corpus of 174 studies (N = 7,951) within the tradition of research on second-language interaction, one of the longest and most influential traditions of inquiry in SLA. Each report was coded for methodological features, statistical analyses, and reporting practices associated with research quality, and the resulting data were examined both cumulatively and over ti...
Published on Jan 1, 2011
Luke Plonsky1
Estimated H-index: 1
STUDY QUALITY IN SLA: A CUMULATIVE AND DEVELOPMENTAL ASSESSMENT OF DESIGNS, ANALYSES, REPORTING PRACTICES, AND OUTCOMES IN QUANTITATIVE L2 RESEARCH By Luke Plonsky I began this study with two assumptions. Assumption 1: Study quality matters. If the means by which researchers design, carry out, and report on their studies lack in rigor or transparency, theory and practice are likely to be misguided or at least decelerated. Assumption 2 is an implication of Assumption 1: Quality should be measured...
Published on Dec 1, 2010in Personnel Psychology 6.93
Stéphane Brutus17
Estimated H-index: 17
(Concordia University),
Harjinder Gill5
Estimated H-index: 5
(U of G: University of Guelph),
Kris Duniewicz2
Estimated H-index: 2
(FIU: Florida International University)
This study investigated self-reported limitations of published papers as an alternative and novel operationalization of the state of science of industrial and organizational (I-O) psychology. A content analysis was conducted of the reported limitations in every I-O psychology articles published in the Academy of Management Journal, the Journal of Applied Psychology, and Personnel Psychology between 1995 and 2008 (N= 2,402). Articles were coded for the number and types of limitations reported, ch...
Published on Oct 1, 2010in Educational and Psychological Measurement 2.05
Susan Troncoso Skidmore7
Estimated H-index: 7
(UTPA: University of Texas–Pan American),
Bruce Thompson55
Estimated H-index: 55
(BCM: Baylor College of Medicine)
The purpose of the present study is to provide a historical account and metasynthesis of which statistical techniques are most frequently used in the fields of education and psychology. Six articles reviewing the American Educational Research Journal from 1969 to 1997 and five articles reviewing the psychological literature from 1948 to 2001 resulted in a total number of 17,698 techniques recorded from the 12,012 articles reviewed. No prior study of analytic practices has considered this broad s...
Published on Jul 1, 2010in Applied Linguistics 3.04
Jenifer Larson-Hall7
Estimated H-index: 7
(UNT: University of North Texas),
Richard Herrington2
Estimated H-index: 2
(UNT: University of North Texas)
In this article we introduce language acquisition researchers to two broad areas of applied statistics that can improve the way data are analyzed. First we argue that visual summaries of information are as vital as numerical ones, and suggest ways to improve them. Specifically, we recommend choosing boxplots over barplots and adding locally weighted smooth lines (Loess lines) to scatterplots. Second, we introduce the reader to robust statistics, a tool that can provide a way to use the power of ...
Published on Mar 1, 2010in Annual Review of Applied Linguistics 2.48
Frederick L. Oswald33
Estimated H-index: 33
(Rice University),
Luke D Plonsky19
Estimated H-index: 19
(MSU: Michigan State University)
Applied linguists are increasingly conducting meta-analysis in their substantive domains, because as a quantitative approach for averaging effect sizes across studies, it is more systematic and replicable than traditional, qualitative literature reviews. Additional strengths, such as increased statistical power, moderator analyses, and model testing, have also contributed to its appeal. The current review describes typical stages of a meta-analysis in second language acquisition (SLA) research: ...
Cited By60
Newest
Published on Sep 1, 2019in Journal of Second Language Writing 4.20
Meixiu Zhang (NAU: Northern Arizona University)
Abstract Understanding the nature of collaboration is critical in collaborative writing (CW), as it impacts the amount of scaffolding that occurs and the amount of linguistic knowledge that can be retained (Storch, 2013). The most prevalent model to examine peer collaboration in CW is based on a global qualitative analysis of learners’ involvement in and control over a writing task (Storch, 2001a). However, this model does not account for the fluctuating nature of peer collaboration in CW. This ...
Fahimeh Marefat2
Estimated H-index: 2
(Allameh Tabataba'i University),
Abdulbaset Saeedian (Allameh Tabataba'i University)+ 4 AuthorsJavad Ahmadi Fatalaki1
Estimated H-index: 1
(Allameh Tabataba'i University)
Published on Jul 11, 2018in Language Teaching Research 2.32
Sarah Sok1
Estimated H-index: 1
(UCI: University of California, Irvine),
Eun Young Kang2
Estimated H-index: 2
(CUNY: City University of New York),
ZhaoHong Han16
Estimated H-index: 16
(Columbia University)
This article offers a methodological synthesis spanning 35 years of instructed second language acquisition research on the efficacy of form-focused instruction. Eighty-eight (quasi-) experimental studies were sampled from six academic journals: Applied Linguistics, Language Learning, Language Teaching Research, Modern Language Journal, Studies in Second Language Acquisition, and TESOL Quarterly. The focus of analysis was on five sets of variables: (1) research design, (2) participants, (3) conte...
Published on Jun 1, 2018in Language Teaching Research 2.32
Heidi Byrnes21
Estimated H-index: 21
(Georgetown University)
Offered as a conceptual and programmatic piece, this article suggests that, due to its explicit educational orientation, the domain of instructed second language acquisition (ISLA) is challenged to align theoretical choices, research preferences, and educational practices in the interest of improving instructed L2 learning. It addresses the current disjuncture by proposing the constructs of ‘development’ along with ‘curricular thinking’, particularly when they are informed by complexity theory, ...
Published on Feb 19, 2019in Studies in Second Language Acquisition 2.60
Yasser Teimouri2
Estimated H-index: 2
(Georgetown University),
Julia Goetze (WashU: Washington University in St. Louis), Luke D Plonsky19
Estimated H-index: 19
Second language (L2) anxiety has been the object of constant empirical and theoretical attention for several decades. As a matter of both theoretical and practical interest, much of the research in this domain has examined the relationship between anxiety and L2 achievement. The present study meta-analyzes this body of research. Following a comprehensive search, a sample of 97 reports were identified, contributing a total of 105 independent samples ( N = 19,933) from 23 countries. In the aggrega...
Talip Gonulal2
Estimated H-index: 2
Abstract The present study investigated second language acquisition (SLA) doctoral students’ statistical training and knowledge of statistics. One hundred and twenty SLA doctoral students in North America took a comprehensive statistics survey, and 16 of them participated in follow-up interviews. The results showed that doctoral students were well trained in basic descriptive statistics, while their training in inferential statistics, particularly advanced statistics, was limited. When looking a...
Published on Apr 1, 2019in SAGE Open
Rining Wei (Xi'an Jiaotong-Liverpool University), Yuhang Hu (Georgetown University), Jianhui Xiong (MOE: Chinese Ministry of Education)
Many surveys of effect size (ES) reporting practices have been conducted in social science fields such as psychology and education, but few such studies are available in applied linguistics. To bridge this gap and to echo the recent calls for more robust statistics from scholars in applied linguistics and beyond, this study represents the first attempt, in the field of applied linguistics, to focus upon ES reporting practices. With an innovative “two-standards” approach for coding, which overcom...
Published on Feb 1, 2019in System 1.93
Yiran Xu (Georgetown University)
Abstract Despite the growing interest in interlanguage complexity development in study abroad (SA) research, no clear-cut conclusions can be made as to whether and to what extent learners' interlanguage complexity increases following a sojourn abroad. The current study meta-analyzed the overall effects of study abroad on measured oral and written complexity, as well as the moderator effects (i.e., learner demographics, SA contextual features, and outcome measures) on the variability of interlang...