Science Forum: Ten common statistical mistakes to watch out for when writing or reviewing a manuscript

Published on Oct 9, 2019in eLife7.551
· DOI :10.7554/ELIFE.48175
Tamar R. Makin21
Estimated H-index: 21
(UCL: University College London),
Jean-Jacques Orban de Xivry19
Estimated H-index: 19
(Katholieke Universiteit Leuven)
: Inspired by broader efforts to make the conclusions of scientific research more robust, we have compiled a list of some of the most common statistical mistakes that appear in the scientific literature. The mistakes have their origins in ineffective experimental designs, inappropriate analyses and/or flawed reasoning. We provide advice on how authors, reviewers and readers can identify and resolve these mistakes and, we hope, avoid them in the future.
  • References (68)
  • Citations (12)
📖 Papers frequently viewed together
5 Citations
523 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Sam Harper (McGill University)H-Index: 33
#2Adam Palayew (McGill University)H-Index: 2
Background Cannabis use has been linked to impaired driving and fatal accidents. Prior evidence suggests the potential for population-wide effects of the annual cannabis celebration on April 20th (‘4/20’), but evidence to date is limited. Methods We used data from the Fatal Analysis Reporting System for the years 1975–2016 to estimate the impact of ‘4/20’ on drivers involved in fatal traffic crashes occurring between 16:20 and 23:59 hours in the USA. We compared the effects of 4/20 with those fo...
3 CitationsSource
3 CitationsSource
Dorothy Bishop describes how threats to reproducibility, recognized but unaddressed for decades, might finally be brought under control. Dorothy Bishop describes how threats to reproducibility, recognized but unaddressed for decades, might finally be brought under control.
21 CitationsSource
#1John L. Kmetz (UD: University of Delaware)H-Index: 2
ABSTRACTp-Values and Null Hypothesis Significance Testing (NHST), combined with a large number of institutional factors, jointly define the Generally Accepted Soft Social Science Publishing Process (GASSSPP) that is now dominant in the social sciences and is increasingly used elsewhere. The case against NHST and the GASSSPP has been abundantly articulated over past decades, and yet it continues to spread, supported by a large number of self-reinforcing institutional processes. In this article, t...
6 CitationsSource
#1Ronald L. Wasserstein (American Statistical Association)H-Index: 4
#2Allen Schirm (Mathematica Policy Research)H-Index: 4
Last. Nicole A. Lazar (UGA: University of Georgia)H-Index: 10
view all 3 authors...
380 CitationsSource
#1Joachim I. Krueger (Brown University)H-Index: 36
#2Patrick R. Heck (GHS: Geisinger Health System)H-Index: 7
ABSTRACTAs the debate over best statistical practices continues in academic journals, conferences, and the blogosphere, working researchers (e.g., psychologists) need to figure out how much time and effort to invest in attending to experts' arguments, how to design their next project, and how to craft a sustainable long-term strategy for data analysis and inference. The present special issue of The American Statistician promises help. In this article, we offer a modest proposal for a continued a...
9 CitationsSource
#1Robert J. Calin-Jageman (Dominican University)H-Index: 10
#2Geoff Cumming (La Trobe University)H-Index: 30
AbstractThe “New Statistics” emphasizes effect sizes, confidence intervals, meta-analysis, and the use of Open Science practices. We present three specific ways in which a New Statistics approach c...
16 CitationsSource
#1Ronald D. Fricker (VT: Virginia Tech)H-Index: 17
#1Ronald D. Fricker (VT: Virginia Tech)H-Index: 1
Last. William H. Woodall (VT: Virginia Tech)H-Index: 56
view all 4 authors...
In this article, we assess the 31 articles published in Basic and Applied Social Psychology (BASP) in 2016, which is one full year after the BASP editors banned the use of inferential statistics. We discuss how the authors collected their data, how they reported and summarized their data, and how they used their data to reach conclusions. We found multiple instances of authors overstating conclusions beyond what the data would support if statistical significance had been considered. Readers woul...
16 CitationsSource
#1Ben Goldacre (University of Oxford)H-Index: 28
#2Henry Drysdale (University of Oxford)H-Index: 3
Last. Kamal R Mahtani (University of Oxford)H-Index: 23
view all 10 authors...
Discrepancies between pre-specified and reported outcomes are an important source of bias in trials. Despite legislation, guidelines and public commitments on correct reporting from journals, outcome misreporting continues to be prevalent. We aimed to document the extent of misreporting, establish whether it was possible to publish correction letters on all misreported trials as they were published, and monitor responses from editors and trialists to understand why outcome misreporting persists ...
31 CitationsSource
#1Tracey L. Weissgerber (Charité)H-Index: 20
#2Oscar Garcia-Valencia (Mayo Clinic)H-Index: 3
Last. Stacey J. Winham (Mayo Clinic)H-Index: 23
view all 5 authors...
: Transparent reporting is essential for the critical evaluation of studies. However, the reporting of statistical methods for studies in the biomedical sciences is often limited. This systematic review examines the quality of reporting for two statistical tests, t-tests and ANOVA, for papers published in a selection of physiology journals in June 2017. Of the 328 original research articles examined, 277 (84.5%) included an ANOVA or t-test or both. However, papers in our sample were routinely mi...
10 CitationsSource
Cited By12
#1Roni Maimon Mor (University of Oxford)H-Index: 1
#2Tamar R. Makin (University of Oxford)H-Index: 21
The potential ability of the human brain to represent an artificial limb as a body part (embodiment) has been inspiring engineers, clinicians, and scientists as a means to optimise human–machine interfaces. Using functional MRI (fMRI), we studied whether neural embodiment actually occurs in prosthesis users’ occipitotemporal cortex (OTC). Compared with controls, different prostheses types were visually represented more similarly to each other, relative to hands and tools, indicating the emergenc...
1 CitationsSource
#1Seoan Huh (NU: Northwestern University)H-Index: 2
#2Cj Heckman (NU: Northwestern University)H-Index: 20
Last. Marin Manuel (University of Paris)
view all 3 authors...
Although ALS is an adult-onset neurodegenerative disease, motoneuron electrical properties are already altered during embryonic development. Motoneurons must therefore exhibit remarkable capacity for homeostatic regulation to maintain a normal motor output for most of the life of the patient. In the present paper, we demonstrate how maintaining homeostasis could come at a very high cost. We studied the excitability of spinal motoneurons from young adult SOD1(G93A) mice to end-stage. Initially ho...
#1Elisabete Ferreiro (UC: University of Coimbra)H-Index: 20
#2Mariagrazia Lanzillo (UC: University of Coimbra)
Last. Paulo S. Pinheiro (UC: University of Coimbra)H-Index: 25
view all 12 authors...
Abstract During aging, lifestyle-related factors shape the brain’s response to insults and modulate the progression of neurodegenerative diseases such as Alzheimer’s disease (AD). This is the case for chronic hyperglycemia associated with type 2 diabetes, reducing brain’s ability to handle the neurodegenerative burden associated with AD. However, the mechanisms behind the effects of chronic hyperglycemia in the context of AD are not fully understood. Hippocampal adult neurogenesis contributes to...
#1Alice TeghilH-Index: 2
#2Antonella Di Vita (Sapienza University of Rome)H-Index: 5
Last. Maddalena Boccia (Sapienza University of Rome)H-Index: 15
view all 4 authors...
: Behavioral evidence suggests that different mechanisms mediate duration perception depending on whether regular or irregular cues for time estimation are provided, and that individual differences in interoceptive processing may affect duration perception only in the latter case. However, no study has addressed brain correlates of this proposed distinction. Here participants performed a duration reproduction task in two conditions: with unevenly spaced stimuli during time estimation/reproductio...
#1Scott C. Bell (UQ: University of Queensland)H-Index: 47
#2Patrick A. Flume (MUSC: Medical University of South Carolina)H-Index: 42
Last. Carlo CastellaniH-Index: 35
view all 3 authors...
2 CitationsSource
#1Luba Sominsky (RMIT: RMIT University)H-Index: 15
#2David W. Walker (RMIT: RMIT University)H-Index: 37
Last. Sarah J. Spencer (RMIT: RMIT University)H-Index: 31
view all 3 authors...
3 CitationsSource
#1Panagiotis Tsigaris (TRU: Thompson Rivers University)H-Index: 8
view all 2 authors...
The issue of “predatory” publishing began to directly affect academics following the creation of two blacklists by a US librarian, Jeffrey Beall. This paper provides a post-publication replication examination of a study that became a high media-profile case. That study involves Derek Pyne, who incorrectly claimed in a 2017 publication in the Journal of Scholarly Publishing that numerous research faculty members at a small business school in Canada were financially remunerated for Beall-blacklist...
#1Scott T Albert (Johns Hopkins University)H-Index: 3
#2Alkis Hadjiosif (Johns Hopkins University)
Last. Reza Shadmehr (Johns Hopkins University)H-Index: 66
view all 9 authors...
Moving an arm requires the brain to send electrical signals to the arm’s muscles, causing them to contract. Neuroscientists call these types of brain signals “move signals”. The brain also sends so-called hold signals, which hold the arm still in a desired position. Part of the brain known as the primary motor cortex helps to calculate the move signals for the arm, but it was unclear how the brain produces the corresponding hold signals. Fortunately, the fact that the brain moves other things be...
1 CitationsSource
#1Kip D Zimmerman (Wake Forest Baptist Medical Center)
#1Kip D. Zimmerman (Wake Forest University)H-Index: 2
Last. Carl D. Langefeld (Wake Forest University)H-Index: 80
view all 3 authors...
Cells from the same individual share a common genetic and environmental background and are not independent, therefore they are subsamples or pseudoreplicates. Empirically, we show this dependence across a range of cell types. Thus, single-cell data have a hierarchical structure that current single-cell methods do not address and subsequently the application of such tools leads to biased inference and reduced robustness and reproducibility. When properly simulating the hierarchical structure of s...