Match!

Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting.

Published on Jan 1, 2017in International Journal of Psychophysiology2.41
· DOI :10.1016/j.ijpsycho.2016.09.005
Peter E. Clayson18
Estimated H-index: 18
(UCLA: University of California, Los Angeles),
Gregory A. Miller52
Estimated H-index: 52
(UCLA: University of California, Los Angeles)
Cite
Abstract
Abstract Failing to consider psychometric issues related to reliability and validity, differential deficits, and statistical power potentially undermines the conclusions of a study. In research using event-related brain potentials (ERPs), numerous contextual factors (population sampled, task, data recording, analysis pipeline, etc.) can impact the reliability of ERP scores. The present review considers the contextual factors that influence ERP score reliability and the downstream effects that reliability has on statistical analyses. Given the context-dependent nature of ERPs, it is recommended that ERP score reliability be formally assessed on a study-by-study basis. Recommended guidelines for ERP studies include 1) reporting the threshold of acceptable reliability and reliability estimates for observed scores, 2) specifying the approach used to estimate reliability, and 3) justifying how trial-count minima were chosen. A reliability threshold for internal consistency of at least 0.70 is recommended, and a threshold of 0.80 is preferred. The review also advocates the use of generalizability theory for estimating score dependability (the generalizability theory analog to reliability) as an improvement on classical test theory reliability estimates, suggesting that the latter is less well suited to ERP research. To facilitate the calculation and reporting of dependability estimates, an open-source Matlab program, the ERP Reliability Analysis Toolbox, is presented.
  • References (116)
  • Citations (13)
Cite
References116
Newest
Published on Jan 1, 2017in Psychophysiology3.38
Kyle E. Mathewson13
Estimated H-index: 13
(U of A: University of Alberta),
Tyler Harrison8
Estimated H-index: 8
(U of A: University of Alberta),
Sayeed A. D. Kizuk2
Estimated H-index: 2
(U of A: University of Alberta)
Dry electrodes are becoming popular for both lab-based and consumer-level electrophysiological-recording technologies because they better afford the ability to move traditional lab-based research into the real world. It is unclear, however, how dry electrodes compare in data quality to traditional electrodes. The current study compared three EEG electrode types: (a) passive-wet electrodes with no onboard amplification, (b) actively amplified, wet electrodes with moderate impedance levels, and lo...
Published on Jan 1, 2017in International Journal of Psychophysiology2.41
Peter E. Clayson18
Estimated H-index: 18
(UCLA: University of California, Los Angeles),
Gregory A. Miller52
Estimated H-index: 52
(UCLA: University of California, Los Angeles)
Abstract Generalizability theory (G theory) provides a flexible, multifaceted approach to estimating score reliability. G theory's approach to estimating score reliability has important advantages over classical test theory that are relevant for research using event-related brain potentials (ERPs). For example, G theory does not require parallel forms (i.e., equal means, variances, and covariances), can handle unbalanced designs, and provides a single reliability estimate for designs with multip...
Published on Jan 1, 2017in International Journal of Psychophysiology2.41
Scott A. Baldwin30
Estimated H-index: 30
(BYU: Brigham Young University)
Abstract Psychology as a field is in the midst of what is sometimes called a “crisis” because false findings are prevalent. Although the focus of the methodological and substantive criticisms of psychology has focused on social psychology, psychophysiology research is not without its problems. The author discusses (a) researcher flexibility and its impact on the stability of conclusions and (b) the role power plays in the probability that a finding is true and the precision of estimates. The aut...
Published on Jan 1, 2017in International Journal of Psychophysiology2.41
Michael J. Larson34
Estimated H-index: 34
(BYU: Brigham Young University),
Kaylie A. Carbine5
Estimated H-index: 5
(BYU: Brigham Young University)
There is increasing focus across scientific fields on adequate sample sizes to ensure non-biased and reproducible effects. Very few studies, however, report sample size calculations or even the information needed to accurately calculate sample sizes for grants and future research. We systematically reviewed 100 randomly selected clinical human electrophysiology studies from six high impact journals that frequently publish electroencephalography (EEG) and event-related potential (ERP) research to...
Published on Mar 1, 2016in Psychophysiology3.38
Gregory A. Miller52
Estimated H-index: 52
(UCLA: University of California, Los Angeles),
Brigitte Rockstroh46
Estimated H-index: 46
(University of Konstanz)
+ 1 AuthorsCindy M. Yee20
Estimated H-index: 20
(UCLA: University of California, Los Angeles)
The Research Domain Criteria (RDoC) initiative of the National Institute of Mental Health shows great promise in providing guidance for research on mental illness but has prompted considerable controversy. Papers by Yancey, Venables, and Patrick and Kozak and Cuthbert illustrate and clarify a number of important features of RDoC. The present commentary evaluates the former paper in light of the latter paper and addresses several common misunderstandings about RDoC. The concept of endophenotypes ...
Published on Mar 1, 2016in Psychophysiology3.38
Michael J. Kozak28
Estimated H-index: 28
,
Bruce N. Cuthbert54
Estimated H-index: 54
This article describes the National Institute of Mental Health's Research Domain Criteria (RDoC) initiative. The description includes background, rationale, goals, and the way the initiative has been developed and organized. The central RDoC concepts are summarized and the current matrix of constructs that have been vetted by workshops of extramural scientists is depicted. A number of theoretical and methodological issues that can arise in connection with the nature of RDoC constructs are highli...
Published on Feb 1, 2016in Psychological Reports1.02
Andrew Brand7
Estimated H-index: 7
(Bangor University),
M. T. Bradley14
Estimated H-index: 14
(UNB: University of New Brunswick)
Confidence interval (CI) widths were calculated for reported Cohen's d standardized effect sizes and examined in two automated surveys of published psychological literature. The first survey reviewed 1,902 articles from Psychological Science. The second survey reviewed a total of 5,169 articles from across the following four APA journals: Journal of Abnormal Psychology, Journal of Applied Psychology, Journal of Experimental Psychology: Human Perception and Performance, and Developmental Psycholo...
Published on Dec 1, 2015in JAMA Psychiatry15.92
Cindy M. Yee20
Estimated H-index: 20
(UCLA: University of California, Los Angeles),
Daniel C. Javitt91
Estimated H-index: 91
(Columbia University),
Gregory A. Miller52
Estimated H-index: 52
(UCLA: University of California, Los Angeles)
Published on Jul 3, 2015in Psychological Inquiry10.27
Gregory A. Miller52
Estimated H-index: 52
(UCLA: University of California, Los Angeles),
Cindy M. Yee20
Estimated H-index: 20
(UCLA: University of California, Los Angeles)
Vaidyanathan et al. (this issue) offer provocative proposals for the striking lack of progress, after decades of basic and clinical research, in understanding the etiology of psychopathology. They engage a number of prominent issues. They do not (and need not) take a stand regarding the relative merits of DSM 5’s categorical premise and its almost exclusively psychological characterization of the manifestations of psychopathology vs. the merits of the Research Domain Criteria (RDoC) initiative o...
Cited By13
Newest
Published on May 14, 2019in Cognitive Neurodynamics3.02
Thorsten Fehr17
Estimated H-index: 17
,
Patricia Milz
Contemporary neuroscience research primarily focuses on the identification of brain activation patterns commonly deviant across participant groups or experimental conditions. This approach inherently underestimates potentially meaningful intra- and inter-individual variability present in brain physiological measures. We propose a parameter referred to as ‘individuality index (II)’ that takes individual variability into account. It quantifies the degree of individual variance of brain activation ...
Published on Jun 1, 2019in Neuropsychologia2.87
Brittany K. Taylor2
Estimated H-index: 2
(CSU: Colorado State University),
William J. Gavin12
Estimated H-index: 12
(CSU: Colorado State University)
+ 3 AuthorsPatricia L. Davies18
Estimated H-index: 18
(CSU: Colorado State University)
Abstract This study demonstrates the utility of combining principles of connectionist theory with a sophisticated statistical approach, structural equation modeling (SEM), to better understand brain-behavior relationships in studies using event-related potentials (ERPs). The models show how sequential phases of neural processing measured by averaged ERP waveform components can successfully predict task behavior (response time; RT) while accounting for individual differences in maturation and sex...
Published on Sep 1, 2019in International Journal of Psychophysiology2.41
Peter E. Clayson18
Estimated H-index: 18
(UCLA: University of California, Los Angeles),
Michael J. Larson34
Estimated H-index: 34
(BYU: Brigham Young University)
Abstract The impact of affective context on performance-monitoring aspects of cognitive control is not clear, and findings in the literature are contradictory. The contradictory findings might be due to failing to consider the impact of recent and concurrent affective contexts, participant ratings of emotional images, and the distinct impact of emotional arousal and valence. The present study utilized multilevel modeling (MLM) on person-specific ratings of arousal and valence to determine how re...
Published on May 1, 2019in International Journal of Psychophysiology2.41
Yuya Ogawa (Osaka University), Hiroshi Nittono23
Estimated H-index: 23
(Osaka University)
Abstract The effect of induced mood on language processing has been examined in behavioral and event-related potential (ERP) studies. A previous study examined the effects of induced mood on word imagery processing by the N400 and N700 components of the ERP and behavioral performance in an imageability judgment task in which participants decided whether a word easily evoked visual imagery or not (Ogawa and Nittono, 2019). The N400 amplitude was larger (more negative-going) under positive mood th...
Published on Aug 1, 2019in International Journal of Psychophysiology2.41
Kaylie A. Carbine5
Estimated H-index: 5
(BYU: Brigham Young University),
Hannah M. Lindsey (BYU: Brigham Young University)+ 1 AuthorsMichael J. Larson34
Estimated H-index: 34
(BYU: Brigham Young University)
Abstract Selective reporting (i.e., only reporting significant findings as opposed to all analyses or results) is a questionable research practice that undermines the integrity of published research. Psychophysiology research may be susceptible to selective reporting, given the high number of decision points and methodological complexity in analyses of psychophysiology data. We aimed to assess the presence of selective reporting and evidential value (i.e., that significant results are due to tru...
Published on Jul 19, 2019in Psychophysiology3.38
Peter E. Clayson18
Estimated H-index: 18
(UCLA: University of California, Los Angeles),
Kaylie A. Carbine5
Estimated H-index: 5
(BYU: Brigham Young University)
+ 1 AuthorsMichael J. Larson34
Estimated H-index: 34
(BYU: Brigham Young University)
Published on May 23, 2019in Health Psychology Review9.07
Kaylie A. Carbine5
Estimated H-index: 5
(BYU: Brigham Young University),
Michael J. Larson34
Estimated H-index: 34
(BYU: Brigham Young University)
Published on Apr 3, 2019in PLOS ONE2.78
Jan Meier3
Estimated H-index: 3
,
Jan Meier1
Estimated H-index: 1
(UHH: University of Hamburg)
+ 3 AuthorsChristoph Mulert2
Estimated H-index: 2
(UHH: University of Hamburg)
Synchronized oscillatory gamma-band activity (30-100Hz) has been suggested to constitute a key mechanism to dynamically orchestrate sensory information integration across multiple spatio-temporal scales. We here tested whether interhemispheric functional connectivity and ensuing auditory perception can selectively be modulated by high-density transcranial alternating current stimulation (HD-tACS). For this purpose, we applied multi-site HD-tACS at 40Hz bilaterally with a phase lag of 180° and re...
Published on Dec 1, 2018in Cortex4.28
Nathaniel Elkins-Brown3
Estimated H-index: 3
(U of T: University of Toronto),
Blair Saunders8
Estimated H-index: 8
(Dund.: University of Dundee),
Michael Inzlicht4
Estimated H-index: 4
(U of T: University of Toronto)
Abstract A growing body of work in social and affective neuroscience suggests that emotion plays an instrumental role in error monitoring processes, rather than only a moderating one. High-powered replications of studies that support this idea, however, are lacking. Here, we attempted a preregistered replication of our own study that had provided evidence for the functional role of emotions in error monitoring: that a neural signal of error monitoring—the error-related negativity—is reduced when...
Published on Oct 1, 2018in International Journal of Psychophysiology2.41
Paige Ethridge2
Estimated H-index: 2
(McGill University),
Anna Weinberg26
Estimated H-index: 26
(McGill University)
Abstract Reward-related event-related potentials (ERPs) are often used to index individual differences that signal the presence or predict the onset of psychopathology. However, relatively little research has explored the psychometric properties of reward-related ERPs. Without understanding their psychometric properties, the value of using ERPs as biomarkers for psychopathology is limited. The present study, therefore, sought to establish the internal consistency reliability and convergent valid...
View next paperERP Reliability Analysis (ERA) Toolbox: An open-source toolbox for analyzing the reliability of event-related brain potentials.