Match!

Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting.

Published on Jan 1, 2017in International Journal of Psychophysiology2.407
· DOI :10.1016/j.ijpsycho.2016.09.005
Peter E. Clayson20
Estimated H-index: 20
(UCLA: University of California, Los Angeles),
Gregory A. Miller53
Estimated H-index: 53
(UCLA: University of California, Los Angeles)
Abstract
Abstract Failing to consider psychometric issues related to reliability and validity, differential deficits, and statistical power potentially undermines the conclusions of a study. In research using event-related brain potentials (ERPs), numerous contextual factors (population sampled, task, data recording, analysis pipeline, etc.) can impact the reliability of ERP scores. The present review considers the contextual factors that influence ERP score reliability and the downstream effects that reliability has on statistical analyses. Given the context-dependent nature of ERPs, it is recommended that ERP score reliability be formally assessed on a study-by-study basis. Recommended guidelines for ERP studies include 1) reporting the threshold of acceptable reliability and reliability estimates for observed scores, 2) specifying the approach used to estimate reliability, and 3) justifying how trial-count minima were chosen. A reliability threshold for internal consistency of at least 0.70 is recommended, and a threshold of 0.80 is preferred. The review also advocates the use of generalizability theory for estimating score dependability (the generalizability theory analog to reliability) as an improvement on classical test theory reliability estimates, suggesting that the latter is less well suited to ERP research. To facilitate the calculation and reporting of dependability estimates, an open-source Matlab program, the ERP Reliability Analysis Toolbox, is presented.
  • References (115)
  • Citations (20)
📖 Papers frequently viewed together
21 Citations
29 Citations
2005
1 Author (Stephen J. Luck)
2,564 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
References115
Newest
#1Michael J. Larson (BYU: Brigham Young University)H-Index: 34
#2Kaylie A. Carbine (BYU: Brigham Young University)H-Index: 6
29 CitationsSource
Abstract Psychology as a field is in the midst of what is sometimes called a “crisis” because false findings are prevalent. Although the focus of the methodological and substantive criticisms of psychology has focused on social psychology, psychophysiology research is not without its problems. The author discusses (a) researcher flexibility and its impact on the stability of conclusions and (b) the role power plays in the probability that a finding is true and the precision of estimates. The aut...
10 CitationsSource
#1Kyle E. Mathewson (U of A: University of Alberta)H-Index: 14
#2Tyler Harrison (U of A: University of Alberta)H-Index: 9
Last. Sayeed A. D. Kizuk (U of A: University of Alberta)H-Index: 2
view all 3 authors...
Dry electrodes are becoming popular for both lab-based and consumer-level electrophysiological-recording technologies because they better afford the ability to move traditional lab-based research into the real world. It is unclear, however, how dry electrodes compare in data quality to traditional electrodes. The current study compared three EEG electrode types: (a) passive-wet electrodes with no onboard amplification, (b) actively amplified, wet electrodes with moderate impedance levels, and lo...
36 CitationsSource
#1Peter E. Clayson (UCLA: University of California, Los Angeles)H-Index: 20
#2Gregory A. Miller (UCLA: University of California, Los Angeles)H-Index: 53
21 CitationsSource
5 CitationsSource
This article describes the National Institute of Mental Health's Research Domain Criteria (RDoC) initiative. The description includes background, rationale, goals, and the way the initiative has been developed and organized. The central RDoC concepts are summarized and the current matrix of constructs that have been vetted by workshops of extramural scientists is depicted. A number of theoretical and methodological issues that can arise in connection with the nature of RDoC constructs are highli...
210 CitationsSource
#1Gregory A. Miller (UCLA: University of California, Los Angeles)H-Index: 53
#2Brigitte Rockstroh (University of Konstanz)H-Index: 43
Last. Cindy M. Yee (UCLA: University of California, Los Angeles)H-Index: 20
view all 4 authors...
The Research Domain Criteria (RDoC) initiative of the National Institute of Mental Health shows great promise in providing guidance for research on mental illness but has prompted considerable controversy. Papers by Yancey, Venables, and Patrick and Kozak and Cuthbert illustrate and clarify a number of important features of RDoC. The present commentary evaluates the former paper in light of the latter paper and addresses several common misunderstandings about RDoC. The concept of endophenotypes ...
17 CitationsSource
#1Andrew Brand (Bangor University)H-Index: 8
#2M. T. Bradley (UNB: University of New Brunswick)H-Index: 15
Confidence interval (CI) widths were calculated for reported Cohen's d standardized effect sizes and examined in two automated surveys of published psychological literature. The first survey reviewed 1,902 articles from Psychological Science. The second survey reviewed a total of 5,169 articles from across the following four APA journals: Journal of Abnormal Psychology, Journal of Applied Psychology, Journal of Experimental Psychology: Human Perception and Performance, and Developmental Psycholo...
4 CitationsSource
#1Cindy M. Yee (UCLA: University of California, Los Angeles)H-Index: 20
#2Daniel C. Javitt (Columbia University)H-Index: 95
Last. Gregory A. Miller (UCLA: University of California, Los Angeles)H-Index: 53
view all 3 authors...
33 CitationsSource
#1Gregory A. Miller (UCLA: University of California, Los Angeles)H-Index: 53
#2Cindy M. Yee (UCLA: University of California, Los Angeles)H-Index: 20
Vaidyanathan et al. (this issue) offer provocative proposals for the striking lack of progress, after decades of basic and clinical research, in understanding the etiology of psychopathology. They engage a number of prominent issues. They do not (and need not) take a stand regarding the relative merits of DSM 5’s categorical premise and its almost exclusively psychological characterization of the manifestations of psychopathology vs. the merits of the Research Domain Criteria (RDoC) initiative o...
4 CitationsSource
Cited By20
Newest
#1Peter E. Clayson (USF: University of South Florida)H-Index: 1
To ensure adequate reliability (i.e., internal consistency), it is common in studies using event-related brain potentials (ERPs) to exclude participants for having too few trials. This practice is particularly relevant for error-related ERPs, such as error-related negativity (ERN), where the number of recorded ERN trials is not entirely under the researcher's control. Furthermore, there is a widespread practice of inferring reliability based on published psychometric research, which assumes that...
1 CitationsSource
Source
#1Elizabeth A. Martin (UCI: University of California, Irvine)H-Index: 15
#2Lilian Yanqing Li (UCI: University of California, Irvine)H-Index: 1
Last. Mayan K. Castro (UCI: University of California, Irvine)H-Index: 1
view all 3 authors...
Individuals at risk for schizophrenia-spectrum disorders display abnormalities related to motivational salience, or the ability of stimuli to elicit attention due to associations with rewards or punishments. However, the nature of these abnormalities is unclear because most focus on responses to stimuli from broad “pleasant” and “unpleasant” categories and ignore the variation of motivational salience within these categories. In two groups at risk for schizophrenia-spectrum disorders—a Social An...
Source
#2Iulia BanicaH-Index: 1
Last. Annmarie MacNamaraH-Index: 17
view all 6 authors...
Psychological assessments typically rely on self-report and behavioral measures. Augmenting these with neurophysiological measures of the construct in question may increase the accuracy and predictive power of these assessments. Moreover, thinking about neurophysiological measures from an assessment perspective may facilitate under-utilized research approaches (e.g., brain-based recruitment of participants). However, the lack of normative data for most neurophysiological measures has prevented t...
Source
#1Peter E. Clayson (USF: University of South Florida)H-Index: 20
#2Kaylie A. Carbine (BYU: Brigham Young University)H-Index: 6
Last. Michael J. Larson (BYU: Brigham Young University)H-Index: 34
view all 3 authors...
Abstract Performance-monitoring event-related brain potentials (ERPs), such as the error-related negativity (ERN) and reward positivity (RewP), are advocated as biomarkers of depression symptoms and risk. However, a recent meta-analysis indicated effect size heterogeneity in the ERN and RewP literatures. Hence, advocating these ERPs as biomarkers of depression might be premature or possibly misguided due to the selective reporting of significant analyses on the part of researchers (e.g., p-hacki...
1 CitationsSource
#1Alexandra Michelle Muir (BYU: Brigham Young University)
#2Ariana Hedges-Muncy (BYU: Brigham Young University)H-Index: 1
Last. Michael J. Larson (BYU: Brigham Young University)H-Index: 34
view all 5 authors...
Error-monitoring processes may be affected by transdiagnostic dimensions of psychopathology symptoms including trait anxiety, worry, and severity of depressive symptoms. We tested the relationship between continuous measures of anxiety and depressive symptomology and neural correlates of error-monitoring as measured by time-frequency domain delta and theta oscillatory power and time domain error-related negativity (ERN) amplitude extracted from the electroencephalogram (EEG). Secondary analyses ...
Source
#1Peter E. Clayson (UCLA: University of California, Los Angeles)H-Index: 20
#2Kaylie A. Carbine (BYU: Brigham Young University)H-Index: 6
Last. Michael J. Larson (BYU: Brigham Young University)H-Index: 34
view all 4 authors...
Source
#1Peter E. ClaysonH-Index: 20
#2Jonathan K. WynnH-Index: 30
Last. William P. HoranH-Index: 51
view all 6 authors...
Source
#1Brittany K. Taylor (CSU: Colorado State University)H-Index: 2
#2William J. Gavin (CSU: Colorado State University)H-Index: 13
Last. Patricia L. Davies (CSU: Colorado State University)H-Index: 19
view all 6 authors...
Abstract This study demonstrates the utility of combining principles of connectionist theory with a sophisticated statistical approach, structural equation modeling (SEM), to better understand brain-behavior relationships in studies using event-related potentials (ERPs). The models show how sequential phases of neural processing measured by averaged ERP waveform components can successfully predict task behavior (response time; RT) while accounting for individual differences in maturation and sex...
Source
#1Yuya Ogawa (Osaka University)H-Index: 1
#2Hiroshi Nittono (Osaka University)H-Index: 23
Abstract The effect of induced mood on language processing has been examined in behavioral and event-related potential (ERP) studies. A previous study examined the effects of induced mood on word imagery processing by the N400 and N700 components of the ERP and behavioral performance in an imageability judgment task in which participants decided whether a word easily evoked visual imagery or not (Ogawa and Nittono, 2019). The N400 amplitude was larger (more negative-going) under positive mood th...
1 CitationsSource