Empirical examination of the replicability of associations between brain structure and psychological variables

Published on Mar 13, 2019in eLife7.551
· DOI :10.7554/eLife.43464
Shahrzad Kharabian Masouleh5
Estimated H-index: 5
(HHU: University of Düsseldorf),
Simon B. Eickhoff86
Estimated H-index: 86
(HHU: University of Düsseldorf)
+ 2 AuthorsAlzheimer's Disease Neuroimaging Initiative1
Estimated H-index: 1
All human brains share the same basic structure. But no two brains are exactly alike. Brain scans can reveal differences between people in the organization and activity of individual brain regions. Studies have suggested that these differences give rise to variation in personality, intelligence and even political preferences. But recent attempts to replicate some of these findings have failed, questioning the existence of such a direct link, specifically between brain structure and human behavior. This had led some disagreements whether there is a general replication crisis in psychology, or if the replication studies themselves are flawed. Kharabian Masouleh et al. have now used brain scans from hundreds of healthy volunteers from an already available dataset to try to resolve the issue. The volunteers had previously completed several psychological tests. These measured cognitive and behavioral aspects such as attention, memory, anxiety and personality traits. Kharabian Masouleh et al. performed more than 10,000 analyzes on their dataset to look for relationships between brain structure and psychological traits. But the results revealed very few statistically significant relationships. Moreover, the relationships that were identified proved difficult to replicate in independent samples. By contrast, the same analyzes demonstrated robust links between brain structure and memory in patients with Alzheimer's disease. They also showed connections between brain structure and non-psychological traits, such as age. This confirms that the analysis techniques do work. So why did the new study find so few relationships between brain structure and psychological traits, when so many links have been reported previously? One possibility is publication bias. Researchers and journals may be more likely to publish positive findings than negative ones. Another factor could be that that most studies use too few participants to be able to reliably detect relationships between brain structure and behavior, and that studies with 200 to 300 participants are still too small. Therefore, future studies should use samples with many hundreds of participants, or more. This will be possible if more groups make their data available for others to analyze. Researchers and journals must also be more willing to publish negative findings. This will help provide an accurate view of relationships between brain structure and behavior.
  • References (69)
  • Citations (9)
📖 Papers frequently viewed together
77 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Paul De Boeck (OSU: Ohio State University)H-Index: 37
#2Minjeong Jeon (UCLA: University of California, Los Angeles)H-Index: 12
12 CitationsSource
#1Benjamin O. Turner (NTU: Nanyang Technological University)H-Index: 10
#2Erick J. Paul (Microsoft)H-Index: 11
Last. Aron Keith BarbeyH-Index: 27
view all 4 authors...
Despite a growing body of research suggesting that task-based functional magnetic resonance imaging (fMRI) studies often suffer from a lack of statistical power due to too-small samples, the proliferation of such underpowered studies continues unabated. Using large independent samples across eleven tasks, we demonstrate the impact of sample size on replicability, assessed at different levels of analysis relevant to fMRI researchers. We find that the degree of replicability for typical sample siz...
30 CitationsSource
#1Thomas J. Vanasse (University of Texas Health Science Center at San Antonio)H-Index: 2
#2P. Mickle Fox (University of Texas Health Science Center at San Antonio)H-Index: 11
Last. Peter T. FoxH-Index: 114
view all 7 authors...
10 CitationsSource
#1Elmira Ismaylova (UdeM: Université de Montréal)H-Index: 4
#2Jessica Di Sante (Concordia University)H-Index: 3
Last. Linda Booij (Concordia University)H-Index: 28
view all 7 authors...
Numerous studies have shown differences in the functioning in the areas of the frontal-limbic circuitry between depressed patients and controls. However, current knowledge on frontal-limbic neural substrates of individual differences in mood states in everyday life in healthy individuals is scarce. The present study investigates anatomical, resting-state and functional neural correlates of daily mood states in healthy individuals. We expected to observe associations between mood and the frontal-...
4 CitationsSource
#1Sarah Genon (HHU: University of Düsseldorf)H-Index: 12
#2Andrew T. Reid (University of Nottingham)H-Index: 16
Last. Simon B. Eickhoff (HHU: University of Düsseldorf)H-Index: 86
view all 5 authors...
Many brain regions have been defined, but a comprehensive formalization of each region’s function in relation to human behavior is still lacking. Current knowledge comes from various fields, which have diverse conceptions of ‘functions’. We briefly review these fields and outline how the heterogeneity of associations could be harnessed to disclose the computational function of any region. Aggregating activation data from neuroimaging studies allows us to characterize the functional engagement of...
24 CitationsSource
#1Stephen J. Smith (University of Oxford)H-Index: 140
#1Shubulade Smith (University of Oxford)H-Index: 113
Last. Thomas E. Nichols (Warw.: University of Warwick)H-Index: 73
view all 2 authors...
Smith and Nichols discuss "big data" human neuroimaging studies, with very large subject numbers and amounts of data. These studies provide great opportunities for making new discoveries about the brain but raise many new analytical challenges and interpretational risks.
58 CitationsSource
When designing a study, the planned sample size is often based on power analyses. One way to choose an effect size for power analyses is by relying on pilot data. A-priori power analyses are only accurate when the effect size estimate is accurate. In this paper we highlight two sources of bias when performing a-priori power analyses for between-subject designs based on pilot data. First, we examine how the choice of the effect size index (η2, ω2 and e2) affects the sample size and power of the m...
34 CitationsSource
The presumed dominance of “original discovery” over replication is an anomaly. Original discovery has more value than replication primarily when scientific investigation can immediately generate numerous discoveries most of which are true and accurate. This scenario is uncommon. A model shows how original discovery claims typically have small or even negative value. Science becomes worthy mostly because of replication.
5 CitationsSource
#1Henk R. Cremers (UvA: University of Amsterdam)H-Index: 9
#2Tor D. Wager (CU: University of Colorado Boulder)H-Index: 72
Last. Tal Yarkoni (University of Texas at Austin)H-Index: 35
view all 3 authors...
Statistically underpowered studies can result in experimental failure even when all other experimental considerations have been addressed impeccably. In fMRI the combination of a large number of dependent variables, a relatively small number of observations (subjects), and a need to correct for multiple comparisons can decrease statistical power dramatically. This problem has been clearly addressed yet remains controversial—especially in regards to the expected effect sizes in fMRI, and especial...
60 CitationsSource
#1Sarah Genon (University of Liège)H-Index: 12
#2Tobias Wensing (RWTH Aachen University)H-Index: 2
Last. Simon B. Eickhoff (HHU: University of Düsseldorf)H-Index: 86
view all 8 authors...
Abstract Recently, we showed that the functional heterogeneity of the right dorsal premotor (PMd) cortex could be better understood by dividing it into five subregions that showed different behavioral associations according to task-based activations studies. The present study investigated whether the revealed behavioral profile could be corroborated and complemented by a structural brain behavior correlation approach in two healthy adults cohorts. Grey matter volume within the five volumes of in...
4 CitationsSource
Cited By9
#1Sofie L. Valk (Forschungszentrum Jülich)H-Index: 8
#2F. Hoffstaedter (Forschungszentrum Jülich)
Last. Simon B. Eickhoff (Forschungszentrum Jülich)H-Index: 86
view all 6 authors...
Local variation in cortical architecture is highly heritable and distinct genes are associated with specific cortical regions. Total surface area has been shown to be genetically correlated with complex cognitive capacities, suggesting cortical brain structure is a viable endophenotype linking genes to behavior. However, to what extend local brain structure has a genetic association with cognitive and emotional functioning is incompletely understood. Here, we study the genetic correlation betwee...
#1Ian A. Clark (UCL: University College London)H-Index: 2
#2Anna Monk (UCL: University College London)H-Index: 1
Last. Eleanor A. Maguire (UCL: University College London)H-Index: 64
view all 7 authors...
Marked disparities exist across healthy individuals in their ability to imagine scenes, recall autobiographical memories, think about the future and navigate in the world. The importance of the hippocampus in supporting these cognitive functions has prompted the question of whether differences in hippocampal grey matter volume could be one source of performance variability. Evidence to date has been somewhat mixed. In this study we sought to mitigate issues that commonly affect these types of st...
#1Masoud Tahmasian (Shahid Beheshti University)H-Index: 18
#2Fateme Samea (Shahid Beheshti University)H-Index: 3
Last. Simon B. Eickhoff (HHU: University of Düsseldorf)H-Index: 86
view all 11 authors...
Humans need about seven to nine hours of sleep per night. Sleep habits are heritable, associated with brain function and structure, and intrinsically related to well-being, mental, and physical health. However, the biological basis of the interplay of sleep and health is incompletely understood. Here we show, by combining neuroimaging and behavioral genetic approaches in two independent large-scale datasets (HCP (n = 1106), age range: 22–37, eNKI (n = 783), age range: 12–85), that sleep, mental,...
#1Yunglin Gazes (CUMC: Columbia University Medical Center)H-Index: 15
#2Seonjoo Lee (CUMC: Columbia University Medical Center)H-Index: 16
Last. Christian G. Habeck (CUMC: Columbia University Medical Center)H-Index: 31
view all 9 authors...
Abstract This study examined within-subject differences among three fluid abilities that decline with age: reasoning, episodic memory and processing speed, compared with vocabulary, a crystallized ability that is maintained with age. The data were obtained from the Reference Ability Neural Network (RANN) study from which 221 participants had complete behavioral data for all 12 cognitive tasks, three per ability, along with fMRI and diffusion weighted imaging data. We used fMRI task activation to...
#1Divyangana Rakesh (University of Melbourne)
#2Nicholas B. Allen (UO: University of Oregon)H-Index: 58
Last. Sarah Whittle (University of Melbourne)H-Index: 33
view all 3 authors...
Abstract Both depression and substance use problems have their highest incidence during youth (i.e., adolescence and emerging adulthood), and are characterized by emotion regulation deficits. Influential neurodevelopmental theories suggest that alterations in the function of limbic and frontal regions render youth susceptible to these deficits. However, whether depression and substance use in youth are associated with similar alterations in emotion regulation neural circuitry is unknown. In this...
#1Corey Horien (Yale University)H-Index: 4
#2Abigail S. Greene (Yale University)H-Index: 3
Last. Dustin Scheinost (Yale University)H-Index: 28
view all 4 authors...
Functional magnetic resonance imaging has proved to be a powerful tool to characterize spatiotemporal patterns of human brain activity. Analysis methods broadly fall into two camps: those summarizi...
#1Raihaan Patel (McGill University)H-Index: 8
#2Christopher J. Steele (Concordia University)H-Index: 15
Last. M. Mallar Chakravarty (McGill University)H-Index: 44
view all 8 authors...
Abstract In this work we use non-negative matrix factorization to identify patterns of microstructural variance in the human hippocampus. We utilize high-resolution structural and diffusion magnetic resonance imaging data from the Human Connectome Project to query hippocampus microstructure on a multivariate, voxelwise basis. Application of non-negative matrix factorization identifies spatial components (clusters of voxels sharing similar covariance patterns), as well as subject weightings (indi...
1 CitationsSource
#1Anthony P. Zanesco (UM: University of Miami)H-Index: 11
#2Brandon G. King (UC Davis: University of California, Davis)H-Index: 13
Last. Clifford D. Saron (UC Davis: University of California, Davis)H-Index: 25
view all 4 authors...
Abstract Microstates reflect transient brain states resulting from the synchronous activity of brain networks that predominate in the broadband EEG. There has been increasing interest in how the functional organization of the brain varies across individuals, or the extent to which its spatiotemporal dynamics are state dependent. However, little research has examined within and between-person correlates of microstate temporal parameters in healthy populations. In the present study, neuroelectric ...
1 CitationsSource
#1Chujun LinH-Index: 1
#2Ümit KeleşH-Index: 1
Last. Ralph AdolphsH-Index: 94
view all 6 authors...
Recent studies in adult humans have reported correlations between individual differences in people’s Social Network Index (SNI) and gray matter volume (GMV) across multiple regions of the brain. However, the cortical and subcortical loci identified are inconsistent across studies. These discrepancies might arise because different regions of interest were hypothesized and tested in different studies without controlling for multiple comparisons, and/or from insufficiently large sample sizes to ful...
#1Shahrzad KharabianH-Index: 1
#2Simon B. EickhoffH-Index: 86
Last. Sarah GenonH-Index: 12
view all 3 authors...
The study of associations between inter-individual differences in brain structure and behaviour has a long history in psychology and neuroscience. Many associations between psychometric data, particularly intelligence and personality measures and local variations of brain cortical thickness (CT) have been reported. While the impact of such findings often go beyond scientific communities, resonating in the public mind, their replicability is rarely evidenced. Here, we empirically investigated the...
2 CitationsSource