Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators

Published on Oct 1, 2017in Resources Conservation and Recycling7.04
· DOI :10.1016/j.respol.2017.06.006
Jian Wang13
Estimated H-index: 13
(Katholieke Universiteit Leuven),
Reinhilde Veugelers36
Estimated H-index: 36
Paula E. Stephan36
Estimated H-index: 36
(GSU: Georgia State University)
Research which explores unchartered waters has a high potential for major impact but also carries a high uncertainty of having minimal impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. We measure novelty by examining the extent to which a published paper makes first time ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to a set of one million research articles across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distinct) new combinations, have more than a triple probability of being a top 1% highly cited paper when using a sufficiently long citation time window to assess impact. Moreover, follow-on papers that cite highly novel research are themselves more likely to be highly cited. However, novel research is also risky as it has a higher variance in the citation performance. These findings are consistent with the “high risk/high gain” characteristic of novel research. We also find that novel papers are typically published in journals with a lower than expected Impact Factor and are less cited when using a short time window. Our findings suggest that science policy, in particular funding decisions which are over reliant on traditional bibliometric indicators based on short-term direct citation counts and Journal Impact Factors, may be biased against novelty.
  • References (55)
  • Citations (54)
#1Ruizhi Zhang (Georgia Institute of Technology)H-Index: 2
#2Jian Wang (Katholieke Universiteit Leuven)H-Index: 13
Last.Yajun Mei (Georgia Institute of Technology)H-Index: 12
view all 3 authors...
#1Mikko Packalen (UW: University of Waterloo)H-Index: 7
#2Jay Bhattacharya (Stanford University)H-Index: 36
#1Dennis Verhoeven (Katholieke Universiteit Leuven)H-Index: 2
#2Jurriën Bakker (Katholieke Universiteit Leuven)H-Index: 3
Last.Reinhilde VeugelersH-Index: 5
view all 3 authors...
#1Andrey Rzhetsky (U of C: University of Chicago)H-Index: 46
#2Jacob G. Foster (UCLA: University of California, Los Angeles)H-Index: 10
Last.James A. Evans (U of C: University of Chicago)H-Index: 19
view all 4 authors...
#1Feng Shi (U of C: University of Chicago)H-Index: 7
#2Jacob G. Foster (UCLA: University of California, Los Angeles)H-Index: 10
Last.James A. Evans (U of C: University of Chicago)H-Index: 19
view all 3 authors...
#1Alfredo Yegros-Yegros (LEI: Leiden University)H-Index: 6
#2Ismael Rafols (Polytechnic University of Valencia)H-Index: 28
Last.Pablo D'Este (Polytechnic University of Valencia)H-Index: 20
view all 3 authors...
#1Jian Wang (Katholieke Universiteit Leuven)H-Index: 13
#2Bart Thijs (Katholieke Universiteit Leuven)H-Index: 23
Last.GlänzelWolfgang (Katholieke Universiteit Leuven)H-Index: 58
view all 3 authors...
Cited By54
#1Marcella Corsi (Sapienza University of Rome)H-Index: 7
#2Carlo D'Ippoliti (Sapienza University of Rome)H-Index: 7
Last.Giulia Zacchia (Sapienza University of Rome)H-Index: 3
view all 3 authors...
#1James R. Bradley (W&M: College of William & Mary)H-Index: 14
Last.George ChackoH-Index: 3
view all 8 authors...
#1Jos J. Winnink (LEI: Leiden University)H-Index: 5
#2Robert J.W. Tijssen (Stellenbosch University)H-Index: 6
Last.A. F. J. Van Raan (LEI: Leiden University)H-Index: 31
view all 3 authors...
#1Mignon L. Wuestman (UU: Utrecht University)
#2Jarno Hoekman (UU: Utrecht University)H-Index: 14
Last.Koen Frenken (UU: Utrecht University)H-Index: 44
view all 3 authors...
View next paperAtypical Combinations and Scientific Impact