The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes

Published on Nov 1, 2013in Scientometrics2.77
· DOI :10.1007/s11192-013-0981-0
J. W. Fedderke1
Estimated H-index: 1
(University of the Witwatersrand)
This paper examines the strength of association between the outcomes of National Research Foundation (NRF) peer review based rating mechanisms, and a range of objective measures of performance of researchers. The analysis is conducted on 1932 scholars that have received an NRF rating or an NRF research chair. We find that on average scholars with higher NRF ratings record higher performance against research output and impact metrics. However, we also record anomalies in the probabilities of different NRF ratings when assessed against bibliometric performance measures, and record a disproportionately large incidence of scholars with high peer-review based ratings with low levels of recorded research output and impact. Moreover, we find strong cross-disciplinary differences in terms of the impact that objective levels of performance have on the probability of achieving different NRF ratings. Finally, we report evidence that NRF peer review is less likely to reward multi-authored research output than single-authored output. Claims of a lack of bias in NRF peer review are thus difficult to sustain.
  • References (58)
  • Citations (8)
#1Giovanni Abramo (National Research Council)H-Index: 27
#2Ciriaco Andrea D'Angelo (University of Rome Tor Vergata)H-Index: 19
Last.Flavia Di Costa (University of Rome Tor Vergata)H-Index: 13
view all 3 authors...
#1Giovanni Abramo (University of Rome Tor Vergata)H-Index: 27
#2Ciriaco Andrea D'Angelo (University of Rome Tor Vergata)H-Index: 19
#1Gemma Derrick (USYD: University of Sydney)H-Index: 10
#2Heidi Sturk (UQ: University of Queensland)H-Index: 8
Last.Wayne Hall (UQ: University of Queensland)H-Index: 87
view all 5 authors...
Cited By8
#1Zaida Chinchilla (CSIC: Spanish National Research Council)H-Index: 14
#2Sandra Miguel Izquierdo (UNLP: National University of La Plata)H-Index: 10
Last.Félix De-Moya-Anegón (CSIC: Spanish National Research Council)H-Index: 33
view all 3 authors...
#1Nick Deschacht (Katholieke Universiteit Leuven)H-Index: 3
#2Tim C. E. Engels (University of Antwerp)H-Index: 12
View next paperAssessment of Research Performance in Biology: How Well Do Peer Review and Bibliometry Correlate?