Branding/Logomark minus Citation Combined Shape Icon/Bookmark-empty Icon/Copy Icon/Collection Icon/Close Copy 7 no author result Created with Sketch. Icon/Back Created with Sketch. Match!

Mapping the evaluation results between quantitative metrics and meta-synthesis from experts’ judgements: evidence from the Supply Chain Management and Logistics journals ranking

Published on Feb 14, 2019
· DOI :10.1007/s00500-019-03837-3
Lili Yuan1
Estimated H-index: 1
(CAS: Chinese Academy of Sciences),
Jianping LiXiaolei19
Estimated H-index: 19
(CAS: Chinese Academy of Sciences)
+ 2 AuthorsDengsheng Wu10
Estimated H-index: 10
(CAS: Chinese Academy of Sciences)
Abstract
Meta-syntheses from experts’ judgements and quantitative metrics are two main forms of evaluation. But they both have limitations. This paper constructs a framework for mapping the evaluation results between quantitative metrics and experts’ judgements such that they may be solved. In this way, the weights of metrics in quantitative evaluation are objectively obtained, and the validity of the results can be testified. Weighted average percentile (WAP) is employed to aggregate different experts’ judgements into standard WAP scores. The Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method is used to map quantitative results into experts’ judgements, while WAP scores are equal to the final closeness coefficients generated by the TOPSIS method. However, the closeness coefficients of TOPSIS rely on the weights of quantitative metrics. In this way, the mapping procedure is transformed into an optimization problem, and a genetic algorithm is introduced to search for the best weights. An academic journal ranking in the field of Supply Chain Management and Logistics (SCML) is used to test the validity obtained by mapping results. Four prominent ranking lists from Association of Business Schools, Australian Business Deans Council, German Academic Association for Business Research, and Comite National de la Recherche Scientifique were selected to represent different experts’ judgements. Twelve indices including IF, Eigenfactor Score (ES), H-index, Scimago Journal Ranking, and Source Normalized Impact per Paper (SNIP) were chosen for quantitative evaluation. The results reveal that the mapping results possess high validity for the relative error of experts’ judgements, the quantitative metrics are 43.4%, and the corresponding best weights are determined in the meantime. Thus, some interesting findings are concluded. First, H-index, Impact Per Publication (IPP), and SNIP play dominant roles in the SCML journal’s quality evaluation. Second, all the metrics are positively correlated, although the correlation varies among metrics. For example, ES and NE are perfectly, positively correlated with each other, yet they have the lowest correlation with the other metrics. Metrics such as IF, IFWJ, 5-year IF, and IPP are highly correlated. Third, some highly correlated metrics may perform differently in quality evaluation, such as IPP and 5-year IF. Therefore, when mapping the quantitative metrics and experts’ judgements, academic fields should be treated distinctively.
  • References (41)
  • Citations (0)
Cite
References41
Newest
Published on May 1, 2019in IEEE Transactions on Fuzzy Systems 8.41
Chunbing Bao3
Estimated H-index: 3
(CAS: Chinese Academy of Sciences),
Dengsheng Wu10
Estimated H-index: 10
(CAS: Chinese Academy of Sciences),
Jianping LiXiaolei19
Estimated H-index: 19
(CAS: Chinese Academy of Sciences)
Risk measures play significant roles in determining the magnitude of risks. The traditional risk measures consider only the consequence $(C)$ and the probability $(P)$ and ignore the support of the knowledge behind to estimate $C$ and $P$ . Several researchers have suggested adding knowledge as a third dimension in the risk measures. However, the issues of how to embed the dimension of knowledge in the risk measures to output an explicit expression of the risk measure and how to measure the stre...
4 Citations Source Cite
Published on Sep 1, 2018in Expert Systems With Applications 3.77
Y. V. Srinivasa Murthy2
Estimated H-index: 2
(KREC: National Institute of Technology, Karnataka),
Shashidhar G. Koolagudi15
Estimated H-index: 15
(KREC: National Institute of Technology, Karnataka)
Abstract The technology of music information retrieval (MIR) is an emerging field that helps in tagging each portion of an audio clip. A majority of the subtasks of MIR need an application that segments vocal and non-vocal portions. In this paper, an effort has been made to segment the vocal and non-vocal regions using some novel features based on formant structure on top of standard features. The features such as Mel-frequency cepstral coefficients (MFCCs), linear prediction cepstral coefficien...
2 Citations Source Cite
Published on Jun 1, 2018
Soumen Atta2
Estimated H-index: 2
(KGEC: Kalyani Government Engineering College),
Priya Ranjan Sinha Mahapatra5
Estimated H-index: 5
(KGEC: Kalyani Government Engineering College),
Anirban Mukhopadhyay21
Estimated H-index: 21
(KGEC: Kalyani Government Engineering College)
The maximal covering location problem (MCLP) deals with the problem of finding an optimal placement of a given number of facilities within a set of customers. Each customer has a specific demand and the facilities are to be placed in such a way that the total demand of the customers served by the facilities is maximized. In this article an improved genetic algorithm (GA)-based approach, which utilizes a local refinement strategy for faster convergence, is proposed to solve MCLP. The proposed alg...
3 Citations Source Cite
Published on May 1, 2018in Journal of Informetrics 3.48
Dengsheng Wu10
Estimated H-index: 10
(CAS: Chinese Academy of Sciences),
Jing Li1
Estimated H-index: 1
(Anda: Anhui University)
+ 1 AuthorsJianping LiXiaolei19
Estimated H-index: 19
(CAS: Chinese Academy of Sciences)
Abstract Assessing the scholarly impact of academic institutions has become increasingly important. The achievements of editorial board members can create benchmarks for research excellence and can be used to evaluate both individual and institutional performance. This paper proposes a new method based on journal editor data for assessing an institution’s scholarly impact. In this paper, a journal editorship index (JEI) that simultaneously accounts for the journal rating (JR), editor title (ET),...
3 Citations Source Cite
Published on Jan 1, 2018in European Journal of Operational Research 3.43
Jianping LiXiaolei19
Estimated H-index: 19
(CAS: Chinese Academy of Sciences),
Xiaoyang Yao2
Estimated H-index: 2
(CAS: Chinese Academy of Sciences)
+ 1 AuthorsDengsheng Wu10
Estimated H-index: 10
(CAS: Chinese Academy of Sciences)
We consider multiple criteria decision aiding (MCDA) in the case of interactions between criteria. In dealing with interactions between criteria, fuzzy measures and integrals have demonstrated great advantages. Nevertheless, the determination of fuzzy measures has proven difficult because the capacities of not only single criterion but also all subsets of criteria need to be identified. Due to the value judgment essence of MCDA, the attitudes of the decision maker (DM) are typically modeled to i...
13 Citations Source Cite
Published on Jan 1, 2018in portal - Libraries and the Academy 1.47
Upeksha Perera1
Estimated H-index: 1
,
Manjula Wijewickrema1
Estimated H-index: 1
Ranking of scholarly journals is important to many parties. Studying the relationships among various ranking metrics is key to understanding the significance of one metric based on another. This research investigates the relationship among four major journal-ranking indicators: the impact factor (IF), the Eigenfactor score (ES), the h . -index (hI), and SCImago Journal Rank (SJR). The authors used 519 journals from a diverse range of subjects and executed a correlation analysis. The results show...
2 Citations Source Cite
Published on Sep 1, 2017in Information Processing and Management 3.44
Jing Li1
Estimated H-index: 1
(CAS: Chinese Academy of Sciences),
Dengsheng Wu10
Estimated H-index: 10
(CAS: Chinese Academy of Sciences)
+ 1 AuthorsMinglu Li6
Estimated H-index: 6
(National Natural Science Foundation of China)
Abstract Periodically ranking institutional research productivity is necessary not only to understand the status of the development of related fields but also to identify gaps and take appropriate corrective steps. Many bibliometric indicators contribute to the assessment of institutional research productivity, but the appropriateness of the indicator and the relationships between different indicators are topics that have not been addressed. For this reason, an indicator framework for the rankin...
8 Citations Source Cite
Alan C. McKinnon25
Estimated H-index: 25
Purpose In a previous paper (McKinnon, 2013), I questioned the principle and practice of journal ranking and discussed its effects on logistics research. Since then several important developments have occurred that have prompted this updated review of the issues. Design/methodology/approach New literature on the journal ranking debate has been reviewed. The validity of the journal ranking as a proxy measure of paper quality is explored using data from the UK Research Excellence Framework (REF) a...
7 Citations Source Cite
Edward C. Rosenthal10
Estimated H-index: 10
(TU: Temple University),
Howard J. Weiss11
Estimated H-index: 11
(TU: Temple University)
We create a data envelopment analysis (DEA) model to rank business journals, using data from the Thomson Reuters Journal Citation Reports® (JCR). As opposed to previous models that ranked journals in only one field and mostly relied on survey data, this model is used to rank 358 business journals from five different JCR categories according to such citation-based factors as the number of articles, the number of citations, impact factor, five-year impact factor, immediacy index, eigenfactor score...
4 Citations Source Cite
Published on Apr 3, 2017in Serials Review 0.31
Khalid Mahmood19
Estimated H-index: 19
(University of the Punjab)
ABSTRACTThis study, based on systematic review and meta-analysis, aimed to collect and analyze evidence on correlation between perception-based journal rankings and the most popular citation-based measure, Thomson Reuters Journal Impact Factor (JIF). A search was conducted in the Web of Science, Scopus, and Google Scholar databases. After the screening of titles, abstracts, and full text, 18 articles were selected as eligible for review and analysis. The included studies belonged to various subj...
4 Citations Source Cite
Cited By0
Newest