[论文解读] Building Journal Impact Factor Quartile into the Assessment of Academic Performance: A Case Study

Keziban Orbay, Rubén Miranda+1 more|arXiv (Cornell University)|May 6, 2020

Educational Methods and Analysis被引用 6

一句话总结

本研究引入了Q概念——在特定学科类别内按期刊影响因子四分位数进行分类——以减少原始影响因子带来的偏差，从而改进学术表现评估。分析阿马西亚大学2014–2018年在Web of Science收录期刊中的出版物发现，36.49%的论文发表于Q4期刊（低影响因子），而仅有14.32%发表于Q1期刊（高影响因子），且Q1期刊论文的平均引用次数是Q4期刊的六倍，凸显了基于四分位数评估在科研评估中的价值。

ABSTRACT

This study aims to provide information about the Q Concept defined as the division of journal impact factors into quartiles based on given field categories so that the disadvantages resulting from the direct use of journal impact factors can be eliminated. While the number of "Original articles published in the Web of Science (WoS) database-indexed journals like SCI, SSCI and A&HCI" is an important indicator for research assessment in Turkey, neither the journal impact factors nor the Q Concept of these papers have been taken into account. Present study analyzes the scientific production of the Amasya University researchers in journals indexed in WoS database in the period 2014-2018 using the Q concept. The share of publications by Q category journals as well as the average citations received by the works from Amasya University were compared to the average situation in Turkey and other different countries in the world. Results indicate that the articles published by Amasya University researchers were mostly published in low impact factor journals (Q4 journals) (36.49%), in fact, only a small share of papers were published in high impact journals (14.32% in Q1 journals). The share of papers published in low impact journals by researchers from Amasya University is higher than the Turkish average and much higher than the scientific leading countries. The average citations received by papers published in Q1 journals was around six times higher than papers published in Q4 journals (8.92 vs. 1.56), thus papers published in Q1 journals received 30.02% citations despite only 14.32% of the papers was published in these journals. The share of papers published which were never cited in WoS was 27.48%, increasing from 9.68% in Q1 to almost half (48.10%) in Q4. The study concludes with some suggestions on how and where the Q Concept can be used.

研究动机与目标

• 解决在学术表现评估中使用原始期刊影响因子所带来的局限性。
• 提出并应用Q概念——在特定学科类别内对期刊进行基于四分位数的分类——以减少科研评估中的偏差。
• 利用Q概念分析阿马西亚大学研究人员（2014–2018年）的出版与引用模式。
• 将阿马西亚大学在各四分位数期刊中的出版分布与土耳其国家及国际基准进行比较。
• 为将Q概念整合到科研评估体系中提供基于证据的建议。

提出的方法

• 通过在特定学术领域内将期刊影响因子划分为四分位数（Q1–Q4），应用Q概念，确保学科内评估的公平性。
• 从Web of Science（WoS）数据库中提取阿马西亚大学2014–2018年期间的科研产出，重点关注被SCI、SSCI和A&HCI收录的期刊。
• 按期刊四分位数（Q1–Q4）对出版物进行分类，并分析各四分位数的引用次数以评估影响力。
• 将各四分位数的出版物占比与土耳其国家平均值及全球平均水平进行比较，以识别差异。
• 计算并比较各四分位数之间的引用率和未被引用论文的占比，以评估科研影响力。
• 将阿马西亚大学的表现与全球基准进行统计比较，以评估其科研质量的相对水平。

实验结果

研究问题

• RQ1 阿马西亚大学的科研产出在期刊影响因子四分位数（Q1–Q4）中如何分布？
• RQ2 阿马西亚大学在Q1与Q4期刊发表的论文，其引用率有何差异？
• RQ3 阿马西亚大学在各期刊四分位数中，未被引用论文的比例有何不同？
• RQ4 阿马西亚大学在高影响力期刊中的出版分布与土耳其国家平均水平及主要科学国家相比如何？
• RQ5 Q概念在多大程度上提升了学术表现评估的公平性与准确性？

主要发现

• 阿马西亚大学36.49%的出版物发表于Q4期刊，表明其在低影响力期刊中集中度较高。
• 仅有14.32%的出版物发表于Q1期刊，显著低于土耳其国家平均水平及全球领先国家。
• Q1期刊论文的平均引用次数为8.92次，而Q4期刊论文仅为1.56次，高出六倍以上。
• 尽管仅占出版物总数的14.32%，但Q1期刊论文却贡献了30.02%的总引用次数，表明其影响力存在显著不均衡。
• 未被引用论文的比例从Q1的9.68%上升至Q4的48.10%，凸显了各四分位数间引用差异的严重性。
• 阿马西亚大学Q4期刊出版比例高于土耳其国家平均水平及主要科学国家，表明其在科研评估中需改进期刊选择策略。