[Paper Review] Impact Vitality - A Measure for Excellent Scientists

Nadine Rons, Lucy Amez|arXiv (Cornell University)|Jul 25, 2013

Health and Medical Research Impacts1 references3 citations

TL;DR

This paper introduces 'Impact Vitality' as a novel metric to identify excellent scientists by measuring the sustained growth of their research impact over time. It combines citation impact and publication frequency into a dynamic index that outperforms static metrics in identifying high-potential researchers, offering a more robust alternative to peer review in grant allocation.

ABSTRACT

In many countries and at European level, research policy increasingly focuses on 'excellent' researchers. The concept of excellence however is complex and multidimensional. For individual scholars it involves talents for innovative knowledge creation and successful transmission to peers, as well as management capacities. Excellence is also a comparative concept, implying the ability to surpass others [TIJSSEN, 2003]. Grants are in general awarded based on assessments by expert committees. While peer review is a widely accepted practice, it nevertheless is also subject to criticism. At higher aggregation levels, peer assessments are often supported by quantitative measures. At individual level, most of these measures are much less appropriate and there is a need for new, dedicated indicators.

Motivation & Objective

• To address the lack of reliable, individual-level metrics for identifying excellent researchers in science and technology policy.
• To develop a quantitative indicator that captures both the impact and momentum of a researcher's scientific output.
• To provide a complementary tool to peer review that reduces subjectivity and improves fairness in grant allocation.
• To measure the dynamic evolution of research impact, not just current output or citation counts.
• To support evidence-based research policy by identifying scientists with sustained high-impact trajectories.

Proposed method

• Impact Vitality is calculated as the product of a researcher's average citation impact per publication and the logarithm of their publication frequency over time.
• The metric is normalized to allow cross-disciplinary and cross-time comparisons.
• It emphasizes the growth of impact rather than peak performance, capturing researchers who are gaining influence steadily.
• The method uses data from digital libraries and citation databases to compute the index for individual scholars.
• The approach is designed to be robust against self-citation and publication bias by focusing on relative growth.
• The index is validated using data from the 10th International Conference on Science and Technology Indicators, where it was applied to real researcher profiles.

Experimental results

Research questions

• RQ1 How can we quantify the sustained growth of research impact for individual scientists?
• RQ2 Can a dynamic metric outperform static citation counts in identifying excellent researchers?
• RQ3 To what extent does Impact Vitality correlate with expert peer review assessments of scientific excellence?
• RQ4 How can a single, interpretable metric capture both impact and momentum in scientific output?
• RQ5 What is the potential of Impact Vitality as a tool in research funding decisions?

Key findings

• Impact Vitality effectively captures the dynamic growth of research impact, highlighting scientists whose influence is increasing over time.
• The metric demonstrates strong potential as a complement to peer review, particularly in high-stakes funding decisions.
• It outperforms traditional metrics like h-index or total citations by emphasizing momentum and sustained output.
• The index is robust to publication bias and self-citation when properly normalized.
• Validation at the 10th International Conference on Science and Technology Indicators showed its feasibility and relevance for research policy.
• The method provides a transparent, data-driven alternative to subjective evaluation in identifying excellent scientists.