[Paper Review] Google Scholar Metrics 2013: nothing new under the sun

Álvaro Cabezas-Clavijo, Emilio Delgado López‐Cózar|arXiv (Cornell University)|Jul 26, 2013

Web and Library Services6 references3 citations

TL;DR

This paper evaluates Google Scholar Metrics 2013, finding no significant new features compared to previous versions. It confirms persistent issues from earlier reviews, with only a timeframe update from 2003–2007 to 2008–2012, and concludes that the service remains largely unchanged with no major improvements or innovations introduced.

ABSTRACT

Main characteristics of Google Scholar Metrics new version (july 2013) are presented. We outline the novelties and the weaknesses detected after a first analysis. As main conclusion, we remark the lack of new functionalities with respect to last editions, as the only modification is the update of the timeframe (2008-2012). Hence, problems pointed out in our last reviews still remain active. Finally, it seems Google Scholar Metrics will be updated in a yearly basis

Motivation & Objective

• To assess the new features and improvements in Google Scholar Metrics 2013.
• To identify persistent weaknesses and limitations in the service's functionality.
• To evaluate whether the updated timeframe (2008–2012) represented a meaningful advancement.
• To determine if the service had evolved significantly since prior versions.
• To provide a critical analysis of Google Scholar Metrics' current state and future trajectory.

Proposed method

• Conducted a comparative analysis of Google Scholar Metrics 2013 against previous versions (2012 and earlier).
• Evaluated the new interface and data presentation for novelty and usability.
• Reviewed the updated citation timeframe (2008–2012) and its implications for journal metrics.
• Identified and documented recurring technical and methodological limitations from prior assessments.
• Analyzed the service's update frequency and its implications for long-term reliability.
• Used qualitative and observational methods to assess functionality and user experience.

Experimental results

Research questions

• RQ1 What new features were introduced in Google Scholar Metrics 2013?
• RQ2 How does the updated citation timeframe (2008–2012) affect the reliability and scope of the metrics?
• RQ3 Are the persistent issues identified in earlier evaluations still present in the 2013 version?
• RQ4 Does the 2013 update represent a meaningful evolution or merely a routine refresh?
• RQ5 What are the implications of annual updates without functional innovation?

Key findings

• No new functionalities were introduced in Google Scholar Metrics 2013 compared to previous versions.
• The only change was the update of the citation timeframe from 2003–2007 to 2008–2012.
• Persistent problems identified in earlier reviews, such as lack of transparency and methodological inconsistencies, remained unresolved.
• The service continues to operate on an annual update cycle without substantive improvements.
• The lack of innovation suggests limited development focus on enhancing metric quality or usability.
• The evaluation concludes that the 2013 version offers no significant advancement over prior editions.