[Paper Review] Altmetrics of "altmetrics" using Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki

Saeed‐Ul Hassan, Uzair Gillani|arXiv (Cornell University)|Mar 25, 2016

Expert finding and Q&A systems15 references19 citations

TL;DR

This study proposes an alt-index to measure the social impact of scientific publications in the 'altmetrics' field using data from Google Scholar, Twitter, Mendeley, Facebook, Google+, CiteULike, blogs, and Wikipedia from 2010–2014. It finds high correlation among social impact indicators and a strong link between social and scholarly citations (r = .646), with social citations being 42.2% more prevalent than traditional citations, and identifies Twitter and Mendeley as the most effective platforms for altmetric dissemination.

ABSTRACT

We measure the impact of "altmetrics" field by deploying altmetrics indicators using the data from Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki during 2010- 2014. To capture the social impact of scientific publications, we propose an index called alt-index, analogues to h-index. Across the deployed indices, our results have shown high correlation among the indicators that capture social impact. While we observe medium Pearson's correlation ( {ho}= .247) among the alt-index and h-index, a relatively high correlation is observed between social citations and scholarly citations ( {ho}= .646). Interestingly, we find high turnover of social citations in the field compared with the traditional scholarly citations, i.e. social citations are 42.2 % more than traditional citations. The social mediums such as Twitter and Mendeley appear to be the most effective channels of social impact followed by Facebook and Google-plus. Overall, altmetrics appears to be working well in the field of "altmetrics".

Motivation & Objective

• To assess the social impact of scientific publications in the 'altmetrics' field using non-traditional citation sources.
• To develop and validate an alt-index as a social impact metric analogous to the h-index.
• To compare the volume and dynamics of social citations versus traditional scholarly citations.
• To evaluate the relative effectiveness of different social media and web platforms in disseminating scientific impact.
• To investigate the correlation between altmetric indicators and traditional citation metrics.

Proposed method

• The authors collected altmetric data from eight platforms: Google Scholar, Twitter, Mendeley, Facebook, Google+, CiteULike, blogs, and Wikipedia for publications in the 'altmetrics' field from 2010 to 2014.
• An alt-index was developed as a social impact metric, analogous to the h-index, to quantify the impact of publications based on social media and web mentions.
• Pearson correlation coefficients were computed to assess the relationship between the alt-index and the traditional h-index, and between social citations and scholarly citations.
• The study analyzed citation turnover rates by comparing the total number of social citations to traditional citations across the same publications.
• Platform-specific impact was evaluated by comparing the volume and frequency of mentions across each medium.
• Data analysis focused on publications related to the 'altmetrics' field to ensure contextual relevance and reduce noise from unrelated domains.

Experimental results

Research questions

• RQ1 How does the proposed alt-index correlate with the traditional h-index in measuring scientific impact?
• RQ2 What is the relationship between social citations (from platforms like Twitter and Mendeley) and traditional scholarly citations?
• RQ3 How does the volume of social citations compare to that of traditional citations in the 'altmetrics' field?
• RQ4 Which social media platforms contribute most significantly to the social impact of scientific publications?
• RQ5 To what extent do altmetric indicators reflect or diverge from traditional citation patterns in the context of altmetrics research?

Key findings

• The alt-index shows a medium correlation (ρ = .247) with the traditional h-index, indicating partial alignment between social and scholarly impact measures.
• A relatively high correlation (ρ = .646) was observed between social citations and scholarly citations, suggesting that social media mentions often reflect scholarly attention.
• Social citations were found to be 42.2% more prevalent than traditional scholarly citations, indicating a higher volume of engagement on social platforms.
• Twitter and Mendeley emerged as the most effective platforms for disseminating social impact, followed by Facebook and Google+.
• High correlation among altmetric indicators across platforms suggests consistency in measuring social impact using diverse digital sources.
• The study confirms that altmetrics are effective in the 'altmetrics' field, with strong signal consistency across multiple platforms.