Research Guides: Scholarly Communications: Research Impact

Metrics Toolkit
"The Metrics Toolkit is a resource for researchers and evaluators that provides guidance for demonstrating and evaluating claims of research impact. With the Toolkit you can quickly understand what a metric means, how it is calculated, and if it’s good match for your impact question."

Metrics Options

Click on each item for explanations and suggested tools. * = resources available to current JMU users only. Note that JMU does not subscribe to Journal Citation Reports.

Journals

Journals
In general terms, journal metrics gauge the influence of scholarly journals using citation analysis. These measures are based upon the assumptions that a citation demonstrates a significant impact on the scholarly conversation and that the more citations a publication accrues, the more important it is to that scholarly community.The most well-known of these is the Impact Factor, which has been in use since the 1970s.

In addition to the common journal metrics in the table below, SNIP (Source Normalized Impact per Paper) is interesting because it corrects for differences in citation practices between disciplines.

	Impact Factor	Eigenfactor	SCImago Journal Rank (SJR)	h-Index
Measures	Relative impact using proxy of frequency with which the journal's "average article" has been cited in 5 year period.	Relative importance according to a stoichiometric estimate of time researchers spent reading the journal.	Prestige metric based on influence of citing journals for all articles in the selected journal in 3 previous years.	Productivity (papers published) and citation impact over a specified number of years.
Calculation	X / Y where: X = cites in a given year to articles published in 2 previous years, and Y = number of articles published in 2 previous years (details).	I / N where: I = importance of citation based on influence of the citing journal, calculated by a network theory-derived algorithm, and N = total number of citations appearing in the citing journal (details).	P / C where: P= size-dependent metric of an whole journal’s prestige, calculated using network theory elements, and C= ratio of citable documents each journal has relative to the total documents in all Scopus journals (details).	Largest number h of a journal’s articles that have received h or more citations over the last 5 years.
Pros	-Most widely used & accepted journal metric. -Easy to understand.	-Accounts for quality of citing publication & varied citation rates across disciplines. -Excludes self-citations.	-Difficult to manipulate the data. -Journal size-independent. -Only includes peer-reviewed articles, reviews, and conference papers.	-Outliers discounted. -Easy to understand. -Publication period easily adjusted.
Cons	-Based on mean (actual citation distributions very skewed). -Counts self-citations. -Counts non-peer-reviewed citations (e.g. editorials). -Cross-discipline comparison difficult.	-Calculation difficult to understand. -Less widely used than Impact Factor. -Counts non-peer-reviewed citations. -Likely to increase with increase in journal size. -Only includes journals in Journal Citation Reports.	-Calculation difficult to understand. Less widely used than Impact Factor. -Cross-discipline comparison difficult. Only includes journals in Scopus.*	-Less widely used than Impact Factor. -Cross-discipline comparison difficult. -Likely to increase with increase in journal size.
Where to find it	Journal Citation Reports (JMU does not have access to this resource)	Journal Citation Reports or Eigenfactor.org	Scopus,* SCImago Journal & Country Rank, or JournalMetrics.com (Elsevier)	Google Scholar Metrics, Scopus,* or SCImago Journal & Country Rank (NOTE: time frames differ.)

Articles/Books
Articles/Books
Article-level metrics constitute relatively new attempts to disaggregate an individual article's impact from the journal in which it appears. These metrics can be employed to better gauge the impact of a scholar's book as well.

Data points used can include:
- scholarly citations,
- usage (i.e. view and download counts),
- captures (i.e. social bookmarking and other recommender systems),
- mentions in grey literature and other informal or popular venues, and
- social media presence.
Combining data on both scholarly and social visibility of a single article or book, these metrics are more granular and immediate than traditional measures and help paint a fuller picture of a work's influence over time.

A clear drawback to these metrics is their inability to take into account differences in quality and intent in feedback given to a scholar's work. (Adapted from Article-Level Metrics: A SPARC Primer (2013)

TOOLS:
- Scopus* and some other subscription databases
- Altmetric: free bookmarklet (what does it do?)
- ImpactStory: paid service.
- Google Scholar (citation counts for books & articles)
- social networking and tagging tools like Academia.edu and Mendley
- repositories like PLoS (Public Library of Science), SSRN (Social Science Research Network), and JMU Scholarly Commons
- WorldCat.org (see how many libraries own a certain book)
Researchers
Researchers
Counting all scholarly citations to a researcher's work is one of the most straightforward measures of that author's impact. Many article databases and other tools do this, but the counts are limited to the publications that the database indexes. In addition to crude citation counts, impact metrics for individuals include:
- H-index: number of papers with citation number higher or equal to h, as a useful index to characterize the scientific output of a researcher. Proposed by physicist J.E. Hirsch in 2005.
- G-index: similar to the h-index, but gives more weight to highly cited papers. Proposed by L. Egghe in 2006.
- i10-index: number of publications with at least 10 citations (used only by Google Scholar).
A key way to shore up the reliability of citation counts and impact metrics is to establish a persistent and unique researcher ID to disambiguate authors (why is this important?).

TOOLS:
- Scopus*: author's citation counts and h-index. Links up with ORCID.
- Publish or Perish: free software created by management researcher Anne-Wil Harzing. Citation counts, researcher h-index, g-index, and more.
- Google Scholar Citations: free service for tracking numbers of citations (including some not peer-reviewed) and calculating your individual h-index and i10-index.
- ORCID: a persistent identifier for researchers. See JMU librarians C. Schubert & S. Holloway's presentation "Give Yourself an Orchid."
- ImpactStory: paid service for collecting altmetrics.
Other Artifacts
Other Artifacts
Citations, views, downloads, and social media recommendations and discussion can be tracked for other types of output, including:
- Datasets
- Software
- Slides
Options depend on both (A) some uniform way of identifying the work, like a doi or stable URL, and (B) the publisher or platform's capabilities.

TOOLS:
- ImpactStory: paid service for collecting altmetrics on articles, datasets, software, slides, and more.
- Open access repositories like FigShare (many content types), GitHub (software code), and many more.
- JMU Scholarly Commons and other institutional repositories.
- Sage Policy Profiles for grey literature.
- SlideShare for slide decks.

Sage Policy Profiles
Create a free profile to search an open data source and see where your research has been cited in policies, government documents, white papers, and other reports.