Home Research Guides Impact Metrics
Announcement(s)
  • NEW: Until the end of October, try a new way to search, browse and read NDSU Libraries' journals
  • NDSU Archives is closed until November 12 to install new shelving.  

Impact Metrics


Introduction

A variety of metrics can be used to quantitatively evaluate academic productivity and the impact your work has in your field. This information can be used to demonstrate the influence and importance of your research to potential funders and to NDSU. Metrics can also be used to identify top researchers in a discipline and the best journals for publishing your research.

Important considerations to keep in mind:

  • There is no one perfect tool for measuring impact
  • Each tool uses its own measurement systems and may evaluate data in very different ways
  • Where you choose to gather your data can have a big impact on what your metrics look like
  • Diversify and utilize more than one metric
  • Research where your publications are indexed to get the most complete view possible of your work.

Many scholars are calling for more complex and qualitative methods for analyzing a researchers impact. For an alternative view of the evaluation of academic output see the Declaration on Research Assessment (DORA) initiated by the American Society for Cell Biology.

Subject librarians are available to meet with you individually, schedule workshops or present at departmental meetings if you would like more information about how using impact metrics.


Researcher Metrics

Citation Counts

A simple count of the number of times a researcher’s publications have been cited can demonstrate productivity and impact.

  • Citation counts are not an absolute – how many are discovered will depend on the search parameters and databases consulted
  • Results also depend on the particular database subscription. Databases frequently cover different time periods even when they index the same journal which can affect citation counts

Find instructions below about how to generate citation counts for the following platforms:

Many of the databases available on the EBSCO platform offer the ability to search specifically for cited references – this can be a time consuming, but effective way for scholars to find citation information, especially when they are in the humanities or social sciences. Unfortunately, it is unlikely that all of a researcher’s publications will be indexed in a single database requiring multiple database searches.

From the search screen of an Ebsco database:

  • Click on Cited References in the top navigation (it may be under a More link).
  • Search for the author (Last name, first initial, example: smith b) or article title - make sure you have the correct author
  • Look for the phrase Times Cited in this Database. Add up the numbers to get a citation count for that database
  • To view the articles citing the original article, check the box(es) next to the results that are cited in the database and click on the Find Citing Articles button.

With a Google account, it is possible to easily set up Google Citations to track and generate citation metrics. Google Scholar can be particularly helpful when it comes to finding citation metrics for books. However, relying on Google Scholar data can result in some issues that should be kept in mind:

  • Google metadata and citation information can be problematic: incomplete, lacking in authority controls, and easy to manipulate (see Further Reading below)
  • Google indexes a wider range of publications than most subscription databases which may include citations from things like conference proceedings (good) or unscholarly sources like course syllabi (bad)
  • It only indexes what is available online
  • Difficult to disambiguate similar/identical author names

In short, it is recommended that Google Citations be used as a complement to other citation counting methods rather than as a substitute for them.

Further Reading:

Google Scholar Citations site

Lopez-Cozar, E., Robinson-Garcia, N. and Torres-Salinas, D. (2013). Manipulating Google Scholar Citations and Google Scholar Metrics: Simple, easy and tempting. Retrieved from: http://arxiv.org/abs/1212.0638

PubMed indexes articles about the medical field and life sciences. The citation count represents the number of articles from within the PubMed Central collection of full text journals that cite your article. It does not include citations from any other sources.

To find citations:

  • Go to http://www.ncbi.nlm.nih.gov/pubmed and search for the article you are looking for.
  • Click on the title in the results list to see the full record.
  • Look at the right hand column and scroll down until you see Cited by # PubMed Central Articles. If this section does not show up, your article has not been cited in PubMed Central.
  • For a particular researcher's citations, search for their name (last name, first initial, second initial if known. Example: smith ba), check to make sure all of the records are authored by the correct researcher, and then click on each article title to view the citations.

Further Reading:

This tool can be used to get citation counts using a PubMed ID. It was developed by Harvard University’s Clinical and Translational Science Center : http://profiles.catalyst.harvard.edu/?pg=bibliometrics&tool=pmid2cites

NDSU Libraries’ subscription to WoS includes Science Citation Index Expanded to 1900 and Social Sciences Citation Index to 1956. To find citations:

  • Click on the Cited Reference Search link just above the search box on the WoS home page.
  • Enter the author's name (Example: buck, lb), the journal the article was published in, and/or the publication year for the work that you want to find citations for and click search.

  • Review the results to make sure you have found the correct article. You will get a citation count on the results page and you can check the box next to the article and click on the Finish Search button to see the actual citations.

  • You can also just search for an author, carefully review the results list to make sure you have the right one, and add up the citations right from the results page.

h-index

The h-index is a measure of how many papers a scholar has published which have reached a certain level of influence or impact. The more papers a scholar has published that have been cited and the more citations per paper, the higher the h-index. Notice that the h-index does not take into account the prestige of the journal in which the article was published.

How is it measured?

Two pieces of information are required: the total number of papers published and the number of times each paper has been cited. To calculate the h-index, determine how many h of a researcher’s publications have at least h citations each. 

Example: you have published one paper and it has been cited one time, you have an h-index of one, if you have published two papers that have been cited at least twice, you have an h-index of two. Alternately, if you have published two papers and they have been cited at least ten times each your h-index is still two.

Using Web of Science (WoS) to find the h-index:

  • Navigate to the Web of Science database through the NDSU Libraries website

  • At the top left, click on Author Search

  • Enter last name, first initial (example: Smith B) and select the research domain to narrow your results and click Search

  • Click on the Record Sets tab to review author names and associated records to make sure you have the correct researcher

  • Check the box next to the desired records and click on View Records

  • Click on the Create Citation Report link on the right side of the page

  • The Citation Report begins with two graphs, Published Items in Each Year and Citations in Each Year. To the right of the graphs, you will find the h-index

Further Reading:

Hirsh, J. (2005). An index to quantify an individual's scientific research output. Retrieved from: http://www.pnas.org/content/102/46/16569.full

Bar-Ilan, J. (2008). Which h-index? – A comparison of WoS, Scopus and Google Scholar. Scientometrics 74 (2), 257-271. doi: 10.1007/s11192-008-0216-y. Retrieved from: http://link.springer.com/article/10.1007%2Fs11192-008-0216-y

Thomson Reuter’s h-index tutorial


Journal Metrics

Journal metrics are designed to measure the impact and influence of journals rather than individual researchers. Below is information about three common journal impact measures and how to find them.

What does it measure?

The Impact Factor measures the number of citations received by a journal.

How is it measured?

The Impact Factor is calculated by dividing the number of citations to a journal by the number of articles published in the journal. For example, the 2012 Impact Factor is calculated by taking the number of citations in 2012 to articles published in the same journal in 2010-2011 divided by the number of articles published by the journal in 2010-2011.

The equation looks like this:

How do I get the data?

  • Go to the Databases A to Z list and click on the link to Journal Citation Reports (Thomson Reuters).
  • Select either the JCR Science or JCR Social Sciences Edition and choose the year.
  • You are given the option to search for a specific journal, view a list of all journals, or view a list of journals by subject, publisher, or country.

The results page will display a table of data with the Impact Factor and other impact metrics. Click on the journal’s title to view more information about the journal and additional citation data. By default, a results list with multiple journals will display alphabetically by title. You can use the drop down menu at the top to sort by Impact Factor, or by another metric.

Further Reading:

Garfield, E. The Thomson Reuters Impact Factor. Retrieved from: http://wokinfo.com/essays/impact-factor/

Journal Citation Reports video tutorials

What does it measure?

SJR measures both the number of citations received by a journal and the quality of the journals that the citations came from. This allows journals to be ranked according to the impact and prestige they have in their discipline.

How is it measured?

SJR was inspired by Google’s PageRank algorithm and assumes that not all citations are equal; that is, citations from prestigious sources boost the prestige of the journal cited. The data for the calculations is drawn from the SciVerse Scopus database and includes citations from the last three years.

How do I get the data?

NOTE: SCImago is run by a European research group. In Europe, number notation is done differently than in the United States. The comma is used to denote decimal place and periods are used for numbers over a thousand.

Example:
United States: 1,000.00
Europe: 1.000,00

Further Reading:

SCImago

Gonzalez-Pereira, B., Guerro-Bote, V. and Moya-Anegon, F. The SJR indicator: A new indicator of journals' scientific prestige. Retrieved from: http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf

What does it measure?

The Eigenfactor Score is a measure of a journal’s importance to the scholarly community. The score is generated from the citation information found in Journal Citation Reports (JCR) and utilizes five years’ worth of citation data. Journals that are frequently cited by other influential journals are scored as influential themselves. Eigenfactor is a research project co-founded by Jevin West and Carl Bergstrom of the Department of Biology at the University of Washington.

Article Influence measures the average influence of each journal’s articles over a five year period.

How do I get the data?

Go to http://www.eigenfactor.org/index.php and enter the journal title or subject area you are interested in to get both the Eigenfactor Score and Article Influence score.

Eigenfactor continues to evolve and is developing new ways to map and visualize the information it is collecting. They are also developing new metrics (for example, their Cost Effectiveness for Open Access Journals measures the prestige of articles versus the publication charges) so they will be worth keeping an eye on as they continue their research.

Further Reading:

Bergstrom, C. (2007). Eigenfactor: Measuring the prestige and value of scholarly journals. Retrieved from: http://octavia.zoology.washington.edu/publications/Bergstrom07.pdf

Eigenfactor FAQ


Impact Metrics Comparison Charts

Compare Journal Metrics

Adapted from: Gonzalez-Pereira, B., Guerro-Bote, V. and Moya-Anegon, F. The SJR indicator: A new indicator of journals' scientific prestige. Retrieved from: http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf

Compare Citation Data Sources

Adapted from: http://guides.lib.uconn.edu/c.php?g=606667&p=4206159

Further Reading:

Meho, L. I. and Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 58, 2105–2125. doi: 10.1002/asi.20677. Retrieved from: http://onlinelibrary.wiley.com/doi/10.1002/asi.20677/full

Last updated: 7/28/2017