A variety of metrics can be used to quantitatively evaluate academic productivity and the impact your work has in your field. This information can be used to demonstrate the influence and importance of your research to potential funders and to NDSU. Metrics can also be used to identify top researchers in a discipline and the best journals for publishing your research.
Important considerations to keep in mind:
- There is no one perfect tool for measuring impact
- Each tool uses its own measurement systems and may evaluate data in very different ways
- Where you choose to gather your data can have a big impact on what your metrics look like
- Diversify and utilize more than one metric
- Research where your publications are indexed to get the most complete view possible of your work.
Many scholars are calling for more complex and qualitative methods for analyzing a researchers impact. For an alternative view of the evaluation of academic output see the Declaration on Research Assessment (DORA) initiated by the American Society for Cell Biology.
Subject librarians are available to meet with you individually, schedule workshops or present at departmental meetings if you would like more information about how using impact metrics.
A simple count of the number of times a researcher’s publications have been cited can demonstrate productivity and impact.
- Citation counts are not an absolute – how many are discovered will depend on the search parameters and databases consulted
- Results also depend on the particular database subscription. Databases frequently cover different time periods even when they index the same journal which can affect citation counts
Find instructions below about how to generate citation counts for the following platforms:
Many of the databases available on the EBSCO platform offer the ability to search specifically for cited references – this can be a time consuming, but effective way for scholars to find citation information, especially when they are in the humanities or social sciences. Unfortunately, it is unlikely that all of a researcher’s publications will be indexed in a single database requiring multiple database searches.
From the search screen of an Ebsco database:
- Click on Cited References in the top navigation (it may be under a More link).
- Search for the author (Last name, first initial, example: smith b) or article title - make sure you have the correct author
- Look for the phrase Times Cited in this Database. Add up the numbers to get a citation count for that database
- To view the articles citing the original article, check the box(es) next to the results that are cited in the database and click on the Find Citing Articles button.
With a Google account, it is possible to easily set up Google Citations to track and generate citation metrics. Google Scholar can be particularly helpful when it comes to finding citation metrics for books. However, relying on Google Scholar data can result in some issues that should be kept in mind:
- Google metadata and citation information can be problematic: incomplete, lacking in authority controls, and easy to manipulate (see Further Reading below)
- Google indexes a wider range of publications than most subscription databases which may include citations from things like conference proceedings (good) or unscholarly sources like course syllabi (bad)
- It only indexes what is available online
- Difficult to disambiguate similar/identical author names
In short, it is recommended that Google Citations be used as a complement to other citation counting methods rather than as a substitute for them.
Lopez-Cozar, E., Robinson-Garcia, N. and Torres-Salinas, D. (2013). Manipulating Google Scholar Citations and Google Scholar Metrics: Simple, easy and tempting. Retrieved from: http://arxiv.org/abs/1212.0638
PubMed indexes articles about the medical field and life sciences. The citation count represents the number of articles from within the PubMed Central collection of full text journals that cite your article. It does not include citations from any other sources.
To find citations:
- Go to http://www.ncbi.nlm.nih.gov/pubmed and search for the article you are looking for.
- Click on the title in the results list to see the full record.
- Look at the right hand column and scroll down until you see Cited by # PubMed Central Articles. If this section does not show up, your article has not been cited in PubMed Central.
- For a particular researcher's citations, search for their name (last name, first initial, second initial if known. Example: smith ba), check to make sure all of the records are authored by the correct researcher, and then click on each article title to view the citations.
This tool can be used to get citation counts using a PubMed ID. It was developed by Harvard University’s Clinical and Translational Science Center : http://profiles.catalyst.harvard.edu/?pg=bibliometrics&tool=pmid2cites
NDSU Libraries’ subscription to WoS includes Science Citation Index Expanded to 1900 and Social Sciences Citation Index to 1956. To find citations:
- Click on the Cited Reference Search link just above the search box on the WoS home page.
Enter the author's name (Example: buck, lb), the journal the article was published in, and/or the publication year for the work that you want to find citations for and click search.
Review the results to make sure you have found the correct article. You will get a citation count on the results page and you can check the box next to the article and click on the Finish Search button to see the actual citations.
You can also just search for an author, carefully review the results list to make sure you have the right one, and add up the citations right from the results page.
The h-index is a measure of how many papers a scholar has published which have reached a certain level of influence or impact. The more papers a scholar has published that have been cited and the more citations per paper, the higher the h-index. Notice that the h-index does not take into account the prestige of the journal in which the article was published.
How is it measured?
Two pieces of information are required: the total number of papers published and the number of times each paper has been cited. To calculate the h-index, determine how many h of a researcher’s publications have at least h citations each.
Example: you have published one paper and it has been cited one time, you have an h-index of one, if you have published two papers that have been cited at least twice, you have an h-index of two. Alternately, if you have published two papers and they have been cited at least ten times each your h-index is still two.
Using Web of Science (WoS) to find the h-index:
Navigate to the Web of Science database through the NDSU Libraries website
At the top left, click on Author Search
Enter last name, first initial (example: Smith B) and select the research domain to narrow your results and click Search
Click on the Record Sets tab to review author names and associated records to make sure you have the correct researcher
Check the box next to the desired records and click on View Records
Click on the Create Citation Report link on the right side of the page
The Citation Report begins with two graphs, Published Items in Each Year and Citations in Each Year. To the right of the graphs, you will find the h-index
Hirsh, J. (2005). An index to quantify an individual's scientific research output. Retrieved from: http://www.pnas.org/content/102/46/16569.full
Bar-Ilan, J. (2008). Which h-index? – A comparison of WoS, Scopus and Google Scholar. Scientometrics 74 (2), 257-271. doi: 10.1007/s11192-008-0216-y. Retrieved from: http://link.springer.com/article/10.1007%2Fs11192-008-0216-y
Journal metrics are designed to measure the impact and influence of journals rather than individual researchers. Below is information about three common journal impact measures and how to find them.
What does it measure?
The Impact Factor measures the number of citations received by a journal.
How is it measured?
The Impact Factor is calculated by dividing the number of citations to a journal by the number of articles published in the journal. For example, the 2012 Impact Factor is calculated by taking the number of citations in 2012 to articles published in the same journal in 2010-2011 divided by the number of articles published by the journal in 2010-2011.
The equation looks like this:
How do I get the data?
- Go to the Databases A to Z list and click on the link to Journal Citation Reports (Thomson Reuters).
- Select either the JCR Science or JCR Social Sciences Edition and choose the year.
- You are given the option to search for a specific journal, view a list of all journals, or view a list of journals by subject, publisher, or country.
The results page will display a table of data with the Impact Factor and other impact metrics. Click on the journal’s title to view more information about the journal and additional citation data. By default, a results list with multiple journals will display alphabetically by title. You can use the drop down menu at the top to sort by Impact Factor, or by another metric.
Garfield, E. The Thomson Reuters Impact Factor. Retrieved from: http://wokinfo.com/essays/impact-factor/
What does it measure?
SJR measures both the number of citations received by a journal and the quality of the journals that the citations came from. This allows journals to be ranked according to the impact and prestige they have in their discipline.
How is it measured?
SJR was inspired by Google’s PageRank algorithm and assumes that not all citations are equal; that is, citations from prestigious sources boost the prestige of the journal cited. The data for the calculations is drawn from the SciVerse Scopus database and includes citations from the last three years.
How do I get the data?
- One journal at a time: Go to http://www.scimagojr.com/journalsearch.php and enter the title of the journal you are interested in. Click on the data tab to see the SJR.
- For an overview of the ranking of journals in your discipline, go to http://www.scimagojr.com/journalrank.php and use the search fields at the top to limit by subject, category, and country.
NOTE: SCImago is run by a European research group. In Europe, number notation is done differently than in the United States. The comma is used to denote decimal place and periods are used for numbers over a thousand.
United States: 1,000.00
Gonzalez-Pereira, B., Guerro-Bote, V. and Moya-Anegon, F. The SJR indicator: A new indicator of journals' scientific prestige. Retrieved from: http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf
What does it measure?
The Eigenfactor Score is a measure of a journal’s importance to the scholarly community. The score is generated from the citation information found in Journal Citation Reports (JCR) and utilizes five years’ worth of citation data. Journals that are frequently cited by other influential journals are scored as influential themselves. Eigenfactor is a research project co-founded by Jevin West and Carl Bergstrom of the Department of Biology at the University of Washington.
Article Influence measures the average influence of each journal’s articles over a five year period.
How do I get the data?
Go to http://www.eigenfactor.org/index.php and enter the journal title or subject area you are interested in to get both the Eigenfactor Score and Article Influence score.
Eigenfactor continues to evolve and is developing new ways to map and visualize the information it is collecting. They are also developing new metrics (for example, their Cost Effectiveness for Open Access Journals measures the prestige of articles versus the publication charges) so they will be worth keeping an eye on as they continue their research.
Bergstrom, C. (2007). Eigenfactor: Measuring the prestige and value of scholarly journals. Retrieved from: http://octavia.zoology.washington.edu/publications/Bergstrom07.pdf
Impact Metrics Comparison Charts
Compare Journal Metrics
Adapted from: Gonzalez-Pereira, B., Guerro-Bote, V. and Moya-Anegon, F. The SJR indicator: A new indicator of journals' scientific prestige. Retrieved from: http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf
Compare Citation Data Sources
Adapted from: http://guides.lib.uconn.edu/c.php?g=606667&p=4206159
Meho, L. I. and Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 58, 2105–2125. doi: 10.1002/asi.20677. Retrieved from: http://onlinelibrary.wiley.com/doi/10.1002/asi.20677/full