/*Font style and formatting for LibGuides*/ Skip to Main Content

Scholarly Communications

Responsible Research Assessment - A Brief Overview

Resources for Responsible Research Assessment

International Responsible Research Assessment Initiatives

  • San Francisco Declaration on Research Assessment (DORA): The first initiative to push for responsible research assessment; started at the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012 in response to increasing pressure to rely exclusively on Journal Impact Factors (JIFs) for the assessment of quality of research, hiring decisions, grant dollars, etc. DORA focuses more on the STEM fields.

  • The Metric Tide: The second initiative for responsible research evaluation, published in 2015. "This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration." Highly recommended that you read the Executive Summary and Recommendations on pages vii - xiv.

  • Bibliometrics: The Leiden Manifesto for research metrics: The third initiative to urge responsible research evaluation, published in 2015. Can be more broadly applied to other fields outside STEM. Urges qualitative evaluation in conjunction or complementary to quantitative assessment in all cases. Written by leading experts in research evaluation and bibliometrics.

  • HuMetricsHSS: HuMetricsHSS is an initiative for rethinking humane indicators of excellence in academia, focused particularly on the humanities and social sciences (HSS). Comprised of individuals and organizations from the academic, commercial, and non-profit sectors, HuMetricsHSS endeavors to create and support a values-based framework for understanding and evaluating all aspects of the scholarly life well-lived and for promoting the nurturing of these values in scholarly practice.

Readings on the Uses and Abuses of Metrics

  • Against metrics: how measuring performance by numbers backfires: A summary and review of the book, The Tyranny of Metrics, by Jerry Z. Muller.
  • The Tyranny of Metrics: Virginia Tech users can read the full eBook by Jerry Z. Muller by going to this link. Brief summary: How the obsession with quantifying human performance threatens our schools, medical care, businesses, and government.
  • The Tyranny of Metrics - Colleges and Universities: Virginia Tech users can read the book chapter, Colleges and Universities, from The Tyranny of Metrics by going to this link. Excerpt: "Let’s take as our first case study the realm of higher education, the ground zero of my own investigations of metric fixation. Comprising a huge sector of the national economy and a central institution of all advanced societies, colleges and universities exemplify many of the characteristic flaws and unintended consequences of measured performance, as well as some of its advantages. Once we become fixated on measurement, we easily slip into believing that more is better."
  • Rethinking impact factors: better ways to judge a journal: "We need a broader, more-transparent suite of metrics to improve science publishing, say Paul Wouters, colleagues and co-signatories."
  • Bibliometrics and Research Evaluation: Uses and Abuses: Why bibliometrics is useful for understanding the global dynamics of science but generate perverse effects when applied inappropriately in research evaluation and university rankings.

Responsible Research Assessment in Practice

Further Readings and Resources

Responsible University Rankings