Responsible metrics

Metrics are used to measure many things in academia, from individual researchers to entire institutions.  These numerical measures were originally developed to help librarians make informed decisions about stock selection but their use has evolved to include major decisions for researchers such as who will be hire and which journal they should aim to publish in.  These are concerns that the metrics used to make such important decisions were not designed for these uses and so are not fit for purpose.  Many researchers are also concerned that the emphasis being placed on metrics means that more importance is being placed on where work is published rather than the quality of the research of the research itself - a dangerous trend. 

The Responsible Metrics movement aims to make sure that the use of metrics is more balanced to produce a more rounded picture of impact. It argues that although metrics have value, they only tell one (numerical) part of the wider story.  Those assessing both researchers and their work should consider multiple factors rather than just often meaningless statistics.  The movement also advocates a more open approach to measurement in line with other open research practices.

Responsible metrics movement

The overreliance on metrics to assess academic outputs has led to the call for a more responsible use of these measures. This short video outlines the key principles of the responsible metrics movement. The video was created as part of the Research Support Ambassador Programme from Cambridge University Libraries.

The movement towards responsible metrics has gained momentum in recent years with several influential documents being produced including DORA - the San Francisco Declaration on Open Research (2012), the Leiden Manifesto for Research Metrics (2014) and the Metric Tide report (2015).

Many institutions across the globe are beginning to sign up to existing principles or issue their own statements encouraging their research community to embrace the responsible use of metrics. The University of Stirling has now signed DORA

Responsible Metrics

Although each of the documents is subtly different, they all contain some themes common to the responsible use of metrics.  

QUALITATIVE AND QUANITITATIVE

Responsible metrics advocates a mixture of both types of impact to produce a more rounded picture of influence.  Although numeric counts have a place, they should be complimented with wider measures such as mentions of the work in the popular press.  Tools such as Altmetric have gone some way towards this but there is still more work to be done.

OPENNESS

Linked to the general move towards open research, the methods used to calculate different metrics should be open and available.  This enables researchers to audit the measures and better understand how they are devised.  Crucially this means that any metrics used are verifiable which is a vital part of ensuring the integrity of the research process.

QUALITY

A key concern from researchers is that the metrics achieved by a particular piece of research or title are becoming more important than the findings themselves.  With authors under pressure to publish in titles with the highest weightings they are concerned that the merits of the actual research are being overlooked. There is also a need to consider the global nature of research and ensure that work being done outside of the global North is judged equally on it's merits and not penalized for not appearing in "high impact" journals

RANGE

It is important to consider a range of measures when assessing any one researcher, institution or output as relying on only one metric can give a distorted interpretation.  Different measures are calculated differently and will present different results for the same output which can be confusing.  Certain metrics are often popular in particular disciplines and again this can create a problem of narrow focus.  Finally it is important to consider a range of metrics to avoid relying on those which are biased towards researchers at a more advanced career stage.

REVIEW

Any metrics used (no matter what they are measuring) should be regularly reviewed and updated to ensure they are still fit for purpose.  This is also a chance to adapt them to take into account developments in scholarly publishing, something not all existing metrics have managed to achieve.  The aim is to keep the metrics used up to date and stop using any which are no longer useful.

FURTHER INFORMATION

It may help to familiarize yourself with responsible metrics by reading through some of the key documentation

• DORA
• The Metric Tide

This work was licensed under a Creative Commons CC-BY 4.0 license by the Office of Scholarly Communications, Cambridge University Libraries