Dutch universities and research funders move away from the impact factor

One year ago, the Dutch universities, university medical centers and research institutes, together with funding agencies NWO and ZonMW put forward the ambition to revisit their collective system of recognition and rewards. Last week the position paper “Room for everyone’s talent” – a collaborative document – was published, marking the result of a year’s worth of meetings and dozens of site visits with academics.

Not excellent in everything

First of all, parties acknowledge that there is a one sided focus on research output in academia, leading to an undervaluation of the many other domains in which academics are active. Additionally, the valuation of research is restricted primarily to those measures that are easily quantified, mistaking numbers for academic quality and other virtues.

Parties argue that a drastic change is required. Academics will see a reduced emphasis on quantitative results, and a stronger appreciation of aspects like scientific integrity, collaboration and teaching. “Not everyone has to be good at everything”, the position paper states, “it is not realistic, and not necessary to expect every academic to excel in each of the core domains.

Those core domains, as indicated by the parties involved, are: education, research, impact, leadership and patient care. While an academic does not have to excel in all these terrains, within universities (read: not research institutes or academic hospitals), academics are still expected to have sufficient competence in both research and education.

Universities, medical centers and research institutes realize that in order to facilitate this transition, much more effort has to be made in the field of human resources and career planning. Allowing for more diverse career paths, and stimulating academic leadership across all levels. Within this vision, open science is considered to be inseparable from the recognition and reward structure.

Complete implementation of DORA principles

By the end of 2019, all parties involved in this project pledge to have signed DORA The San Francisco Declaration on Responsible Assesment (DORA) aims to halt the use of the imact factor of academic articles for the purpose of evaluating the value of individual academics’ work. The VSNU signed DORA in 2014. The KNAW, NWA and ZonMW did so last April, and the NFU will do so before the end of 2019. . This commitment has to be more than an empty gesture. For example, norms such as ‘four publications to obtain a PhD’ will be abolished, and NWO and ZonMw will no longer inquire about h-index or journal impact factor when academics submit grant applications. Instead of asking for a publication list and CV, they will ask for a more ‘narrative’ approach – inquiring about why this research is important, and why the applicant is the right person to carry it out.

This change will be fast, but not instant. The parties involved acknowledge that change takes time. Considering that to focus metrics such as impact factors took decades to become part of established practices, unlearning these routines will require a considerable amount of time, energy and perseverance. Correctly identifying diverse forms of talent and ‘good research’ will be a learning experience: “To accelerate the desired cultural change in recognition and rewards, we at NWO and ZonMW will strongly focus on training and instruction for our grant evaluation committees.”

Function profiles and evaluation protocols

Adjusting the culture of recognition and reward will also change the collective labor agreements and function profiles of universities. The plan also indicates the universities’ function ordering system – essentially an extension of the labor agreement – will be revised in 2020, and a recalibrated version will take effect in 2021.

The new criteria for recognition and reward will also be applied at the level of research groups within the Standard Evaluation Protocol (SEP). “With this, we implement the DORA principles for evaluating research quality at the level of the research group. More focus will be placed on societal impact, open science, diversity and talent policy.”

Many thanks to Jasper Winkel who assisted in translating this article from Dutch.