These are the six principles for assessing scientists

Friday 22 June 2018

We all know the problem: when hiring or promoting scientists, you will never know who’s contribution will be the greatest and we also know that publication volume is not the best –although by far the easiest – indicator of quality. For this reason, many institutions consider the JIF (journal impact factor) as a more reliable sign of competence. But even though JIF serves as a good indicator for citation influence of the journal as a whole, it does not offer information about the individual publication or the authors of the paper. To address this problem, Florian Naudet, John P. A. Ioannidis, Frank Miedema, Ioana A. Cristea, Steven N. Goodman and David Moher present six principles for assessing scientists for hiring, promotion and tenure.

Not only does the JIF fail to give proper information about the authors or a specific scientist, the phenomenon of publication bias plays a big role. Up to 97% of the papers published in three leading journals in the field, presented statistical significance. This clearly demonstrates the fact that statistical significance of results will affect the publication prospects considerably. This allows for practises such as p-hacking, salami science, sloppy methods, selective reporting, low transparency and other practices that promote publication and research, but not necessarily reproducible research.
To address these issues, the Meta-research Innovation Center at Stanford (METRICS) convened a workshop in January 2017 to discuss and propose strategies to hire, promote and tenure scientists. The outcomes of this workshop mainly consisted of a list of current proposals, summarised in the following perspective:

PRINCIPLE 1. Contributing to societal needs is an important goal of scholarship.
PRINCIPLE 2. Assessing faculty should be based on responsible indicators that reflect more fully the contribution to the scientific enterprise
PRINCIPLE 3. We should reward publishing and/or reporting of all research completely and transparently regardless of the results
PRINCIPLE 4. The culture of Open Research needs to be rewarded
PRINCIPLE 5. It is important to fund research that can provide an evidence base to inform optimal ways to assess science and faculty
PRINCIPLE 6. Funding out-of-the-box ideas needs to be valued in promotion and tenure decisions

Many of these principles are not new, they need preparation to introduce them into the daily scientific life. Also, the principles need to be further defined in order to implement them correctly, and once these principles are implemented, the outcomes (both positive and negative) need to be monitored. We are at a crucial time in the movement of research reform and need to take full advantage of this opportunity. We need to make changes now. We are not there yet, but we might be getting closer.

Read more about the six principles on the LSE Impact Blog.

« back to the overview

No Comments