# Leiden Manifesto

> Mediated Wiki article. Canonical URL: https://mediated.wiki/source/Leiden_Manifesto
> Markdown URL: https://mediated.wiki/source/Leiden_Manifesto.md
> Source: https://en.wikipedia.org/wiki/Leiden_Manifesto
> Source revision: 1345731860
> License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/)

List of "ten principles to guide research evaluation"

This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (December 2020) (Learn how and when to remove this message)

***The Leiden Manifesto for research metrics*** (**LM**) is a list of "ten principles to guide research evaluation",[1] published as a comment in [*Nature*](/source/Nature_(journal)) on 22 April 2015. It was formulated by [public policy](/source/Public_policy) professor Diana Hicks, [scientometrics](/source/Scientometrics) professor Paul Wouters, and their colleagues at the 19th International Conference on Science and Technology Indicators, held between 3–5 September 2014 in [Leiden](/source/Leiden), The Netherlands.[2]

The Leiden Manifesto was proposed as a guide to combat misuse of [bibliometrics](/source/Bibliometrics) when evaluating scientific research literature. Examples of commonly used bibliometrics for science, or [scientometrics](/source/Scientometrics), are the [h-index](/source/H-index), [impact factor](/source/Impact_factor), other [indicators](/source/Indicator_(statistics)) such as [Altmetrics](/source/Altmetrics). According to the Manifesto's authors, these metrics often pervasively misguide evaluations of scientific material.[3]

The Leiden Manifesto for research metrics Authors Diana Hicks, Paul Wouters, Ludo Waltman, Sarah de Rijcke, Ismael Rafols Publisher Nature Publication date 22 April 2015 Website leidenmanifesto.org

## Motivation

Motivations for codification of the Leiden Manifesto arose from a growing worry that "impact-factor obsession"[1] was leading to inadequate judgement of scientific material that should be worthy of fair evaluation. Lead author Diana Hicks hoped that publishing the list in *Nature* would spread its ideas, already commonplace in the scientometrics sphere, to the broader scientific community.[4]

### DORA and other predecessors

The scientific community has long been interested in reforming assessment of the impact of scientific and academic research. The 2013 [San Francisco Declaration on Research Assessment (DORA)](/source/San_Francisco_Declaration_on_Research_Assessment), which has been signed by over 27,000 individuals as of March 2026, was a major influence on the Leiden Manifesto. The Declaration denounced common practices in research assessment, such as using journal impact factor to assess the contributions of individual researchers.[5]

One of the main concerns about overuse of citation-based performance indicators came from the observation that smaller research organizations and institutions may be negatively affected by their metric indices. In one public debate at the Centre for Science and Technology Studies at [Leiden University](/source/Leiden_University), it was acknowledged that indicators which measure citations may give "more weight to publications from fields with a high expected number of citations than to publications from fields with a low expected number of citations".[6]

Although the main focus of the Leiden Manifesto is the use of scientometrics for research evaluation, the authors also consider how overuse of metrics can adversely affect the wider scholarly community, such as the position of universities in global rankings.[7][8] According to Hicks et al., scientific metrics such as citation rate are used far too much for ranking the quality of universities (and thus the quality of their research output).

### Journal impact factor

Further information: [Impact factor § Criticism](/source/Impact_factor#Criticism)

The background of the Leiden Manifesto describes why misusing metrics is becoming a larger problem in the scientific community. The [journal impact factor](/source/Journal_impact_factor), originally created by [Eugene Garfield](/source/Eugene_Garfield) as a method for librarians to collect data to facilitate selecting journals to purchase, is now mainly used as a method of judging journal quality.[9] This is seen by the authors as an abuse of data in order to examine research too hastily. For example, an impact factor, while a good metric to measure the size and experience of a journal, may or may not be sufficient to accurately describe the quality of its papers, and even less so for a single paper.

## Content

The Leiden Manifesto consists of ten principles which aim to reform how research quality is assessed by academic publishers and institutions. It emphasizes detailed and close evaluation of research, rather than relying exclusively on quantitative data. It also aims to remove possible [perverse incentives](/source/Perverse_incentive) for using scientometrics, such as judgement of academic capability and university quality.[10]

### Ten principles

The ten principles of the Leiden Manifesto are as follows:[1]

1. Quantitative evaluation should support qualitative, expert assessment.

1. Measure performance against the research missions of the institution, group, or researcher.

1. Protect excellence in locally relevant research. - Allow research taking place in a certain area or field to be published in corresponding local research publications, instead of prioritizing [high-impact journals](/source/Mega_journal). Many high-impact journals are in English, which may decrease needed specificity when publishing a paper meant to study locational characteristics. As an example, in high-impact Spanish-language papers, "topics such as local labor laws" and other features designated for sociologists may be lost.[11]

1. Keep data collection and analytical processes open, transparent, and simple.

1. Allow those evaluated to verify data and analysis.

1. Account for variation by field in publication and citation practices. - Peer-review and citation rate can vary wildly across differing disciplines, for example, "top-ranked journals in mathematics have impact factors of around 3; top-ranked journals in cell biology have impact factors of about 30".[12]

1. Base assessment of individual researchers on a qualitative judgement of their portfolio.

1. Avoid misplaced concreteness and [false precision](/source/False_precision). - Use of scientific indicators may precede strong assumptions that are not necessarily correct. For example, when looking at a specific scientist, a low citation rate may lead the investigator to assume low research quality, which is [implying causation from correlation](/source/Correlation_does_not_imply_causation). Providing clarification, as well as multiple, robust indicators, may reduce inappropriate concreteness. [False precision](/source/False_precision) is possible when indicator producers, such as [Clarivate](/source/Clarivate) (which publishes the annual [Journal Citation Reports](/source/Journal_Citation_Reports)) attempt to create an exact journal impact factor (i.e. three decimal places). Conceptual ambiguity and [random variability](/source/Random_variable) of citation counts make it unnecessary to distinguish indices such as journal impact factors to such a precise extent, because it can foster excessive comparison and competition between publishers.[1]

1. Recognize the systemic effects of assessment and indicators.

1. Scrutinize indicators regularly and update them.

## Reception

### 2016 John Ziman Award

In 2016, the [European Association for the Study of Science and Technology (EASST)](/source/European_Association_for_the_Study_of_Science_and_Technology) gave its [John Ziman](/source/John_Ziman) Award to the Leiden Manifesto for its effort to widen scientometrics knowledge to the scientific community as a whole. EASST president Fred Steward stated that it "emphasizes situatedness, in terms of different cognitive domains and research missions as well as the wider socioeconomic, national and regional context".[13]

### Public endorsement

[LIBER](https://libereurope.eu/), a collaboration of European research libraries, issued a substantial review of the Leiden Manifesto in 2017, concluding that it was a "solid foundation" on which academic libraries could base their assessment of metrics.[14]

[Elsevier](/source/Elsevier), a global leader in research publishing and information analytics, announced on 14 July 2020 that it would use the Leiden Manifesto to guide its development of improved research evaluation. Elsevier stated that the principles of the manifesto were already close in nature to their 2019 [CiteScore](/source/CiteScore) metrics, which was in summary "improved calculation methodology" for "a more robust, fair and faster indicator of research impact".[15]

[Loughborough University's](/source/Loughborough_University) LIS-Bibliometrics committee chose to base their own principles on the Leiden Manifesto, instead of the DORA, because the manifesto takes a "broader approach to the responsible use of all bibliometrics across a range of disciplines and settings", according to their policy manager Elizabeth Gadd.[16] Stephen Curry, chair of the DORA steering committee, commented on this statement by emphasizing that DORA was aiming to extend its "disciplinary and geographical reach".[17]

### Further applications

[David Moher](/source/David_Moher) and his co-authors referenced the Leiden Manifesto in a perspective for *[Issues in Science and Technology](/source/Issues_in_Science_and_Technology),* writing that academic institutions were not asking the "right questions" (concerning research planning, timeframe, reproducibility, and results) when assessing scientists. They criticize what they see as an obsession with journal impact factors and the "gaming" of scientometrics, advocating that institutions use DORA and the Leiden Manifesto principles instead when assessing individual scientists and research.[18]

In a letter in *[Science and Engineering Ethics](/source/Science_and_Engineering_Ethics),* T. Kanchan and [Kewal Krishan](/source/Kewal_Krishan_(forensic_anthropologist)) called the Leiden Manifesto "one of the best criteria" for assessing scientific research, especially considering the "rat race" for publications in the scholarly community. They also argue that use of the Manifesto will lead to "progress of science and society at large".[19]

## See also

- [San Francisco Declaration on Research Assessment](/source/San_Francisco_Declaration_on_Research_Assessment)

- [Nature (journal)](/source/Nature_(journal))

- [Scientometrics (journal)](/source/Scientometrics_(journal))

- [Bibliometrics](/source/Bibliometrics)

- [Scientometrics](/source/Scientometrics)

- [H-index](/source/H-index)

- [Impact factor](/source/Impact_factor)

- [Altmetrics](/source/Altmetrics)

## References

1. ^ [***a***](#cite_ref-:0_1-0) [***b***](#cite_ref-:0_1-1) [***c***](#cite_ref-:0_1-2) [***d***](#cite_ref-:0_1-3) Hicks, Diana; Wouters, Paul; Waltman, Ludo; de Rijcke, Sarah; Rafols, Ismael (2015-04-22). ["Bibliometrics: The Leiden Manifesto for research metrics"](https://doi.org/10.1038%2F520429a). *Nature*. **520** (7548): 429–431. [Bibcode](/source/Bibcode_(identifier)):[2015Natur.520..429H](https://ui.adsabs.harvard.edu/abs/2015Natur.520..429H). [doi](/source/Doi_(identifier)):[10.1038/520429a](https://doi.org/10.1038%2F520429a). [hdl](/source/Hdl_(identifier)):[10261/132304](https://hdl.handle.net/10261%2F132304). [ISSN](/source/ISSN_(identifier)) [0028-0836](https://search.worldcat.org/issn/0028-0836). [PMID](/source/PMID_(identifier)) [25903611](https://pubmed.ncbi.nlm.nih.gov/25903611).

1. **[^](#cite_ref-2)** ["STI 2014 Leiden - 19th International Conference on Science and Technology Indicators"](http://sti2014.cwts.nl/Home/). *STI 2014*. Retrieved 2020-11-30.

1. **[^](#cite_ref-3)** ["Leiden manifesto for research Metrics"](http://www.leidenmanifesto.org/). *Leiden manifesto for research Metrics*. Retrieved 2020-12-04.

1. **[^](#cite_ref-:1_4-0)** ["Best Practices in Research Metrics: A Conversation with the lead author of the Leiden Manifesto - YouTube"](https://www.youtube.com/watch?v=UY7vYmAd9d0&feature=youtu.be). *www.youtube.com*. 14 October 2020. Retrieved 2020-12-04.

1. **[^](#cite_ref-5)** ["San Francisco Declaration on Research Assessment"](https://sfdora.org/read/). *DORA*. Retrieved 2026-03-27.

1. **[^](#cite_ref-6)** van Raan, Anthony F.J.; van Leeuwen, Thed N.; Visser, Martijn S.; van Eck, Nees Jan; Waltman, Ludo (July 2010). ["Rivals for the crown: Reply to Opthof and Leydesdorff"](https://linkinghub.elsevier.com/retrieve/pii/S1751157710000301). *Journal of Informetrics*. **4** (3): 431–435. [arXiv](/source/ArXiv_(identifier)):[1003.2113](https://arxiv.org/abs/1003.2113). [doi](/source/Doi_(identifier)):[10.1016/j.joi.2010.03.008](https://doi.org/10.1016%2Fj.joi.2010.03.008). [hdl](/source/Hdl_(identifier)):[1887/15078](https://hdl.handle.net/1887%2F15078). [S2CID](/source/S2CID_(identifier)) [10780959](https://api.semanticscholar.org/CorpusID:10780959).

1. **[^](#cite_ref-7)** ["World University Rankings"](https://www.timeshighereducation.com/world-university-rankings). *Times Higher Education (THE)*. 2018-09-19. Retrieved 2020-12-04.

1. **[^](#cite_ref-8)** ["ARWU World University Rankings 2020 | Academic Ranking of World Universities"](http://www.shanghairanking.com/). *www.shanghairanking.com*. Retrieved 2020-12-04.

1. **[^](#cite_ref-9)** Larivière, Vincent; [Sugimoto, Cassidy R.](/source/Cassidy_Sugimoto) (2019), Glänzel, Wolfgang; Moed, Henk F.; Schmoch, Ulrich; Thelwall, Mike (eds.), "The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects", *Springer Handbook of Science and Technology Indicators*, Springer Handbooks, Cham: Springer International Publishing, pp. 3–24, [arXiv](/source/ArXiv_(identifier)):[1801.08992](https://arxiv.org/abs/1801.08992), [doi](/source/Doi_(identifier)):[10.1007/978-3-030-02511-3_1](https://doi.org/10.1007%2F978-3-030-02511-3_1), [ISBN](/source/ISBN_(identifier)) [978-3-030-02511-3](https://en.wikipedia.org/wiki/Special:BookSources/978-3-030-02511-3), [S2CID](/source/S2CID_(identifier)) [3677889](https://api.semanticscholar.org/CorpusID:3677889){{[citation](https://en.wikipedia.org/wiki/Template:Citation)}}: CS1 maint: work parameter with ISBN ([link](https://en.wikipedia.org/wiki/Category:CS1_maint:_work_parameter_with_ISBN))

1. **[^](#cite_ref-10)** [*The Leiden Manifesto for Research Metrics*](https://vimeo.com/133683418), retrieved 2020-12-05

1. **[^](#cite_ref-11)** López Piñeiro, Carla; Hicks, Diana (2015-01-01). ["Reception of Spanish sociology by domestic and foreign audiences differs and has consequences for evaluation"](https://academic.oup.com/rev/article/24/1/78/1544815). *Research Evaluation*. **24** (1): 78–89. [doi](/source/Doi_(identifier)):[10.1093/reseval/rvu030](https://doi.org/10.1093%2Freseval%2Frvu030). [ISSN](/source/ISSN_(identifier)) [0958-2029](https://search.worldcat.org/issn/0958-2029).

1. **[^](#cite_ref-12)** Waltman, Ludo; Calero-Medina, Clara; Kosten, Joost; Noyons, Ed C. M.; Tijssen, Robert J. W.; Eck, Nees Jan van; Leeuwen, Thed N. van; Raan, Anthony F. J. van; Visser, Martijn S.; Wouters, Paul (2012). ["The Leiden ranking 2011/2012: Data collection, indicators, and interpretation"](https://onlinelibrary.wiley.com/doi/abs/10.1002/asi.22708). *Journal of the American Society for Information Science and Technology*. **63** (12): 2419–2432. [arXiv](/source/ArXiv_(identifier)):[1202.3941](https://arxiv.org/abs/1202.3941). [doi](/source/Doi_(identifier)):[10.1002/asi.22708](https://doi.org/10.1002%2Fasi.22708). [ISSN](/source/ISSN_(identifier)) [1532-2890](https://search.worldcat.org/issn/1532-2890). [S2CID](/source/S2CID_(identifier)) [935166](https://api.semanticscholar.org/CorpusID:935166).

1. **[^](#cite_ref-13)** ["2016 John Ziman Award for Leiden Manifesto"](https://www.cwts.nl:443/news?article=n-q2x294). *CWTS*. Retrieved 2020-12-05.

1. **[^](#cite_ref-14)** Coombs, Sarah K.; Peters, Isabella (2017-11-13). "The Leiden Manifesto under review: what libraries can learn from it". *Digital Library Perspectives*. **33** (4): 324–338. [doi](/source/Doi_(identifier)):[10.1108/dlp-01-2017-0004](https://doi.org/10.1108%2Fdlp-01-2017-0004). [hdl](/source/Hdl_(identifier)):[11108/323](https://hdl.handle.net/11108%2F323). [ISSN](/source/ISSN_(identifier)) [2059-5816](https://search.worldcat.org/issn/2059-5816).

1. **[^](#cite_ref-15)** ["Elsevier endorses Leiden Manifesto to guide its development of improved research evaluation"](https://www.elsevier.com/about/press-releases/corporate/elsevier-endorses-leiden-manifesto-to-guide-its-development-of-improved-research-evaluation). *www.elsevier.com*. Retrieved 2020-12-05.

1. **[^](#cite_ref-16)** Gadd, Lizzie (2018-07-09). ["DORA, the Leiden Manifesto & a university's right to choose"](https://thebibliomagician.wordpress.com/2018/07/09/dora-the-leiden-manifesto-a-universitys-right-to-choose/). *The Bibliomagician*. Retrieved 2020-12-05.

1. **[^](#cite_ref-17)** Stephen. ["DORA, the Leiden Manifesto & a university's right to choose: a comment"](http://occamstypewriter.org/scurry/2018/07/18/dora-leiden-manifesto-right-to-choose/). *Reciprocal Space*. Retrieved 2020-12-05.

1. **[^](#cite_ref-18)** Moher, David; Naudet, Florian; Cristea, Ioana A.; Miedema, Frank; Ioannidis, John P. A.; Goodman, Steven N. (2018-03-29). ["Assessing scientists for hiring, promotion, and tenure"](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5892914). *PLOS Biology*. **16** (3) e2004089. [doi](/source/Doi_(identifier)):[10.1371/journal.pbio.2004089](https://doi.org/10.1371%2Fjournal.pbio.2004089). [ISSN](/source/ISSN_(identifier)) [1545-7885](https://search.worldcat.org/issn/1545-7885). [PMC](/source/PMC_(identifier)) [5892914](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5892914). [PMID](/source/PMID_(identifier)) [29596415](https://pubmed.ncbi.nlm.nih.gov/29596415).

1. **[^](#cite_ref-19)** Kanchan, Tanuj; Krishan, Kewal (April 2019). ["The Leiden Manifesto and Research Assessment"](http://link.springer.com/10.1007/s11948-017-0012-2). *Science and Engineering Ethics*. **25** (2): 643–644. [doi](/source/Doi_(identifier)):[10.1007/s11948-017-0012-2](https://doi.org/10.1007%2Fs11948-017-0012-2). [ISSN](/source/ISSN_(identifier)) [1353-3452](https://search.worldcat.org/issn/1353-3452). [PMID](/source/PMID_(identifier)) [29264830](https://pubmed.ncbi.nlm.nih.gov/29264830). [S2CID](/source/S2CID_(identifier)) [29821298](https://api.semanticscholar.org/CorpusID:29821298).

---
Adapted from the Wikipedia article [Leiden Manifesto](https://en.wikipedia.org/wiki/Leiden_Manifesto) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/Leiden_Manifesto?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.