By Ruth A. Pagell*
(8 Feb 2021) I presented existing journal quality metrics in Ruth’s Rankings 46 Part 1. JCR is adding “Early Access” documents to its 2021 calculations (Quaderi, McVeigh). Before writing about this and other updates from Clarivate Analytics and Scopus, I realized I needed to know more about the make-up of the scholarly publications’ landscape. In 2020 there were discussions about it as a year of an increase in publications. The changing landscape of scholarly publications interests me more, including Early Access (Appendix A), Preprints (Appendix B) and Retractions¹ (Appendix C). I was also interested in what impact COVID-19 is having on document types².
DOCUMENTS In 2020
Comparison to 2019: The pandemic has led to a large increase in article submissions that are overwhelming the publication chain and exposing the strengths and weaknesses of science and scholarly publishing (Anderson, 2020; Yong, 2020). I was interested in the year-on-year change for number of documents, document types, and the change in percent of articles for each category. When these data were collected in mid-January 2021, it did not show an overall unusual year on year change between 2020 and 2019 at this time. Some journal volumes dated 2020 will be published into 2021.
COVID-19: “In the fall of 2019, exactly zero scientists were studying COVID-19” but by the end of March thousands of researchers dropped whatever intellectual puzzles had previously consumed their curiosity and began working on the pandemic instead. In mere months, science became thoroughly COVID-ized” (Yong).
While it felt as if everything was about COVID-19, documents using the terms [Covid-19, Covid19, Coronavirus, OR “Corona virus”] make up about 2.5 percent of all WOS and Scopus documents (2). The COVID-19 documents are more likely to be open access and include more letters and editorials than the overall dataset. For example, the most highly cited letter/correspondence in Scopus and WOS, “COVID-19 consider cytokine storm … “, published in The Lancet had over 2,100 citations as of 2 February 2021. A COVID related editorial in International Journal of Infectious Diseases, “The continuing 2019-nCoV epidemic threat of novel coronavirus…” had over 850 citations in Scopus. I have not limited my data to citable items because of the importance of these other document types for COVID articles. For those interested in viewing the percentages, see Table 1, Background Data, which also shows the role of Letters and Editorials in COVID literature.
Top medical journals such as The Lancet and JAMA, Journal of the American Medical Association, have increased open access documents and a relative unknown Journal of Medical Virology has risen to the top 10 for citations. Except for Nature, these journals increased the numbers of Letters and Editorials. (See Table 2)
Medical subject areas have been top producers of documents and output has grown along with the growth in the size of the overall datasets. In 2020, medical topics have outperformed the database set in terms of percent growth as shown in Figure 1 below.
Figure 1: Percentage growth in documents by subject area (Data from SCOPUS, January 2021)
Nature tracked Covid output throughout 2020 with a summary in seven charts in December (Else).
This is relevant to us for many reasons:
- From the bibliometric perspective, it raises questions about how JCR and Citescore will be handling “articles in print” and how, if at all, other cited document types and preprints will be integrated in an institution’s output. In Ruth’s Rankings 46 Part 1 we revisited the traditional journal evaluation model, with a journal’s rating being limited to citations from a limited number of document types from a limited number of journals.
- Will Ranking organizations, who are being asked to expand their metrics, consider expanding beyond those bibliometrics used by JCR and Citescore? (Hazelkorn and Usher)
See Table 3 for bibliometric and international metrics used by rankers that may be affected by 2020 scholarly publication changes.
- Researchers now have more choices about where and when to publish and to decide if they want their publications Open Access.
- Administrators have to determine how they evaluate their institutions’ scholarly contributions. Do they look at different article types, personal publication sites such as ORCID, or include social media metrics such as Altmetrics? Do they include contributions from support services such as the library and data specialists? (Meadows)
- Librarians and information scientists have to continue to monitor these changes and be able to keep their user groups up to date.
We have been watching the disaggregation of the journal as a physical object and in this decade, it may disappear as a virtual object. The value of the journal as an object is to create a proper citation, but as more journals are making their articles available before they are incorporated into a journal volume, the proper citation may be migrating to a “DOI”.
These rapid changes should be bringing together publication data suppliers and users and librarians and information professionals.
Changes to Clarivate Analytics and Scopus will appear in our next News update and a discussion on the purpose of rankings will continue in our next article, which includes the new World University Research Rankings (WURR).
NOTES –
¹ Terminology 1 – Featured Document Types
Early Access/Articles in Press: Articles that are available on the publisher’s website as having gone through the publishing process but have not been assigned a journal volume and issue (Appendix A).
Preprints: – Manuscript drafts that have not yet been subject to formal peer review that are often distributed to receive early feedback on research from peers (Appendix B).
Retractions/Retracted from WOS definitions (Appendix C).
- Retracted Publication: An article that has been withdrawn by an author, institution, editor, or publisher because of errors or unsubstantiated data.
- Retraction: A public notice that an article should be withdrawn because of errors or unsubstantiated data
Retraction Watch uses different terminology: https://retractionwatch.com/retraction-watch-databaseuser-guide/retraction-watch-database-user-guide-appendix-c-article-types/
² The Virus: There has been no standardization in terminology. The World Health Organization (WHO) refers to it as “Coronavirus disease (COVID-19)”. Coronavirus is their health topic, and COVID-19 is the name for this particular strain of coronavirus while pandemic refers to the overall worldwide situation. Coronavirus first appeared in an article in Nature in 1968. For the purpose of this article I will use COVID-19 to refer to the complete dataset. All searches were performed at the end of 2020 and revisited in January 2021.
Anderson, K. (21 Dec 2020). Maximizing benefit, minimizing harm. The Geyser, accessed at https://thegeyser.substack.com/p/maximizing-benefit-minimizing-harm
Else, H. (16,17 Dec 2020). Covid in papers: A torrent of science (with seven charts)
https://media.nature.com/original/magazine-assets/d41586-020-03564-y/d41586-020-03564-y.pdf
Hazelkorn, E. and Usher, A. (25 Jan 2021). Are rankings fit for purpose? Center for International Higher Education Webinar, view at https://www.youtube.com/watch?v=ZyGjRdEht00
McVeigh, M. (28 Jan 2021). Adding Early Access content to Journal Citation Reports: choosing a prospective model. Web of Science Blog accessed at https://clarivate.com/webofsciencegroup/article/adding-early-access-content-to-journal-citation-reports-choosing-a-prospective-model/
Meadows, A. (12 Aug 2020). Beyond publication – Increasing opportunities for recognizing all research contributions. Scholarly Kitchen accessed at https://scholarlykitchen.sspnet.org/2020/08/12/beyond-publication-increasing-opportunities-for-recognizing-all-research-contributions/
Quaderi, N. (26 Oct 2020). The JCR reload and a look ahead to the introduction of early access content in 2021. Web of Science Blog accessed at https://clarivate.com/webofsciencegroup/article/the-jcr-reload-and-a-look-ahead-to-the-introduction-of-early-access-content-in-2021/
Yong, E. (14 Dec 2020). How science beat the virus and what it lost in the process. The Atlantic accessed at https://www.theatlantic.com/magazine/archive/2021/01/science-covid-19-manhattanproject/617262/
Note that there are some documents in WOS and Scopus with journal volumes dated 2019 but that does not mean the volume was published in 2019.
ATTACHMENTS: All three Appendices include examples
Appendix A: Articles in Press containing Figure A.1 Growth of Early Access/ Articles in Press from 2019 to 2020 and Figure A. 2 CHANGE TO Journal Citation Report for 2021
Appendix B: Preprints contains Figure B.1.
Appendix C: Retractions contains Figure C.1. Retractions over time from WOS
Table 1: Background Data
Table 2: Journal Document Types
Table 3: Impact of Covid-19 on Rankings
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is emeritus faculty librarian at Emory University. After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674
