By Ruth A Pagell*
(30 Sep 2024) Several rankings were released while I relived the past, writing RR 60. Changes include:
- Webometrics adding ROR (Research Organizations Registry) identity records to thousands of institutions and revising its university citation rankings
- SIR (Scimago Institutions Rankings) adding three new metrics to its Societal Factors
- US News Global modifying its methodology to include specialized institutions
- RUR (Round University Rankings) changing its bibliographic data sources from Clarivate to the open-access product LENS.
- Nature Index changing its name to Nature Index Research Leaders
- THE Impact Rankings announced it will be using AI in evaluating SDG evidence for 2025
I approached this update with a few questions:
- Given the negative articles about rankings, did the current year’s rankings include fewer universities than the previous year?
- With the reorganization of the French higher education system in 2019 how were the rankers managing the citation bibliometrics? Any ranking with citation metrics uses a multiple-year period that overlaps the old and new universities.
- What data are rankers providing for Russian institutions?
- Are Hong Kong and Macao included as individual locations or incorporated into mainland China?
WEBOMETRICS – Ranking Web of Universities is a product of CSIC, the Spanish National Research Council. The first release of the ranking was in 2004. It had over 23,000 post-secondary institutions in 2015 when it was introduced in Ruth’s Rankings. Today Webometrics has a list of about 12,000 top universities. It also has a full dataset of over 32,000 institutions. In 2024 it adapted using ROR identifiers for university names. Users are encouraged to contact ROR when a record is unavailable. Webometrics includes a separate ranking based on citations that I hadn’t included. It used Google Scholar. This edition has three indicators, Visibility, Transparency, and Excellence. For the World Ranking, Webometrics uses citation data from SIR. Transparent Ranking: Top Universities by Citations in Top Google Scholar Profiles is a work in progress. The current list includes about 6,000 universities that have at least 1000 citations.
Website: https://www.webometrics.info/en/current_edition
TOP 12000: https://www.webometrics.info/en/world?page=120
Transparent Ranking: https://www.webometrics.info/en/transparent
Methodology: https://www.webometrics.info/en/Methodology
Russian Federation: 997 universities are ranked, the largest number of any of the other rankers. Only rankings are provided.
France: The ROR names include detailed relationships, as shown in the record for Sorbonne
Ruth’s Rankings 9 (31 Mar 2015). https://librarylearningspace.com/ruths-rankings-9-expanding-measurement-science-citations-web-visibility-tweets/
Contact: Isidro Francisco Aguillo Caño <[email protected]>
SIR – Scimago Institutions Rankings is a Spanish Scimago Research Group product. The 2024 release has 4762 universities and in 2023 it had 4533. Rankings are available from 2009. It has a separate module for journal and county rankings It also experiments with infographics for those who do not like numbers. SIR has three categories of indicators, Research Factor, Innovation Factor, and Societal Factor. SIR is worth checking for mom top-tier universities. SIR manually disambiguates institutions’ names. It uses third-party data for all of its indicators. SCOPUS is the basis for inclusion and Unpaywall is the source for Open Access data.
In 2024 SIR added three size-dependent societal metrics: SDG-related papers, Impact on Public Policy from OVERTON, and Female Scientific Talent Pool. These metrics contribute 11 percent to overall rankings. The definitions include whether the indicator is size-independent or dependent. I asked how they determined gender. “… we employed a method based on given names, rooted in “cultural consensus theory” (Van Buskirk et. al.). SIR uses the freely available Python package. “This tool utilizes data from 150 countries and 36 publicly accessible sources, providing diverse and extensive coverage suitable for identifying genders, especially for less conventional scientific authors.“ SCImago team. (17 July 2024, email).
Website for academic institutions 2024: https://www.scimagoir.com/rankings.php?sector=Higher%20educ.
Methodology: https://www.scimagoir.com/methodology.php
Russian Federation: 164 institutions with data
France – data from 2019 with no link to previous status
Ruth’s Rankings 8 (27 Feb 2015) https://librarylearningspace.com/ruths-rankings-8-something-everyone and RR 9 (31 Mar 2015) https://librarylearningspace.com/ruths-rankings-9-expanding-measurement-science-citations-web-visibility-tweets
Contact: [email protected]
US NEWS Global – 2024-2025 Best Global Universities Rankings has 2459 universities in 2024 up from 2000 in 2023. Only the current year’s data are provided. The only ranking is by universities’ composite scores. Indicators, which include bibliometrics and reputation, come from Clarivate. US News Global modified the current methodology to include all institutions of higher education that are in the top 250 of the global research reputation survey. This modification allowed post-graduate institutions such as the University of California San Francisco and Rockefeller University to be part of the database. Since it serves a student audience, the following statement is included:” … a school’s Best Global Universities Rankings should be one consideration – not the lone determinant – in where a student applies. The rankings assess academic research and reputation, but personal considerations, including location, campus culture, strength of particular programs, and cost, are also important considerations”. (Morse, R. & Willington, S).
France: No link to previous status
Russia: 27 ranked with metrics and 42 overall.
Website: https://www.usnews.com/education/best-global-universities/rankings
Methodology: https://www.usnews.com/education/best-global-universities/articles/methodology
RR 11 (20 May 2015). https://librarylearningspace.com/ruths-rankings-11-u-s-news-world-report-goes-global/
Contact: [email protected]
NATURE INDEX RESARCH LEADERS is the new name for the Annual Tables. Rankings are available from 2016. The Annual Nature Index Research Leaders Academic Index with 2023 data, includes 500 universities. The Current Index, covering May 1, 2023, to April 30, 2024, consists of 4995 universities ranked from one to 3453. The complete dataset includes over 17,000 institutions. The current index is more useful than a list of just the top 500. As emphasized by its new name, it acknowledges its limited target audience. Originally only covering the natural sciences, health sciences publications were added in 2023. The list of the 145 journals is available: https://www.nature.com/nature-index/faq#journals. The location for universities from Hong Kong and Macao is China.
Methodology: https://www.nature.com/nature-index/brief-guide
Russia: 84 with data
Annual Index of Leading Academic Institutions: https://www.nature.com/nature-index/research-leaders/2024/institution/academic/all/global
Current Index: https://www.nature.com/nature-index/institution-outputs/generate/all/global/academic
RR 7: (Feb 2015) https://librarylearningspace.com/ruths-rankings-7-asian-institutions-grow-nature
RR 56 Part 2: (27 Sep 2023) – https://librarylearningspace.com/rr-56-part-2-new-metrics-for-old-ranking-covering-qs-nature-and-cwts-leiden/
RUR – Round University Rankings, previously based in Russia, it is now in Georgia. There are fewer universities in 2024 than in 2023, dropping to 1169 from 1217. RUR uses ratings as well as rankings. Data from The Lens have replaced bibliometric data from Clarivate. There have been minor changes in indicators in the 2024 version. Nine of the 2023 top 10 world universities, using Clarivate data, and eight of the 2024 top 10 Asian universities, using Lens are the same. The 2024 version has minor indicator changes, with links to the earlier versions. I will examine RUR and LENS in more depth when CWTS Leiden releases its OPEN version using OpenAlex, which is now scheduled for an October release.
Website: https://roundranking.com/ranking/world-university-rankings.html#world-2024
Methodology: https://roundranking.com/methodology/methodology.html
Russia: 131 Russian universities with detailed metrics
The LENS and Cambria: http://cambia.org/the-lens/
RR: 47 Part 2 (12 Aug, 2021): https://librarylearningspace.com/ruths-rankings-47-part-2-the-quest-continues-for-new-ranking-metrics-for-universities/ ; Select Appendix C
Contact: [email protected]
CWTS Leiden Ranking (Traditional) methodology has not been amended for 2024. It posted its traditional ranking on July 3, 2024. There are 1506 universities in 2024, 95 more than in 2023. The first ranking was in 2013 with 500 universities, with seven of the overall top 10 from the US and one each from Canada, Japan, and the UK. In 2024, eight of the top 10 universities in total publications are from China, and one from the US and Canada. Using the ranking metric based on the number of citations in the top ten percent in their fields, five were the same as Webometrics Transparent rankings using Google. When the updated Open version is released in October, I will compare OpenAlex and Lens. The location for universities from Hong Kong and Macao is China.
Website:: https://www.leidenranking.com/ranking/2024/list
Methodology – Indicators: : https://www.leidenranking.com/information/indicators
Russia: 15 with data
France: 32
Methodology under Information. https://www.leidenranking.com/information/general
CWTS Gender has been an impact category since 2019. It uses the following sites to determine gender which differ from those used by SIR:
https://genderize.io/ – free and simple
https://pypi.org/project/gender-guesser/0.4.0/
RR 8 (27 Feb 2015). https://librarylearningspace.com/ruths-rankings-8-something-everyone/
RR 59 (24 Mar 2024). https://librarylearningspace.com/ruths-ranking-59-cwts-leiden-introduces-openalex-to-its-rankings-is-this-the-start-of-a-new-era-of-rankings-bibliometrics
Contact: [email protected]
Mark Neijssel – email 3 July – next open Sept 10, 2024
THE Impact 2024’s methodology remained the same as in 2023. While many top universities do not participate in the Impact rankings, the number of universities that received a ranking increased from 1591 in 2023 to 1963 in 2024. All universities are eligible to participate. To receive an overall score, universities must submit SDG 17 and three other SDGs. Adding universities that do not have an overall rank brings the number of participating universities to 2152.
The top five countries, based on the number of participating universities, are India, Turkey, Pakistan, the Russian Federation, and Thailand. Russian institutions are ranked and the scores are grayed out but are legible. There is no additional information about the universities. Looking at all the top 10 universities in Table 1, only MIT and Viet Nam’s Duy Tan University are in the Impact rankings. A highlight of the 2024 ranking is the identification of Hot Spots. See the map that identifies areas that are strong in individual SDGS. There will be two major changes for the 2025 rankings. One is the availability of the ranking in multiple languages including Arabic, Russian, and Thai. The second is using AI to evaluate the evidence. If AI rejects the evidence, it will go on to a team of humans.
Figure 1: Extract from Hot Spot Map
Website: https://www.timeshighereducation.com/impactrankings
Methodology: https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2024-methodology
Times Higher Education Impact Rankings Digital Edition
RR 41 (25 Jul, 2019) https://librarylearningspace.com/ruths-rankings-41-thes-university-impact-rankings-sustainable-development-goals-impactful-universities-world
RR 56 Part 1 (7/27/2023) https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2024-methodology
QS World University Rankings 2025 is included here for comparison. It increased the number of universities to 1503 from 1498. The 2025 edition will be covered in more detail in RR 61 Part 2, a review of the big three, ARWU, QS and THE.
France: Extensive information on the components of the new universities
Russia:47 with data
RR 56 2 (27 Sep 2023) https://librarylearningspace.com/rr-56-part-2-new-metrics-for-old-ranking-covering-qs-nature-and-cwts-leiden/
Contact: [email protected]
There are acknowledgments of the limitations of the rankings, as noted by THE and US News Global (Baty; Morse & Willington). IREG ( International Ranking Expert Group) has issued guidelines on how to use rankings wisely. CWTS has issued an animation on responsible use (CWTS).
Table 1 below lists the top 10 universities and top five countries for the rankings released in 2024. Check out the initial Ruth’s Ranking articles for more details about these rankings and the list of articles and updates. RR 61 Part 2 will be available after the October release of THE’s World rankings, allowing for comparisons among the big three featured in RR 60, QS, ARWU, and THE. The November follow-up will be on the use of OpenAlex and Lens.
RESOURCES
Baty, P. (3 Jan 2024). “One ranking to rule them all? Absolutely not”. https://www.timeshighereducation.com/world-university-rankings-2010-11-methodology
CWTS Leiden Rankings,” Responsible Use” accessed September 17, 2024. https://www.leidenranking.com/information/responsibleuse
Morse, R. & Willington S. (24 Jun 2024). How U.S. News calculated the 2024-2025 Best Global Universities Rankings. https://www.usnews.com/education/best-global-universities/articles/methodology
Siwiński, W. & Federkeil, G. (2023) . IREG Guidelines for Stakeholders of Academic Rankings.
Van Buskirk, I., Clauset, A., & Larremore, D. B. (2023). An Open-Source Cultural Consensus Approach to Name-Based Gender Classification. Proceedings of the International AAAI Conference on Web and Social Media, 17(1), 866-877. https://doi.org/10.1609/icwsm.v17i1.22195
Main Ranking Websites
Webometrics: https://www.webometrics.info/en
SIR: https://www.scimagoir.com/
US News Global : https://www.usnews.com/education/best-global-universities/ranking
RUR: https://roundranking.com/ranking/world-university-rankings.html#world-2024
Nature Research Leaders: https://www.nature.com/nature-index/research-leaders/2024/institution/all/all/global
CWTS Leiden Rankings: https://www.leidenranking.com
THE World: https://www.timeshighereducation.com/world-university-rankings; https://www.timeshighereducation.com/impactrankings
QS: https://www.topuniversities.com/world-university-rankings
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is an emeritus faculty librarian at Emory University. After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674
