By Ruth A. Pagell*
(12 Oct 2021) I had intended to write a short update on recent ARWU and THE releases. In looking for something interesting to write about, since these rankings are stable, I began to drill down into individual metrics, a mistake for someone who likes to ask herself questions such as how can a university with a score of 100 in citations have a score of 8.6 for research? While trying to slog through the underlying data and figure out what that means for the viability of the ranking, my computers were compromised twice.
This article covers ARWU’s 2021 release, THE’s 2022 release, both in August of 2021, and a comparison of the top three, adding QS, updated early this summer. I explored the one size-independent metric in ARWU, Per Capita Performance (PCP). For THE, I planned on focusing on changes in international scores, not only because of Covid but also political issues. THE focused on changes from Covid papers. That led me to the larger issue of how citations are measured in the rankings. This update compares the top universities in the 2021 releases of the big three, THE, ARWU and QS in Table 4. Finally, there are links to information about new releases not covered here.
ACADEMIC RANKING of WORLD UNIVERSITIES 2021 (ARWU)
ARWU’s press release for its 2021 release is modest, with an overview of the minor changes that took place during the year (ShanghaiRankings). There are 200 U.S. universities, including some liberal arts colleges. ARWU lists 180 universities under China including 157 from the mainland, 14 from Taiwan, seven from Hong Kong, and two from Macau. The world’s top 11 universities are the same in this year’s release as in the first ranking in 2003.
One change to ARWU is a modernernised interface.
One reason for the stability in ARWU’s rankings is the stability of its metrics. Instead of using surveys as a measure of faculty quality, 30% of the weighting is based on number of awards. Another 60% of the rankings come from size-dependent output measures. The world results are as expected. Eight of the world’s top ten are from the U.S. and the other two from the U.K. Seven of the top 10 remained the same as 2020. There is one different U.S. institution and one from Italy and Sweden. Seven were also top 10 using the Per Capita Performance (PCP) metric. Table 1 shows the impact of using PCP for the world’s top ten and for selected Asia-Pac universities. The differences in rankings are more obvious in the Asia/Pac rankings. For example, Korea’s POHANG is in the 400 band in the overall rank and the 101 in PCP rankings and Kyoto University is 37 overall and 181 using PCP.
A second change is the repackaging of subject rankings under the acronym GRAS, Global Rankings of Academic Subjects, with the same 54 subjects as last year. GRAS uses a different set of indicators from ARWU. See the methodology for indicators, weights, number of institutions by subject, and a mapping to Web of Science subjects. Universities outside the top level or not in the top 1,000, should check subjects. As examples, the University of Hawai’i Manoa ranked 301-400 overall, is 15th in Atmospheric Science. Unranked Singapore Management University is ranked in eight categories with a world ranking of 30 in Finance. It is only within individual subject rankings that searching for a non-top 1,000 university is possible.
ShanghaiRankings has had continuous and consistent coverage since 2003. Indicators have not changed. Thirty percent of the rankings are based on a university’s staff or alumni winning a limited number of awards. WOS’ s Arts & Humanities Index is not covered.
TIMES HIGHER EDUCATION (THE) WORLD UNIVERSITY RANKINGS 2022
The 2022 rankings list 2,112 universities from 99 countries, an increase of almost 40% over 2021. 1,661 are ranked. 451 have “Reporter” status. Reporters have submitted their data but did not meet the ranking requirements. Many are not well-known institutions from developing countries. A few are world class professional schools such as London Business school and the US’ medical research Rockefeller University. Nine of THE’s top ten are the same as the first release of the THE/QS rankings in 2004.
In preparation for the release, THE posted an article on internationalisation. Lau writes about international student movements within Asia, taking data from the ICEF Monitor. This prompted me to look at international metrics. Given Covid and political issues in 2020-2021 I asked: Were there noticeable changes in rankings for International students and Outlook? In January, THE released the “Most International Universities in the World”, using international students, international outlook, and survey data as the basis for the rankings. See Table 2 for international rankings. Five of the 10 universities with the highest percent of international students are from Europe, four from the Middle East and one from Macau. Because international outlook does not sort correctly, I selected top countries and their top five, expecting to check again in 2022. Macau University of Science and Technology is first in both categories in 2022 and 2021. Changes are not reflected in the data for the top universities.
CITATIONS
I had not planned on also drilling down into citations, until THE promoted China’s gain with increased articles on Covid- 19. (Bothwell). Finding one reliable metric for citations is challenging. “How many citations” is size dependent. Other options are using citations per paper, per faculty, the average number of times a university’s work has been cited, percent of papers in a certain percentile, such as top 10%, or some field weighted average.
THE World uses average number of citations per paper. 11 universities receive a score of 100. None are from the U.S. or China. Six U.S. universities are in the top 25 and none from China. The highest ranked Chinese University is Southern University of Science and Technology, ranked 33 in the world. Nine of THE’s universities that scored 100 are not even ranked by QS! Since citations are weighted at 30%, universities that do not score well in other categories find themselves ranked between 300-500 out of over 1,600 universities.
Other rankings use different metrics. QS uses citations per faculty. Nine universities score 100, with six from the US, and the Indian Institute of Science, and two from South Korea. ARWU uses the number of highly cited researchers rather than a citation metric based on paper. Seven are from the US, two from the UK, and Tsinghua is the highest from Asia/Pac in 5th place.
See Appendix A and the three tables associated with the Appendix to see rankings based on THE’s subject “Medicine and Dentistry” and metrics for special topics from Elsevier’s SciVal and Clarivate’s InCItes.
COMPARING TOP UNIVERSITIES IN THE BIG THREE
These are the rankings that get the most coverage and have been around in the global arena the longest. Since the inception of ARWU in 2003 and THE/QS in 2004, more universities from more countries have been ranked, subjects have been added, and except for the breakup between QS and THE, metrics have remained stable as have the names at the top. Chinese universities have moved up the ranks, U.S. institutions at the top have stayed the same since they have nowhere to go. See Table 4 for world comparisons with the first rankings and for current Asia/Pac ranks.
Since these rankings are so popular, target audiences tend to take them for granted. Over the seven years I have been writing this column, I have tried to play the role of the factual presenter of the news, answering questions such as “Who is number one in Asia/Pac”? Now I am asking myself questions such as:
- How can a university score 100 on Citations and 8.6 on Research?
- Does having a high score on a citation metric based on papers or authors really tell the citation story? Is it better to have 42,000 citations for fewer than 200 papers produced in a five- year period or one million citations for 60,000 papers?
- Is having a large percentage of international students better than having a balance of international and local students?
UPDATES: THE has announced changes to Impact Rankings Methodology, and released six of its 11 subjects by October 11, 2022 and its Wall Street Journal/THE US College rankings. WST/THE ranks liberal arts colleges and Historically Black Colleges along with the usual suspects. Registration is required to read articles. QS rolled out its Graduate Employability rankings.
RESOURCES
Bothwell, E. (2 Sep 2022). World university rankings, 2020: Results announced. https://www.timeshighereducation.com/news/world-university-rankings-2022-results-announced
ICEF Monitor (free daily and weekly updates). Market intelligence for international student recruitment. https://monitor.icef.com/about-icef-monitor/
Lau, J. (25 Aug 2021). Asia internationalises in its own backyard. Affordable quality education and job opportunities are encouraging more Asian students to travel closer to home,, https://www.timeshighereducation.com/world-university-rankings/asia-internationalises-its-own-backyard
Rankings links:
GRAS, Global Rankings of Academic Subjects (2 May 2021). https://www.shanghairanking.com/rankings/gras/2021
ShanghaiRanking’s GRAS methodology 2021 https://www.shanghairanking.com/methodology/gras/2021
ShanghaiRankings ((15 August 2021). ShanghaiRankings of academic universities 2021. Press Release https://www.shanghairanking.com/news/arwu/2021
THE (28 Jan 2021). Most international universities in the world, https://www.timeshighereducation.com/student/best-universities/most-international-universities-world
THE (26 Aug 2021). Methodology: https://www.timeshighereducation.com/world-university-rankings/world-university-rankings-2022-methodology
THE (1 Sep 2021). THE World Rankings 2022,
https://www.timeshighereducation.com/world-university-rankings/2022/world-ranking#
QS Rankings 2022, https://www.topuniversities.com/qs-world-university-rankings
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is emeritus faculty librarian at Emory University. After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674
