By Ruth A. Pagell*
- What new metric has been added to THE Impact Rankings?
- How do Japan University Rankings differ from Japan in World Rankings?
- Is it better to rank 112 or 47 in QS Subject rankings (See Example 1)?
(5 May 2022) This is a quick take on the usual spring updates, including THE’s Japan Rankings and QS Subject rankings. In looking at the rankings this time around, I was more aware of the metrics and the need to understand their methodologies and implications.
I am waiting for the release of QS’ new impact rankings scheduled for November to write a new Ruth’s Ranking on SDGs.
There is an accounting method called LIFO, Last In First Out, so I am starting with THE Impact. The number of universities that have chosen to participate in four SDGs, including SDG 17, to qualify for an overall rank, is 1,406 from 106 countries.
Changes were made to SDGs 4, 10, and 11 to include weighting for support for refugees (Ross). That means that the changes were made in the past two months. The methodology is described in a 190 page document that does discuss verification. I have to admit that I am skeptical about how much verification could have taken place with a new indicator added at the last minute. THE has admitted that there are biases in the rankings and they have brought together an advisory group to look at modifications. Table 1 has the top universities in the world and Asia for 2022.
The United States is underrepresented with only 42 universities. US universities participate in STARS, from AASHE, the Association for the Advancement of Sustainability in Higher Education. Universities receive ratings. STARS has over 1,000 members, of whom 680 have been rated and over 200 current rankings. As much as I hate using adjectives, I don’t want to hand count so over a vast majority are from the United States https://stars.aashe.org/, with participants list.
THE Japan University Rankings
Unlike other THE rankings, which reach out to all rankings’ users, the Japanese ranking is designed for students. The 2022 Japan Rankings includes 273 universities, five fewer than last year. The ranking uses four pillars. Tohoku University ranks number one and is the only university to be in the top 10 in all four. The pillars, their weighting, the number one with its overall Japan rank are listed below:
- Resources (34%): Support for the student’s educational experience: The University of Tokyo (2)
- Engagement (30%): Opinions from the High School Advisors survey and scores from THE’s Student survey: Akita International University (17)
- Outcomes (16%): Academic and employers survey: Kyoto University (5)
- Environment (20%): Diversity based on international students, staff, and courses in Japanese: Ritsumeikan Asia-Pacific University
To do: See Table 2: Japan University ranking and notice the differences among the rankings in each category and with their rankings in THE’s world rankings.
Japan University ranking: https://www.timeshighereducation.com/rankings/japan-university/2022#!/page/
Methodology: https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2022-methodology
Ruth’s Rankings Japan articles:
News Flash (April 2017): https://librarylearningspace.com/ruths-rankings-news-flash-times-higher-education-japan-new-ranking-universities-top/
Ruth’s Rankings 28 (August 2018): Japanese Universities: https://librarylearningspace.com/ruths-rankings-28-japanese-universities-sun-setting-japanese-higher-education/
QS WORLD UNIVERSITY RANKINGS BY SUBJECT 2022
The 2022 ranks have the same number of subjects, 51, three more countries, now 88, with an increase of 103 universities. The QS Subject Ranking is separate from their world rankings. It uses a different set of metrics, with different weightings, by subject. 243 universities not included in world rankings are in the subject rankings.
A University World News article (Reporters) has a detailed breakdown of the results from the 2022 rankings. Despite the positives, I cannot get as excited about the release as University World News did. Out of the 51 subjects, only six have a change in number one from 2021. In fact, looking at the results for number one in each subject just seemed to exemplify the problems with the methodology. A new metric has been added to the broad category rankings, International Research Network, to go along with the existing Academic and Employer Reputations survey results and the bibliometric Research citations per paper and H-Index from Scopus for the sub-topics. The IRN “reflects the ability of institutions to diversify the geography of their international research network by establishing sustainable research partnerships with other higher education institutions”. (https://support.qs.com/hc/en-gb/articles/360021865579-International-Research-Network-IRN-Index )
Even with the addition of the new metric, there are only six new number one’s in the world in 2022 and six different number ones in East Asia, with National University of Singapore dominating the rankings. In the top 15 categories with 500 or more universities, including the overall categories, Harvard and MIT have 14 number one top spots, and Oxford has one. In the bottom 15 with fewer than 200 universities in more specialized categories, there are 14 different number ones, with Oxford having two.
If you are familiar with your institution’s strengths, choose your subject rankings. QS may not be your best choice. For example, QS combines Physics and Astronomy. The University of Hawaii ranks 251-300 in Physics and Astronomy and 15th in Atmospheric Science in Shanghai rankings Global Ranking of Academic Subjects, GRAS. If you are not sure if your university is ranked in any of the QS subjects, there is no master list. GRAS lists all universities included in ARUW or GRAS and for the GRAS universities, all their subject categories. A limitation with GRAS is that it does not include Arts & Humanities.
To Do: See Table 4 for all the number ones in the world, Asia, and Oceania. See Example 1 on analyzing subject results. The example demonstrates the misrepresentation of Mahidol University, Thailand, receiving a high rank of 47 in Performing Arts when its rank of 122 in Medicine was much better.
University World News reporter (16 April 2022). “ US universities lead in 28 of 51 subject rankings by QS”. https://www.universityworldnews.com/post.php?story=20220415152142330
NOTE: Many articles in University World News are written by QS’ head of research, Ben Sowter
Subject rankings are a way for universities that will never be number one to display their accomplishments. The different subject rankings may use different terminology for categories and of course different metrics. They also have different interfaces that enhance a user’s experience.
CONCLUSION: I listened to a webinar sponsored by ISKOUK, the International Society for Knowledge Organization UK. Ludo Waltman from CWTS Leiden talked about the openness of bibliographic metadata in which he pointed out the difference between open and free. This can easily be applied to rankings as well, where the rankings are free, and the data are proprietary. Leiden is my favorite, especially for readers who do not have access to a data analytics source such as Clarivate’s Incites or Elsevier’s SciVal. https://www.iskouk.org/event-4728034
While a casual user may be interested in ranks, people using these sources for decision making need to understand the methodology and see if there is any supporting data.
O, Craig (14 April 2022). QS University Rankings by subject: Methodology. https://topuniversities.com/subject-rankings/methodology
Ross, D. (30 March 2022). “Why we are rewarding support for refugees in our Impact Rankings https://www.timeshighereducation.com/world-university-rankings/why-we-are-rewarding-support-refugees-our-impact-rankings
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is emeritus faculty librarian at Emory University. After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674
