By Ruth A Pagell*
(7/27/2023) 20 years ago Shanghai Rankings issued its first global ranking of five hundred universities, followed a year later by a joint venture from THE and QS, of 200 universities. (Praphamontripong). The number of rankings, countries, institutions, and the scope of the indicators has grown along with criticism of the ranking in recent years (Fischer). This year’s updates reflect changes that rankers have made in response to the criticisms. Ratings have been suggested as an alternative to rankings and have been introduced in Money Magazine U.S. rankings, which focus on acceptance and graduate rates relative to costs (Diep; Money). For a balanced analysis, see Usher’s Rankings Discourse: West, East and South. Usher states that “it comes down to a difference in how universities in different parts of the world think about the value of universities and what is worth measuring”. Asia remains focused on research output while the West is looking at different aspects of a university’s value, as in the Impact Rankings.
Four companies recently made newsworthy changes to their rankings. Ruth’s Rankings will present them in a series of three articles. Part 1 covers changes in THE’s 2023 Impact Rankings. Part 2 covers changes to QS’ 2024 World University Rankings and Nature Index 2023 Annual tables, and CWTS Leiden proposed changes to future rankings. THE’s new ranking, with a new set of indicators for Sub-Saharan Africa, will have an article of its own, looking at how other rankings have treated that region.
In examining these rankings, I checked to see if the changes in methodologies resulted in changes in the top rankings and if they are helping to capture a broader scope of universities’ purposes.
THE’S IMPACT RANKINGS, in its fifth iteration, was the first major ranking of universities in non-academic roles. It is based on the UN’s Sustainable Development Goals (SDGs). According to Duncan Ross, THE’s Chief Data Officer, SDG rankings are designed to encourage universities to do more in terms of sustainability, The target audience is not only universities, students, and their families but also governments. The 2023 methodology asked universities more challenging questions for SDG 3, 4, 11,16, and 17. Figure one shows growth of participants in rankings that have 2023 modifications. Between 2022 and 2023, only SDG 16 grew at a slower pace than the overall number of participants.
Due to changes in methodology and an increasing number of participants, Ross does not recommend year-on-year comparisons (Ross, May 2023).
INSTITUTIONS AND COUNTRIES
1,591 universities from 112 locations chose to participate. Any university teaching undergraduate or graduate students is eligible. Each university selects the SDGs it will cover. To receive an overall ranking it must provide data for at least four SDGS, including SDG 17. The three highest ranking SDGS plus the score for SDG 17 are calculated to receive an overall score. This year’s overall score includes the average for 2022 and 2023. Additional universities that do not meet the criteria for an overall score appear in individual SDGS. There is no list that identifies them. Including the overall score, universities from nine different countries are number one in the world for the individual SDGs. Universities from eight different countries have a number one university in East Asia. Australia leads all countries capturing eight top spots. Indonesia captures four top spots in East Asia. See Table 56.1 for the leading universities for all the SDGs.
An interesting THE article examines the participating countries in relation to their World Bank income categories. (Hamadeh; World Bank). THE’s article concludes that “middle-Income countries are most active in Impact Rankings” (Ellis) The headline goes on to say that these universities ”lead the globe when it comes to commitment to sustainability”. Since universities opt into the rankings, most of the world’s top 100 universities (THE World 2023) do not participate. 57 of the top 100 universities in the Overall rankings are from high-income countries, Australia, the United Kingdom, and Canada. They are not the top universities in those countries. 16 East Asian countries are in the top 100, eight each from high and middle-income countries. Universities that have not participated are also committed to SDGs. Check out universities’ websites such as Oxford or my affiliate, Emory University, which has an award-winning project WaterHub (Ricker).
CASE STUDY – UN SDG 16, Peace, Justice and strong institutions, including Academic Freedom
For each SDG, there are fixed quantitative and qualitative metrics for which evidence must be provided. Ranks change on a yearly basis based on subtle changes to metrics’ wording and universities joining or leaving the ranking. I selected SDG 16 since it includes Academic Freedom, a topic covered recently in RR 54 and its addendum. The topic is not covered anywhere by the UN. SDG 4, Quality education, focuses on pre-tertiary education. Recognizing the absence of the topic, THE added questions to SDG 16. Click here for the UN’s SDG 16. Below are the 2023 updated questions for SDG 16, highlighted in red (THE Impact Rankings Methodology(. The framework for each SDG is available for each year (SDG 16).
Example: Interpreting the rankings for Univeriti Sains Malaysia
The overall ranking screen displays the university’s scores for the top four SDGs. Universiti Sains Malaysia participated in each year’s rankings.
Click on a university’s name to see all its impact ranks. The score is not a predictor of the ranking.
Example 1: Universiti Sains Malaysia 2022 and 2023
It is first in SDG 16 and fourth overall for both years and first in 2022, and sixth in 2023 for SDG 17. In SDG 5, which did not have any changes to its methodology, it was seventh in 2022 and 48th in 2023 with a score of 70.8.
One other Asian star is the Universitas of Indonesia which is top 20 in SDGs 2,3,5,6, and 8. It has its own sustainability report, Green Metric, released annually since 2010 (RR 48.1)
RR 56. Table 2: Top five universities for SDG 16 for each ranking year, with their world rankings and overall Impact rankings
Universiti Sains Malaysia is the only top five in 2023 in SDG 16 that has been in each ranking. Three other universities were also in all five years’ rankings. It illustrates the volatility in the ranking for the SDG and the differential between the Impact rank with the universities’ world ranks.
CONCLUSION:
THE Impact rankings added another dimension to the rankings of universities and the roles they play. More universities and countries opt into the rankings every year. In order for this ranking to be meaningful, it is the responsibility of the user to select the correct SDG. If I had not read the fine print and selected SDG 4 Quality, I would not have known that academic freedom was not included.
THE has been working with UNHCR (United Nations High Commissioner for Refugees), adding metrics in 2022 for universities supporting refugees and support for refugees has been added to many of the SDG statements. What is not available is any data for the underlying metrics. Though definitions continue to be improved, it is important to note that the individual metrics are built around the United Nations’ definitions. THE has added metrics, as in the case of SDG 16.
My other ongoing concern is the assumption that these are the best universities in SDG impact in the world. These are the best universities that have chosen to participate. In addition, not captured in these rankings is the current focus on universities’ roles in the area of DEIs (Diversity, Equality, and Inclusion) and approach which we will see in Part 2.
After doing my own comparisons, I recommend listening to Ross’ warning about year-to-year comparisons, with ranks. Most universities will never be in the top 100 and different universities and countries have different priorities. The methodology gives everyone who participates an opportunity to receive international recognition and a platform to use for benchmarking with their peers.
RESOURCES
Ruth’s Rankings and THE Impact Rankings
RR 38 (11 Jan 2019), https://librarylearningspace.com/ruths-rankings-38-coming-attractions-un-sustainable-development-goals-times-higher-education-innovation-impact-rankings-demystified/
RR 41 (25 Jul 2019), https://librarylearningspace.com/ruths-rankings-41-thes-university-impact-rankings-sustainable-development-goals-impactful-universities-world/
RR Update (26 Apr 2020), https://librarylearningspace.com/releases-2020-impact-rankings-based-sustainable-development-goals-anything-changed-ruths-rankings-update/
THE’s 2021 Impact Rankings: Is the third time a charm? A Ruth’s Rankings News Update (29 Apr 2021), https://librarylearningspace.com/thes-2021-impact-rankings-is-the-third-time-the-charm-a-ruths-ranking-news-update/
RR Update (5 May 2022), https://librarylearningspace.com/ruths-rankings-news-update-april-2022-rankings-are-as-good-or-bad-as-the-people-using-them/
Other Ruth’s Rankings
Pagell, R.(25 Jan 2017) Ruth’s Rankings 23: Are higher education rankings flawed or misunderstood? https://librarylearningspace.com/ruths-rankings-23-global-higher-education-rankings-flawed-misunderstood-personal-critique/
RR 48 Part1( 20 Nov 2021). How green is my university? New metrics and new leaders, https://librarylearningspace.com/ruths-rankings-48-part-1-how-green-is-my-university-ranking-green-universities-new-metrics-and-new-leaders/
RR 48 Part 2: Country and company environmental rankings add new sets of indictors to our vocabulary, https://librarylearningspace.com/ruths-rankings-48-part-2-country-and-company-environmental-rankings-add-new-sets-of-indicators-to-our-vocabulary/
Cited resources
Diep, F. (22 June 2023). This year, year one major college ranker is turning from rankings to ratings. Chronicle of Higher Education, https://www.chronicle.com/article/this-year-one-major-college-ranker-is-turning-from-rankings-to-ratings#:~:text=Now%2C%20one%20ranker%2C%20Money%20magazine,instead%20of%20a%20numbered%20rank.
Best Colleges in America 2023 ( 22 Jun 23) Money Magazine, https://money.com/best-colleges/
Ellis, R. (1 Jun 2023). Middle-income countries are most active in Impact Rankings https://www.timeshighereducation.com/world-university-rankings/middle-income-countries-are-most-active-impact-rankings
Fischer, K. (12 Jul 2023). What’s the future for international rankings? Latitudes, Chronicle of Higher Education, https://www.chronicle.com/newsletter/latitudes/2023-07-12?utm_source=Iterable&utm_medium=email&utm_campaign=campaign_7236447_nl_Latitudes_date_20230712&cid=lt&source=&sourceid=
Hamadeh, N., Van Pompaey, C, & Metreau, E. (30 Jun 2023). World Bank Group country classifications by income Level for FY24 (July1, 2023 – JUNE 20, 2024). Data Blog https://blogs.worldbank.org/opendata/new-world-bank-group-country-classifications-income-level-fy24
Praphamontripong, P. and Levy, D. Analysis of 2004 THE/QS Rankings: World University Rankings, 2004 Modified from the Times Higher Education Supplement 2004 World University Rankings, https://www.albany.edu/dept/eaps/prophe/data/International_Data/WorldUniversityRanking2004_ModifiedFromTHES.pdf
Ricker, M. (11 Jul 2023). How Emory works : the WaterHub, Emory report, https://news.emory.edu/stories/2023/07/er_the_waterhub_11-07-2023/story.html?utm_source=Emory_Report&utm_medium=email&utm_campaign=Emory_Report_EB_071323 https://news.emory.edu/stories/2023/07/er_the_waterhub_11-07-2023/story.html?utm_source=Emory_Report&utm_medium=email&utm_campaign=Emory_Report_EB_071323
Ross, D. (11 May 2023). Changes for stability in THE’s 2023 Impact Rankings, https://www.timeshighereducation.com/world-university-rankings/changes-stability-thes-2023-impact-rankings
Ross, D. ( 1 Jun 2023). Impact rankings 2023: a demonstratable commitment to the goals. https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2023-demonstrable-commitment-goals
SDG16 methodology, from 2019 to 2023:
2019 https://www.timeshighereducation.com/university-impact-rankings-2019-methodology
THE Impact ranking methodology (2023) V 1.2, https://the-ranking.s3.eu-west-1.amazonaws.com/IMPACT/IMPACT2023/THE.ImpactRankings.METHODOLOGY.2023_v1.2.pdf
Usher, A. (3 May 2023). Rankings Discourse: West, East and South. Higher Education Strategy Associates. https://higheredstrategy.com/rankings-discourses-west-east-and-south/
World Bank Country and lending groups (2023). Data , https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups
World University Rankings 2023 methodology (Oct 2022), pg. 6 https://www.timeshighereducation.com/sites/default/files/breaking_news_files/the_2023_world_university_rankings_methodology.pdf
NOTE: It might be necessary to register to read THE articles: https://www.timeshighereducation.com/world-university-rankings/impact-rankings-2023-demonstrable-commitment-goals
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is emeritus faculty librarian at Emory University. After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674
