Ruth’s Rankings 41: THE’s University Impact Rankings and Sustainable Development Goals: Are these the most impactful universities in the world?

By Ruth A. Pagell*

  • What university is tops in the world in the Impact Rankings?
  • Which Asia-Pac nation has the most universities that participated in the survey?
  • Approximately how many universities are involved in worldwide SDG initiatives? A: 460; B: 1200; C: 7000?

(25 Jul 2019) Phil Baty, THE’s chief knowledge officer said, “The rankings offer an alternative view of university excellence” and “develop an unprecedented picture of the extraordinary impact that universities have across a huge range of activities”. Patrick Paul Walsh, professor at University College Dublin and senior advisor to the UN Sustainable Development Solutions Network,  highlighted its combining “hard” bibliometric data with qualitative data on social, environments and partnership impact (Bothwell).

These statements are true.  Expanding university rankings based on their contributions to society is important. This article presents the Rankings’ results and methodologies and includes alternative sustainable development university initiatives. I ended up questioning whether the rankings deliver what they promise.

REVIEWING SDGS

I wrote Ruth’s Rankings 38 in preparation for analyzing these rankings. These are the 11 goals included in THE’s first ranking:

  • Goal 3: Good health and well-being – Ensure healthy lives and promote well-being for all at all ages.
  • Goal 4: Quality education – Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.
  • Goal 5: Gender equity – Achieve gender equality and empower all women and girls.
  • Goal 8: Decent work and economic growth – Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all.
  • Goal 9: Industry, innovation and infrastructure – Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation.
  • Goal 10: Reduce inequalities – Reduce inequality within and among countries.
  • Goal 11: Sustainable cities and communities – Make cities and human settlements inclusive, safe, resilient and sustainable.
  • Goal 12: Responsible consumption and production – Ensure sustainable consumption and production patterns.
  • Goal 13: Climate action – Take urgent action to combat climate change and its impacts.
  • Goal 16: Peace, justice and strong institutions – Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels.
  • Goal 17: Partnerships for the goals – Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development.

Universities teaching undergraduates that are accredited by a recognized body were eligible for inclusion. Over 500 universities from 80 countries participated in the survey and 467 received a composite score. See Table 1 (in pdf) for tops in the World and Asia-Pac for each SDG.

Simplified Methodology:

  • Universities answering SDG 17 and at least three other SDGs receive a composite score.
  • The composite score is weighted with SDG 17 at 22% and the top three SDGs at 26%.
  • Each SDG has a different metric and number of respondents.
  • Each SDG includes bibliometrics worth 27% of the composite score.
  • Elsevier supplies bibliometric data. Each SDG has a search query relevant to the SDG topic. The queries are not available.
  • Universities not responding to SDG 17 are ranked under individual SDGs.
  • There is no master list of all responding universities.
  • Only a score and ranks on four SDGs are available as displayed below:

The bibliometrics may include the following, as they apply to the individual SDG topic:

  • Number of publications (included in all SDGs at 7%)
  • Proportion of papers that are viewed or downloaded
  • Proportion of research papers in the top 10% of journals as defined by CiteScore
  • Field weighted citation index of papers produced by the university

See Appendix A (in pdf) for the list of bibliometrics by SDG.

IMPACT RANKINGS

No top 20 universities in THE’s World rankings participated in the survey.  See Table 2 (in pdf) for the top 20 universities in the Impact Rankings with their 2019 World Rankings. 11 of the top 20 are from Europe. There is one from Asia and none from the U.S. Table 3 (in pdf) lists the Impact Rankings’ top 20 for Asia-Pac, top 100 for all Asia, and top 200 for the U.S. with their World Rankings.  Highlights from Table 3 include The University of Hong Kong (HKU) coming in second in Asia-Pac and tops in all of Asia.  Asia-Pac has 23 in the top 100, all Asia 18 and the United States eight. Three U.S. universities in the top 100 for impact are in the World Top 100. 41 U.S. universities are in the top 100 in World Rankings.

Author’s note: A question this ranking does not answer is whether U.S. universities are less impactful or whether most chose not to participate.

Below are two examples of using the rankings.

EXAMPLE 1 provides information on a top 25 U.S. university which chose to participate

Emory University, ranked 21st in U.S. News and 84th in THE’s  World Rankings is taking impact seriously by creating a new website to promote its impact in areas such as community, caring and healing, and economic impact.  I could not find Emory in the composite list. I checked each SDG separately. Emory provided data for six SDGs:  3,4,8, 8,9,11 and 13 and is top 20 for 3 and 9.  Because it did not respond to 17, it has no composite score.

EXAMPLE 2: SDG 9

I selected SDG 9, Industry, Innovation and Infrastructure since I could compare outcomes with existing indicators. SDG 9’s bibliometric methodology includes Publications and Patents citing research.  Reuters Top 100: The World’s Most Innovative Universities and its Asia Pacific Top 75 Most Innovative Universities also use publications and patents in its annual rankings, produced in conjunction with Clarivate Analytics. RR 36 discussed the Reuters rankings. The third indicator, research income from industry is in THE’s World Rankings. The fourth metric is university spinoffs.  See Table 4 (in pdf) for SDG 9 rankings and analysis. University of North Carolina Chapel Hill is in the top 10 on both lists and Tokyo, KAIST and Sungkyunkwan are top 10 in Asia on both lists.

ACADEMIC SDG INITIATIVES

Over 1,000 universities are working in conjunction with the UN on two different initiatives. One is UNAI (United Nations Academic Impact) with over 1,300 member institutions in over 130 countries. Established in 2010, UNAI is a “network of students, academics, scientists, researchers, think tanks, institutions of higher education, continuing education and educational associations”

IAU (International Association of Universities), a membership organization with representatives from 120 countries, is under the auspices of UNESCO. In July 2019 it had 597 institutional members. Sustainable development, Leadership, Internationalization and Technology are its priorities. It published the “Global Survey on the Role of Higher Education in Fostering Sustainable Development “in 2016, surveying how much universities knew about the SDGs and how they were being integrated into the universities (van’t Land & Herzog).

Combined networks of over 7,000 universities, including groups affiliated with the U.K.-Ireland based Alliance for Sustainability Leadership in Education (EAUC) , U.S. (Second Nature),  and  UN Environment have made a commitment to address climate emergency.  O’Malley’s articles in World University News list many of the organizations involved in the initiatives and the importance of the roles played by higher education. The annual Green Gown Awards recognizes universities for best Sustainability practices.  Click here for information about the 2019 awards.  McGill University won the award for 2019.  McGill highlighted SDGs 4,11,12 and 13 as its strengths. Only SDG 12 appears in its THE composite ranking of 101-200. RMIT (THE 82) and Chiba (THE 101-200) are runner ups.

CAUL, the Council of Australian Librarians (Rolf) and ACU, the Association of Commonwealth Universities (Makoni) have also issued statements regarding involvement with SDGs.

Concurrent with THE releasing its rankings, Clarivate Analytics posted “Navigating the Structure of Research on Sustainable Development Goals”, examining the geographic output of articles about SDG goal topics. (Masafumi, N. et al).  Developing, lower income African countries had the highest percent of research on SDG goals.

COUNTRIES AND UNIVERSITIES

SDGs were developed for countries, not institutions. Each SDG has a UN list of targets, both quantitative and qualitative. The UN Sustainable Development Solutions Network publishes a spreadsheet including over 400 indicators for 166 countries with limited data for 30 more locations. THE’s Data Collection Portal lists the qualitative and quantitative indicators that they have mapped to the UN targets but no data are available. See Appendix B (in pdf) and Table 5 (in pdf) for data and analysis of countries represented in the different academic SDG initiatives.

CONCLUSION

This is the fifth anniversary of Ruth’s Rankings. We counted publications and citations. New metrics calculate universities’ proportion of recycled waste (SDG 12) and the proportion of first generation of female students (SDG 5). We commend THE for going beyond their traditional university performance metrics and including countries and universities not included in their World Rankings. THE claims that “These universities are making the most impact on society” (Bothwell). There is no research showing that the metrics chosen are accurately measuring the SDGs. From Ruth’s Rankings perspective, these are the ratings of a subset of universities who chose to respond to the survey and answered SDG 17.

NOTES 

Association of Commonwealth Universities. (15 May 2019). The road to 2030: Building a better world through higher education Strategy 2019-2025. Accessed at https://www.acu.ac.uk/about-us/road-2030

Bothwell, Ellie (24 May 2019). These universities are making the most impact on society.  European Business Review  accessed 18 June, 2019 at https://www.europeanbusinessreview.eu/page.asp?pid=3056

Data collection portal: THE university impact rankings (8 November 2018). Times Higher Education.  http://www.uc.pt/efs/docs/THEimpactRankingManual

Makoni, M.  (25 May 2019).  SDGs at centre of new Commonwealth universities’ strategy.  University World News accessed 3 July 2019 at https://www.universityworldnews.com/post.php?story=20190521092748564

Masafumi, N, Pendlebury, D, Schnell, J and Szomszor, M.  (April 2019). Navigating the structure of research on sustainable development goals.  Clarivate Analytics. Accessed  at https://www.clarivate.com/wp-content/uploads/dlm_uploads/2019/03/Navigating-the-Structure-of-Research-on-Sustainable-Development-Goals.pdf

O’Malley, B. (10 July 2019). Networks of 7000 universities declare climate emergency.  University World News.  Accessed  at https://www.universityworldnews.com/post.php?story=20190710141435609

O’Malley, B.  (20 July 2019).  SDGs ‘not attainable without contribution of HE’ UN told.  University World News. Accessed at https://www.universityworldnews.com/post.php?story=20190719135507840

Rolf, H. (10 April 2019).  CAUL sustainable development report goals 2019. Council of Australian Librarians. Accessed at https://www.caul.edu.au/news/caul-sustainable-development-goals-report-2019

van’t Land, H. and Herzog. F. (Sept 2017).  Higher education paving the way to sustainable development:  A global perspective – reports on the 2016 IAU Global Survey on Higher Education and Research for Sustainable Development.  Accessed 30 June 2019   at  https://www.iau-aiu.net/IMG/pdf/higher-education-paving-the-way-to-sd-iau-2017.pdf

 

Ruth’s Rankings

  1. Introduction: Unwinding the Web of International Research Rankings
  2. A Brief History of Rankings and Higher Education Policy
  3. Bibliometrics: What We Count and How We Count
  4. The Big Two: Thomson Reuters and Scopus
  5. Comparing Times Higher Education (THE) and QS Rankings
  6. Scholarly Rankings from the Asian Perspective
  7. Asian Institutions Grow in Nature
  8. Something for Everyone
  9. Expanding the Measurement of Science: From Citations to Web Visibility to Tweets
  10. Do-It-Yourself Rankings with InCites
  11. U S News & World Report Goes Global
  12. U-Multirank: Is it for “U”?
  13. A Look Back Before We Move Forward
  14. SciVal – Elsevier’s research intelligence – Mastering your metrics
  15. Analyzing 2015-2016 Updated Rankings and Introducing New Metrics
  16. The much maligned Journal Impact Factor
  17. Wikipedia and Google Scholar as Sources for University Rankings – Influence and popularity and open bibliometrics
  18. Rankings from Down Under – Australia and New Zealand
  19. Rankings from Down Under Part 2: Drilling Down to Australian and New Zealand Subject Categories
  20. World Class Universities and the New Flagship University: Reaching for the Rankings or Remodeling for Relevance
  21. Flagship Universities in Asia: From Bibliometrics to Econometrics and Social Indicators
  22. Indian University Rankings – The Good the Bad and the Inconsistent
  23. Are Global Higher Education Rankings Flawed or Misunderstood?  A Personal Critique
  24. Malaysia Higher Education – “Soaring Upward” or Not?
  25. THE Young University Rankings 2017 – Generational rankings and tips for success
  26. March Madness –The rankings of U.S universities and their sports
  27. Reputation, Rankings and Reality: Times Higher Education rolls out 2017 Reputation Rankings
  28. Japanese Universities:  Is the sun setting on Japanese higher education?
  29. From Bibliometrics to Geopolitics:  An Overview of Global Rankings and the Geopolitics of Higher Education edited by Ellen Hazelkorn
  30. Hong Kong and Singapore: Is Success Sustainable?
  31. Road Trip to Hong Kong and Singapore – Opening new routes for collaboration between librarians and their stakeholders
  32. The Business of Rankings – Show me the money
  33. Authors:  Part 1 – People and processes
  34. Authors: Part 2 – Who are you?
  35. Come together:  May updates lead to an investigation of Collaboration
  36. Innovation, Automation, and Technology Part 1:  From Scholarly Articles to Patents;     Innovation, Automation, and Technology Part 2: Innovative Companies and Countries
  37. How Important are Journal Quality Metrics in the Era of Predatory Journals? Part 1: Journal Citation Metrics; Part 2: How Important are Journal Quality Metrics in the Era of Potential/ possible/ probable predatory publishers and publications?
  38. Coming Attractions: The UN Sustainable Development Goals and Times Higher Education Innovation and Impact Rankings Demystified
  39. Business School Rankings: Monkey Business for an Asia/Pac audience
  40. Deconstructing QS Subjects and Surveys 
  41. THE’s University Impact Rankings and Sustainable Development Goals: Are these the most impactful universities in the world?

*Ruth A. Pagell is emeritus faculty librarian at Emory University.  After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii.  She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674