Ruth’s Rankings 1. Introduction: Unwinding the Web of International Research Rankings

By Ruth A. Pagell*

(5 July 2014) I find rankings fascinating.  I wrote my first article about rankings over 25 years ago when I was trying to sort out different listings of “biggest” U.S. companies and  moved on to researching biggest companies world-wide.  Who was biggest varied among different sources based on different criteria. Two decades after that first article, I was asked to examine the new world-wide university research rankings (Pagell, 2009) and I have just finished a third round of research which appears in a chapter of a new Springer book that was co-sponsored by Wuhan University (2014).   I found similar differences in rankings based on sources, criteria and methodology.  These rankings are impacting the strategies and policies of individual institutions, funding bodies and governments.

A few years before my first article,  U.S. News & World Report began publishing its list of best American universities followed in the next decade by a growing emphasis on the lists of best MBA programs.  While working at an aspiring MBA program, I saw the impact these rankings had on the school’s strategies and policies as we tried to find ways to improve our rankings.

We have  entered  the second decade of world-wide rankings. The first world-wide rankings were Shanghai Jiao Tong University’s Academic Ranking of World Universities (ARWU)  published  in 2003.  The number of rankings agencies and criteria has been growing along with the analysis and criticism.

North American and European universities still hold most of the tops spots but Asian universities have been rising in rankings in line with their increased output of the world’s science and engineering publications.  As illustrated in Figure 1 below, from the U.S. National Science Foundation’s 2014 Digest,   “Asia’s S&E research article output is approaching parity with the United States and the European Union. Between 1997 and 2011, Asia’s output more than doubled, led primarily by China.  In 2011, China produced 11% of the world’s S&E articles, more than any country except the United States.”

Figure 1: Science & Engineering articles

Source: http://www.nsf.gov/statistics/seind14/index.cfm/digest

Regular readers of ACCESS have seen seven articles on rankings over the past 12 months alone.  Who is number one in any ranking depends on who is doing the rankings, what is being ranked, why the rankings are being done and even where the rankings are being done.

The intersection of these rankings with librarians and LIS research is in the use of bibliometrics as a major rankings component.  Librarians, as the keepers of the subscriptions to underlying sources such as Web of Science, Scopus and Journal Citation Reports, need to have some understanding of the rankings and their research and scholarly components.

In order to help librarians and researchers who are trying to decipher what these rankings mean to themselves and their institutions, ACCESS will publish a series of short articles on the different strands of the web of rankings.  I will look at these issues through the eyes of a practicing academic librarian and LIS  instructor with special emphasize on Asia

The next article will be an overview of the evolution of rankings and the use of bibliometrics at national and multi-national levels. The following articles will include:

  • A description of the key bibliographic metrics
  • The companies that provide the underlying data
  • Templates comparing the different ranking sources
  • Newer ranking metrics such as Webometrics and Altmetrics
  • Specialized rankings, such as those by subject area
  • We will also see if we can find evidence to show how rankings have been changing higher education

Rankings are published at different times of the year and are compiled by specialized research institutions, universities and by commercial publishers. Table 1 below lists the top 10 Asian universities as of June 25th 2014 from nine different rankings. The University of Tokyo is number one in six of these.  The University of Tokyo,  Seoul National University and National University of Singapore are on eight of the lists.

Table 1: Top 10 Asian Universities from Nine Different Rankings (pdf)

Table 2 shows a selected list of where these top Asian universities are ranked relative to the world-wide rankings.

Table 2: World-wide Rankings of Asian Universities (pdf)

The articles that follow in the coming months will provide the information needed to decipher the different rankings. Further, at this time next year, we will re-examine the rankings to see the changes over the year.  We should have a better understanding of what we are seeing and evaluate what impact, if any, it will have on the bigger picture of higher education performance and policy.

When rankings were confined to national borders, articles about rankings appeared in popular local  magazines.  With the growth of scientometrics as an academic discipline, more scholarly articles are appearing not only about the metrics themselves but also about the intersection of the metrics and university research rankings. Readers more interested in the scholarly aspects of these topics can check the bibliographies in the references below.   Most of the research is being done outside the U.S. in countries such as Spain, Netherlands, China, U.K.  and newcomer, Italy.

The next article will provide a brief history of rankings and focus on the increased emphasis on scholarly accountability at national, regional and international levels.

References 

Pagell, R. A. 2014. Bibliometrics and University Research Rankings Demystified for Librarians. Chen, C. and Larsen, R. (eds.) Library and Information Science: Trends and Research.(Open Access )  Bibliometrics and University Research Rankings Demystified for Librarians – Springer 

Pagell, R. A. 2009. University Research Rankings : From Page Counting to Academic Accountability. Evaluation in Higher Education, Vol. 3, No. 1,  pp.71-101. Retrieved September 25, 2013 from http://ink.library.smu.edu.sg/library_research/1.

European Journal of Education (March 2014) Special issue: Global University Rankings.  A Critical

Ruth’s Rankings

  1. Introduction: Unwinding the Web of International Research Rankings
  2. A Brief History of Rankings and Higher Education Policy
  3. Bibliometrics: What We Count and How We Count
  4. The Big Two: Thomson Reuters and Scopus
  5. Comparing Times Higher Education (THE) and QS Rankings
  6. Scholarly Rankings from the Asian Perspective 
  7. Asian Institutions Grow in Nature
  8. Something for Everyone
  9. Expanding the Measurement of Science: From Citations to Web Visibility to Tweets
  10. Do-It-Yourself Rankings with InCites 
  11. U S News & World Report Goes Global
  12. U-Multirank: Is it for “U”?
  13. A Look Back Before We Move Forward
  14. SciVal – Elsevier’s research intelligence –  Mastering your metrics
  15. Analyzing 2015-2016 Updated Rankings and Introducing New Metrics
  16. The much maligned Journal Impact Factor
  17. Wikipedia and Google Scholar as Sources for University Rankings – Influence and popularity and open bibliometrics
  18. Rankings from Down Under – Australia and New Zealand
  19. Rankings from Down Under Part 2: Drilling Down to Australian and New Zealand Subject Categories
  20. World Class Universities and the New Flagship University: Reaching for the Rankings or Remodeling for Relevance
  21. Flagship Universities in Asia: From Bibliometrics to Econometrics and Social Indicators
  22. Indian University Rankings – The Good the Bad and the Inconsistent
  23. Are Global Higher Education Rankings Flawed or Misunderstood?  A Personal Critique
  24. Malaysia Higher Education – “Soaring Upward” or Not?
  25. THE Young University Rankings 2017 – Generational rankings and tips for success
  26. March Madness –The rankings of U.S universities and their sports
  27. Reputation, Rankings and Reality: Times Higher Education rolls out 2017 Reputation Rankings
  28. Japanese Universities:  Is the sun setting on Japanese higher education?
  29. From Bibliometrics to Geopolitics:  An Overview of Global Rankings and the Geopolitics of Higher Education edited by Ellen Hazelkorn 

*Ruth A .Pagell is currently teaching in the Library and Information Science Program at the University of Hawaii.   Before joining UH, she was the founding librarian of the Li Ka Shing Library at Singapore Management University.  She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS.