By Ruth A. Pagell*
(5 July 2014) I find rankings fascinating. I wrote my first article about rankings over 25 years ago when I was trying to sort out different listings of “biggest” U.S. companies and moved on to researching biggest companies world-wide. Who was biggest varied among different sources based on different criteria. Two decades after that first article, I was asked to examine the new world-wide university research rankings (Pagell, 2009) and I have just finished a third round of research which appears in a chapter of a new Springer book that was co-sponsored by Wuhan University (2014). I found similar differences in rankings based on sources, criteria and methodology. These rankings are impacting the strategies and policies of individual institutions, funding bodies and governments.
A few years before my first article, U.S. News & World Report began publishing its list of best American universities followed in the next decade by a growing emphasis on the lists of best MBA programs. While working at an aspiring MBA program, I saw the impact these rankings had on the school’s strategies and policies as we tried to find ways to improve our rankings.
We have entered the second decade of world-wide rankings. The first world-wide rankings were Shanghai Jiao Tong University’s Academic Ranking of World Universities (ARWU) published in 2003. The number of rankings agencies and criteria has been growing along with the analysis and criticism.
North American and European universities still hold most of the tops spots but Asian universities have been rising in rankings in line with their increased output of the world’s science and engineering publications. As illustrated in Figure 1 below, from the U.S. National Science Foundation’s 2014 Digest, “Asia’s S&E research article output is approaching parity with the United States and the European Union. Between 1997 and 2011, Asia’s output more than doubled, led primarily by China. In 2011, China produced 11% of the world’s S&E articles, more than any country except the United States.”
Figure 1: Science & Engineering articles
Source: http://www.nsf.gov/statistics/seind14/index.cfm/digest
Regular readers of ACCESS have seen seven articles on rankings over the past 12 months alone. Who is number one in any ranking depends on who is doing the rankings, what is being ranked, why the rankings are being done and even where the rankings are being done.
The intersection of these rankings with librarians and LIS research is in the use of bibliometrics as a major rankings component. Librarians, as the keepers of the subscriptions to underlying sources such as Web of Science, Scopus and Journal Citation Reports, need to have some understanding of the rankings and their research and scholarly components.
In order to help librarians and researchers who are trying to decipher what these rankings mean to themselves and their institutions, ACCESS will publish a series of short articles on the different strands of the web of rankings. I will look at these issues through the eyes of a practicing academic librarian and LIS instructor with special emphasize on Asia
The next article will be an overview of the evolution of rankings and the use of bibliometrics at national and multi-national levels. The following articles will include:
- A description of the key bibliographic metrics
- The companies that provide the underlying data
- Templates comparing the different ranking sources
- Newer ranking metrics such as Webometrics and Altmetrics
- Specialized rankings, such as those by subject area
- We will also see if we can find evidence to show how rankings have been changing higher education
Rankings are published at different times of the year and are compiled by specialized research institutions, universities and by commercial publishers. Table 1 below lists the top 10 Asian universities as of June 25th 2014 from nine different rankings. The University of Tokyo is number one in six of these. The University of Tokyo, Seoul National University and National University of Singapore are on eight of the lists.
Table 1: Top 10 Asian Universities from Nine Different Rankings (pdf)
Table 2 shows a selected list of where these top Asian universities are ranked relative to the world-wide rankings.
Table 2: World-wide Rankings of Asian Universities (pdf)
The articles that follow in the coming months will provide the information needed to decipher the different rankings. Further, at this time next year, we will re-examine the rankings to see the changes over the year. We should have a better understanding of what we are seeing and evaluate what impact, if any, it will have on the bigger picture of higher education performance and policy.
When rankings were confined to national borders, articles about rankings appeared in popular local magazines. With the growth of scientometrics as an academic discipline, more scholarly articles are appearing not only about the metrics themselves but also about the intersection of the metrics and university research rankings. Readers more interested in the scholarly aspects of these topics can check the bibliographies in the references below. Most of the research is being done outside the U.S. in countries such as Spain, Netherlands, China, U.K. and newcomer, Italy.
The next article will provide a brief history of rankings and focus on the increased emphasis on scholarly accountability at national, regional and international levels.
References
Pagell, R. A. 2014. Bibliometrics and University Research Rankings Demystified for Librarians. Chen, C. and Larsen, R. (eds.) Library and Information Science: Trends and Research.(Open Access ) Bibliometrics and University Research Rankings Demystified for Librarians – Springer
Pagell, R. A. 2009. University Research Rankings : From Page Counting to Academic Accountability. Evaluation in Higher Education, Vol. 3, No. 1, pp.71-101. Retrieved September 25, 2013 from http://ink.library.smu.edu.sg/library_research/1.
European Journal of Education (March 2014) Special issue: Global University Rankings. A Critical
Ruth’s Rankings
A list of Ruth’s Rankings and News Updates is here.
*Ruth A. Pagell is currently teaching in the Library and Information Science Program at the University of Hawaii. Before joining UH, she was the founding librarian of the Li Ka Shing Library at Singapore Management University. She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS. https://orcid.org/0000-0003-3238-9674
