Ruth’s Rankings 42: ASEAN – a special analysis of ASEAN nations

  • What ASEAN country’s citations have the greatest impact?
  • What country has the most top 10 ASEAN universities?
  • What ASEAN country’s population is most satisfied with its education system?

By Ruth A. Pagell*

(30 Oct 2019) Writing an overview report on ASEAN nation’s’ university rankings had been on my agenda but interesting reports or new rankings have come along and ASEAN got pushed back. This month, Clarivate Analytics’ Global Research Report – South and East Asia caught my attention (Adams).

10 countries are members of Association of Southeast Asian Nations (ASEAN).  The goals of the organization are about mutual cooperation while the Fundamental Principles articulate independence and non-interference. The ten countries are Brunei Darussalam, Cambodia, Indonesia, Laos (Lao PDR), Malaysia, Myanmar (formerly Burma), the Philippines, Thailand and Vietnam (Viet Nam). Included are large and small countries, by size and population, rich and poor countries and countries with flawed democracies, authoritarian governments and an absolute monarchy. They all have their educational challenges. In addition to ASEAN, the CA report covers Bangladesh, India, Pakistan and Sri Lanka, which are not included in this analysis.

All the ASEAN countries except Thailand have colonial histories:

  • Commonwealth: Brunei, Malaysia, and Singapore are members while Burma chose not to join when it gained its independence in 1948
  • Former French colonies: Cambodia, Laos and Vietnam
  • Dutch – Indonesia
  • Spanish and then US. – The Philippines

The focus of the CA Report is country-level research impact, with emphasis on citation impact and collaboration. We tie this data to university rankings.

We cannot replicate the Report’s data. They include the 14 countries in the report. India by itself has one and a half times more publications and twice the population of all of ASEAN. In order to better compare the data to the university rankings, this article uses proprietary bibliometric datasets covering 2014-2018 derived from Clarivate Analytics’ Web of Science (extracted from InCites) and Elsevier’s SCOPUS (extracted from SciVal).  Unless otherwise noted, we limited our output to articles and reviews and use a median score rather than an average to limit the effect of outliers.  Also, both bibliometric and economic data are dependent on definitions, calculations, exchange rates (for monetary figures) and time frame.

Publication Output

Research from ASEAN institutions is gradually increasing as a percent of all output. Using the time period 1981, the first available Incites data, through the most recent complete year (2018) the percent of publication output from ASEAN countries rose from 0.25% of the world total to 2.79%, an increase of over 1100%.  See Figure 42.1 below.  WOS Output from 1981 – 2018.

During the same time, the total world output rose at 421% and the number of publications that are indexed also rose at 252%. Data from 2014-2018 on SciVal provide somewhat similar results for the current time period.  ASEAN institutions comprise 2.7% of all institutions and 3.07% of all publication output. The ASEAN output is still not commensurate with its eight percent of the world population.

Collaboration

The CA Report emphasizes international collaboration as an important factor contributing to the increase in ASEAN output. Worldwide, countries with low output often have a high percent of international collaboration.  22 countries have 100% international collaboration, and none have more than 200 2018 publications, China, Japan, South Korea and the US are all in the bottom 12.

Over 90 percent of articles published by researchers from Cambodia, Myanmar and Laos are international collaborations.  30 of Myanmar’s 32 researchers publish with international collaborators. From my experience in US universities, for many years, collaboration of any kind was discouraged and went unrewarded in promotion and tenure evaluations.

Category Normalized Citation Impact (CNCI)

The Report acknowledges that a measurement of citation impact is not on its own a measure of quality. However, it emphasizes CNCI in its report in relation to no collaboration, domestic collaboration and international collaboration.  We can see the need for care interpreting CNCI when North Korea’s score is higher than South Korea’s.  Only 13 of the 53 countries with the highest output for 2018 are in the top quartile for CNCI.   See Figure 42.2 (in pdf) using Cambodia as the example of the Report’s presentation of the collaboration /CNCI relationship.

RANKINGS

The Report emphasizes the economic aspects of the region’s research output.  This article ranks countries using our usual metrics and other metrics that affect the research and educational environment.

Countries

When evaluating the ASEAN nations’ institutional performance, other indicators need to be examined. Metrics such as population, income, expenditures on R&D, literacy rate, democracy and press freedom all contribute to the scholarly environment of a country. See Appendix A (in pdf, updated 11/11/2019). It has selected news stories about the educational environment in each ASEAN country. See Table 42.1 (in pdf) for bibliometric data from both WOS and SCOPUS.  Table 42.2 (in pdf) presents other indicators for those interested in socio-economics.  It is not surprising that the lower income countries, such as Cambodia, Laos and Myanmar spend less on R&D and have lower literacy rates. Indonesia is the largest country with 41% of ASEAN population.  It is also the largest in geographic size and number of tertiary graduates.  What is surprising is that Cambodia has the highest score on citizen satisfaction for education system.  What is consistent among all ASEAN nations is their low rankings on Freedom of the Press.  Out of 180 ranked countries, Malaysia at 123 has the best rank!  Singapore, in the bottom quartile in press freedom ranks in the top quartile in human freedom.

University Rankings

How do the countries’ university rankings map to their socio-economic data?  Not surprisingly Cambodia, Laos and Myanmar are only ranked in Webometrics which covers over 28,000 tertiary institutions.  History, war, and politics play a role in the absence of these countries’ institutions on the world scene.  Singapore, with the largest percent of its budget going to research and development has only two universities that are consistently ranked.  Much of Singapore’s R&D money is funneled through A*Star, a statutory board that works with universities and with private research institutions.  Only 25% of Singapore’s research publications are from its universities.

The following rankings are included, arranged by the provider of the bibliometric data that I used and then by number of universities.  All rankings were released in 2019.

– Using Scopus (SciVal dataset)

  1. Webometrics – Ranking Web of Universities – Ranking Web of Universities
  2. Scimago Institutions Ranking (SIR)
  3. QS Asian Rankings 2019 with 38 additional ASEAN universities
  4. QS World University Rankings 2020 – Top Universities
  5. THE World University Rankings
  6. THE University Impact Rankings 2019

– Using WOS (InCites dataset)

  1. U.S. News Best Global Universities 2020
  2. Academic Rankings of World Universities (ARWU) 2019
  3. NTU Performance Ranking of Scientific Papers for World Universities 2019
  4. CWTS Leiden Rankings 2019
  5. Nature index – Includes mainly non-academic institutions for ASEAN see tables at https://www.nature.com/collections/fbfjafhcbb/tables

Table 42.3 (in pdf) lists the top 10 universities overall from the rankers and from WOS and Scopus datasets. THE giving individual ranks to only the top 200 and ARWU with individual ranks to only the top 100, have more than 10 each because of banding. National University of Singapore (NUS), Nanyang Technological University (NTU), and University of Malaya are top 3 in all rankings.   Malaysia takes the prize for the top country with 8 universities.  6 are in all 10 rankings and 1 in 9 rankings.  7 Thai universities appear in at least 1 top 10 ranking, with Mahidol appearing in 9 top 10 lists.  Also represented are 2 more universities from Singapore and The Philippines and 1 each from Brunei and Indonesia.  Looking at the distribution of top universities by country, I have to ask myself if it is better to have the 2 top universities in the region or a total of 8 recognized in the top 10 overall.

Table 42.4 (in pdf) ranks the top 10 universities from each ASEAN country (when there are 10). Most ASEAN universities are not ranked in the top 100 let alone the top 500. Malaysia (RR 24) and Singapore (RR 30) were covered in past articles and appear here in relationship to the other ASEAN nations.

CONCLUSION:

The CA Report concludes that improvement in 3 factors would decrease the gap between ASEAN and the rest of the world, in my words,

  • Human capital since the number of researcher is still relatively small
  • A university driven research environment free from government control
  • More investment in science and technology education and research

It also recognizes that there is “insufficient capacity in some countries” to even contribute at a local level.  We have seen in Table 42.2 that many ASEAN nations are not investing in research and development and they are operating in countries with a high level of government control.  The issue of language has not been raised, but on a list from Leicester University Singapore was the only ASEAN nation that qualified for adequate English.

Thinking back on other articles, I need to remind readers of the balance between growing a larger role on the world stage or building up a country’s educational infrastructure.

NOTE:  U.S. News Global released its 2020 list with 1,500 universities and Nature has a new list of rankings which will be included in a November update.

 

RESOURCES

Adams, J. et; al (2 Oct 2019).  Global research report – South and East Asia.  Institute for Scientific Information.  Accessed at  https://clarivate.com/webofsciencegroup/campaigns/south-and-east-asia/

ASEAN statistical yearbook 2018 (Dec 2018) https://asean.org/storage/2018/12/asyb-2018.pdf

Democracy Index 2018:  Me too (2019). The  Economist Intelligence Unit accessible online with registration: https://www.eiu.com/public/topical_report.aspx?campaignid=Democracy2018

Makoni, M. (19Oct 2019).  Economic growth, political stability fuel research rise.  University World News, accessed at https://www.universityworldnews.com/post.php?story=20191018081259845

Mallow, S, (Oct 2016) Higher education in ASEAN.  International Association of Universities accessed at https://www.iau-aiu.net/Higher-Education-in-ASEAN

OECD/AADB (10 Sep 2019).  Government at a glance: Southeast Asia.  OECD Publishing, Paris accessed at http://doi.org/10.1787/9789264305915-en

Vásquez, I. & Porcnik, T. (10 Dec 2018).  Human Freedom Index -2018/ Cato, et al.  accessed at  https://www.cato.org/sites/cato.org/files/human-freedom-index-files/human-freedom-index-2018-revised.pdf

Willliams, R. & Leahy, A. (Mar 2019).  Ranking of higher education systems 2019.  Accessed at https://universitas21.com/what-we-do/u21-rankings/u21-ranking-national-higher-education-systems-2019

 

Ruth’s Rankings

  1. Introduction: Unwinding the Web of International Research Rankings
  2. A Brief History of Rankings and Higher Education Policy
  3. Bibliometrics: What We Count and How We Count
  4. The Big Two: Thomson Reuters and Scopus
  5. Comparing Times Higher Education (THE) and QS Rankings
  6. Scholarly Rankings from the Asian Perspective
  7. Asian Institutions Grow in Nature
  8. Something for Everyone
  9. Expanding the Measurement of Science: From Citations to Web Visibility to Tweets
  10. Do-It-Yourself Rankings with InCites
  11. U S News & World Report Goes Global
  12. U-Multirank: Is it for “U”?
  13. A Look Back Before We Move Forward
  14. SciVal – Elsevier’s research intelligence – Mastering your metrics
  15. Analyzing 2015-2016 Updated Rankings and Introducing New Metrics
  16. The much maligned Journal Impact Factor
  17. Wikipedia and Google Scholar as Sources for University Rankings – Influence and popularity and open bibliometrics
  18. Rankings from Down Under – Australia and New Zealand
  19. Rankings from Down Under Part 2: Drilling Down to Australian and New Zealand Subject Categories
  20. World Class Universities and the New Flagship University: Reaching for the Rankings or Remodeling for Relevance
  21. Flagship Universities in Asia: From Bibliometrics to Econometrics and Social Indicators
  22. Indian University Rankings – The Good the Bad and the Inconsistent
  23. Are Global Higher Education Rankings Flawed or Misunderstood?  A Personal Critique
  24. Malaysia Higher Education – “Soaring Upward” or Not?
  25. THE Young University Rankings 2017 – Generational rankings and tips for success
  26. March Madness –The rankings of U.S universities and their sports
  27. Reputation, Rankings and Reality: Times Higher Education rolls out 2017 Reputation Rankings
  28. Japanese Universities:  Is the sun setting on Japanese higher education?
  29. From Bibliometrics to Geopolitics:  An Overview of Global Rankings and the Geopolitics of Higher Education edited by Ellen Hazelkorn
  30. Hong Kong and Singapore: Is Success Sustainable?
  31. Road Trip to Hong Kong and Singapore – Opening new routes for collaboration between librarians and their stakeholders
  32. The Business of Rankings – Show me the money
  33. Authors:  Part 1 – People and processes
  34. Authors: Part 2 – Who are you?
  35. Come together:  May updates lead to an investigation of Collaboration
  36. Innovation, Automation, and Technology Part 1:  From Scholarly Articles to Patents;     Innovation, Automation, and Technology Part 2: Innovative Companies and Countries
  37. How Important are Journal Quality Metrics in the Era of Predatory Journals? Part 1: Journal Citation Metrics; Part 2: How Important are Journal Quality Metrics in the Era of Potential/ possible/ probable predatory publishers and publications?
  38. Coming Attractions: The UN Sustainable Development Goals and Times Higher Education Innovation and Impact Rankings Demystified
  39. Business School Rankings: Monkey Business for an Asia/Pac audience
  40. Deconstructing QS Subjects and Surveys 
  41. THE’s University Impact Rankings and Sustainable Development Goals: Are these the most impactful universities in the world?
  42. ASEAN – a special analysis of ASEAN nations
  43. Predatory practices revisited – misunderstandings and positive actions
  44. Part 1: What’s the best university for an international student? Metrics from the student’s perspective

*Ruth A. Pagell is emeritus faculty librarian at Emory University.  After working at Emory, she was the founding librarian of the Li Ka Shing Library at Singapore Management University and then adjunct faculty [teaching] in the Library and Information Science Program at the University of Hawaii.  She has written and spoken extensively on various aspects of librarianship, including contributing articles to ACCESS – https://orcid.org/0000-0003-3238-9674