Media Statement by Dr. Ong Kian Ming, Head of Penang Institute in Kuala Lumpur, on the 8th of September 2017
The Ministry of Higher Education (MOHE) should not be overly obsesses with global university rankings but instead focus on locally developed indicators to improve our universities
The Times Higher Education (THE) World University Rankings for 2018 were released earlier this week. The initial headlines focused on the fact that two United Kingdom universities, the University of Oxford and the University of Cambridge, topped the rankings coming in at number one and two respectively.
Malaysian universities, however, did not fare so well. Universiti Malaya (UM) was the highest ranked Malaysian university at the 351-400 range. Universiti Tunku Abdul Rahman (UTAR) was the next highest ranked Malaysian university at the 501-600 range. Universiti Kebangsaan Malaysia (UKM), Universiti Putra Malaysia (UPM), Universiti Sains Malaysia (USM), Universiti Teknologi Malaysia (UTM), Universiti Teknologi Petronas (UTP) were all ranked in the 601-800 range followed by Universiti Tenaga Nasional (UniTEN) in the 801-1000 range and Universiti Utara Malaysia (UUM) in the 1001+ range (See Table 1 below).
Table 1: Ranking of Malaysian Universities in the Times Higher Education (THE) World University Rankings 2018
The position of Malaysian universities in the THE rankings stand in stark contrast to the 2018 QS World University Rankings where five Malaysian universities were ranked in the top 300 with UM occupying the 114th position (UPM was ranked 229, UKM was ranked 230, UTM was ranked 253 and USM was ranked 264).
If we examine the Shanghai Jiao Tong Academic Ranking of World Universities (ARWU) and the US News and World Report’s Best Global University Rankings, two of the other well-known global university rankings, we also find Malaysian universities being ranked outside the top 300. For example, in the ARWU rankings for 2017, UM and USM were ranked in the 401-500 range while UKM and UPM were ranked in the 501-600 range and UTM in the 701-800.
In the US News 2017 ranking, UM was the highest ranked Malaysian university at 356 followed by USM (576), UTM (639), UPM (670) and UKM (783).
Figure 1 below shows the difference in the rankings achieved by Malaysian universities in the QS World University Rankings versus the other three global university rankings – THE, ARWU and the US News ranking.
Figure 1: Latest Global University Rankings of the Top 5 Research Universities in Malaysia (2017/2018)
What can explain the differences in the performance of Malaysian universities in the QS rankings versus the other well-known global university rankings? One likely reason is that the QS rankings allocate the lowest percentage of its overall score to research and citation based measures. Table 2 below summarizes the components of these four global university rankings and calculates the overall weightage which is given to research and citation based measures i.e. the publication output of a university. The QS ranking only allocates 20% of its overall score to research and citation measures. In comparison 60% of the THE, 70% of the Shanghai AWRU and 75% of the US News rankings are allocated to research and citation measures.
Under the QS rankings, academic reputation and employer reputation account for 50% of the overall score. These are subjective measures which can be heavily influenced by the sample of respondents surveyed. For example, in the latest 2018 QS rankings, the representation of Malaysian academics in the academic survey is unduly large, considering that Malaysia makes up merely 0.41% of the world’s population yet its representation in the academic survey is 3.7%. The percentage of Malaysian respondents in the academic survey is even greater than countries like China (1.7%), Germany (2.9%) and Japan (3.2%).
The percentage of international academics and students make up 10% of the overall QS score and discerning universities which want to improve their QS ranking can increase these figures without necessarily increasing the quality of teaching, of academic research or of student quality. Indeed, one should not forget that the QS rankings [1] in 2004 ranked UM at 89th in the world because Chinese and Indian Malaysian students and lecturers were mistakenly classified as foreigners thereby artificially inflating UM’s International Faculty and Student scores. [2]
Table 2: Components and Weightage of the QS, THE, Shanghai ARWU and US News Rankings
The Ministry of Higher Education’s (MOHE) decision to benchmark Malaysian universities using the QS World University Rankings is thus short-sighted and faulty since the QS rankings do not give much emphasis on research output. MOHE is fooling itself and the Malaysian public if it continues to use the improvement of Malaysian universities in the QS rankings as a sign that our universities are improving, especially on the research front.
This does not mean that we should rely on the other global university rankings which give more weightage to research output. The other ranking systems are also not without their own weaknesses. For example, the Shanghai ARWU rankings gives too much weightage to the science subjects and publications and also to universities with previous winners of Nobel prizes and Field medals. A Malaysian university can conceivably improve its Shanghai ARWU ranking by giving short term fellowships to a few Nobel prizes and Fields Medals winners but this may not improve the overall research output of that university.
Rather than be obsessed with the ranking game, MOHE should instead work to improve the existing academic indicators and measures which have been developed locally by the Ministry and the Malaysian Qualifications Agency (MQA) to assess the quality of local public and private universities such as the Malaysian Research Assessment Instrument (MyRA), the Rating System for Malaysian Higher Education Institutions (SETARA) and the Discipline-Based Rating System (D-SETARA). Currently, there is very little transparency or disclosure on how the data for these measures are being collected and even less discussion on how to improve these measures so that they can be used by the public to evaluate the quality of these institutions and for these institutions to benchmark themselves. For example, MySetara loses much of its value if the majority of our private and public universities are given five stars (out of a possible six). How do we differentiate between these universities in terms of the quality of research output or the standard of teaching, just to name two?
The Penang Institute in Kuala Lumpur has published a report entitled “An unhealthy obsession with Global University Rankings?” [3] as a way to increase awareness on the strengths and weaknesses of each global university ranking system and to make the argument that MOHE should not be overly obsessed with Malaysia’s performance on these rankings but instead focus on improving the local measures of university quality developed by the MOHE and MQA. If we focus on improving our universities according to more suitable locally developed indicators, the output of our universities in terms of the quality of research, the quality of teaching and the quality of graduates will also improve. And if a by-product of this improvement is that our rankings rise in these global university rankings systems, then this can be seen as a bonus.
[1] This was when these rankings were published jointly by QS and THE. They started publishing separate rankings in 2010.
[2] http://rankingwatch.blogspot.my/2012/08/universiti-malaya-again-in-many.html#links
[3] Authored by Lee Zi-Sheng, an intern with Penang Institute in KL and Dr. Ong Kian Ming, Head of Penang Institute in KL. A copy of this report will be uploaded to the Penang Institute in KL website: https://penanginstitute.org/v3/research/penang-institute-in-kuala-lumpur