Ranking Universities

How seriously should we take the Times Higher Education Supplement World University Rankings?

Very seriously. Not because the rankings are accurate, since there are a number of questions that can be and have been raised about the criteria used, but because foundations around the world prefer to fund universities on the list rather than those not on the list.

It makes good business sense for universities to try to get high rankings in this annual survey.

Times has announced the top 400 universities in the world. On top of the list are the usual ones (in order): Harvard (undisputed 1st place), Cambridge, Oxford, and Yale (tied for 2nd place), Imperial College London, Princeton, Caltech and Chicago (tied for 7th place), University College London, and MIT.

Oxford, Yale, Imperial, Princeton, and Chicago moved up from their rankings last year. The most spectacular rise is recorded by University College London, which was only 25th last year. Going down in reputation and quality are MIT, Stanford (6th last year to 19th this year), and UC Berkeley (8th to 22nd).

How do universities in Asia compare with the best in the world?

The top Asian, non-ASEAN university is Tokyo (17th), followed closely by Hong Kong (18th from 33rd last year). Kyoto is in 25th place, followed by Peking (36th), Chinese University of Hong Kong (38th), Tsinghua (40th), Osaka (46th), Seoul (51st), and Hong Kong University of Science and Technology (53rd).

For ASEAN, the top university is still National University of Singapore, though it has moved down to 33rd from 19th last year. Following NUS are Nanyang (69th), Chulalongkorn (223rd), Malaya (246th), Sains (307th), Kebangsaan (309th), Gadja Mada (360th), Putra (364th), Bandung (369th), and Indonesia (395th).

The University of the Philippines is tied for 398th to 400th place, at the very bottom of the list but at least still world-ranked.

The Times criteria have remained the same: peer review of research quality (40%), citations per faculty (20%), graduate employability or recruiter review (10%), proportion of international faculty (5%), proportion of international students (5%), and teaching quality reflected in student/faculty ratio (20%). The amount of money that pours into a university is not considered a factor in the rankings (although, of course, the less non-tuition money, the more students per faculty).

Times, however, has revised the way it uses its criteria for this year’s rankings. The biggest change is the removal of Thomson ISI from the Times process. Instead, Times now uses Scopus, which according to Times, “has a less pronounced bias towards the US, resulting in a reduced advantage in their favor in this indicator, covers a larger number of papers and journals overall leading to greater representation from lesser known universities and institutions from academic systems with less emphasis on publication, and covers more sources in languages other than English resulting in better numbers for institutions with large volumes of high quality research in their own language.”

This is a welcome development. I have time and again attacked the bias displayed by Thomson ISI in their listing, and I am really happy that Times has dropped Thomson ISI.

Another welcome change is the use of standard scores (z-scores), which are much more reliable than the type Times used last year. This change has removed the bias for universities that excel only in a few areas. The list now reflects overall excellence.

In listing the world’s top universities, Times uses the following criteria: peer review (40%), employer review (10%), staff per student (20%), citations per staff (20%), international staff (5%), and international students (5%).

Let us look at each of these criteria.

Times says: “The core of our methodology is the belief that expert opinion is a valid way to assess the standing of top universities. The opinions are gathered, like the rest of the rankings data, by our partners QS Quacquarelli Symonds (www.topuniversities.com), which has built up a database of e-mail addresses of active academics across the world. They are invited to tell QS what area of academic life they come from, choosing from science, biomedicine, technology, social science or the arts and humanities. They are then asked to list up to 30 universities that they regard as the leaders in the academic field they know about, and in 2007 we have strengthened our measures to prevent anyone voting for his or her own institution. This year we have the opinions of 5,101 experts, of whom 41 per cent are in Europe, the Middle East and Africa, 30 per cent in the Americas, and 29 per cent in the Asia-Pacific region.”

Since peer review is really a popularity contest among academics, the results depend on who are doing the voting. If Philippine universities want to be included in the list, they must be proactive as far as academic opinion is concerned. We have numerous Ph.D. holders in our universities. We should have ourselves listed in the QS database. This is not difficult; all we have to do is to send our names online. We cannot vote for our own universities, but we can vote for other Philippine universities. If we vote for each other, our peer review scores will rise. I am not completely sure that this is ethical, but if we do not look out for each other, who will look out for us?

The next criterion – employer review – is something we do not have to worry about. Because of the large number of Filipino college graduates going abroad, numerous employers all over the world are aware of our universities. This does not mean that, ideologically speaking, we should be educating for overseas employment, but since the majority of our graduates apparently go abroad anyway, and since our economy depends on remittances, there is not much we can do to reverse the OFW phenomenon.

The third criterion – staff per student – is something we can do something about. Our universities, by and large, depend on student tuition for their income. That is why we pack students into classrooms, thus enlarging our class sizes and decreasing the ratio of staff (faculty) to students.

That is not the case with the top universities in the world. Other universities earn money outside of tuition. For example, like many top universities, Oxford and Stanford have malls inside their campuses. Like other top universities, Harvard earns from the patents of every invention produced by its faculty. This may sound like heresy to Filipino professors, but it is about time that our universities claim the intellectual rights to everything produced by professors during their time of employment. Since full-time professors are legally obliged to work for 40 hours a week, this means, in effect, that every invention or creation done by a professor is owned completely by her or his university.

A major criterion used by Times is the number of citations researchers in a university get in scholarly journals and books.

This is the way the process works. A scholar writes a book or journal article. Another scholar refers to that book or article. The citation by the second scholar is listed by Thomson ISI, Scopus, and similar databases. The more citations by the second and succeeding scholars a book or article receives, the higher the score is for the university of the original scholar.

Since ISI or Scopus cannot cover all the books and journals being published every year (not even Google Scholar does), there are a number of journals that are considered major. These are the only ones actually covered by the databases. (There is a theory justifying the choice of only a few journals. Early this year, I argued in Canada against this outdated theory.) These journals are known as ISI or Scopus journals.

Libraries around the world, not having enough money to subscribe to the more than 36,000 journals, limit their subscriptions to the ISI or Scopus-listed journals. This means that scholars cite only articles in these journals, not those in non-listed journals. There is, then, a Catch-22 situation here for us: in order for Philippine journals to be listed, they must be read by foreign scholars, but foreign scholars cannot read them unless they are already listed.

One way to break this impasse is to send complimentary copies to libraries around the world. This is extremely expensive, and our universities cannot afford to do this while waiting to be cited.

An easier and less expensive way is to go online. There are a number of Philippine journals online (those published by Ateneo, IRRI, UP, to cite a few), but with the billions of websites online, there is little chance for these journals to be included by Google Scholar on its first few search pages.

One solution is something that I am currently trying to push. There is an NGO in London called International Network for the Availability of Scientific Publications (INASP). It puts journals from underread countries on their website, thus ensuring greater visibility on the Web. I have already obtained the commitment of INASP to include our journals on its site.

The INASP Philippine representative is Lourdes David of Ateneo’s Rizal Library. David and I have written to most Philippine universities to have their journals included in a project I have called Philippine Journals Online.

Asia-Pacific College has offered to host the journals on its IBM and SM-funded state-of-the-art computers. C&E Publishing has offered to translate the articles in the journals to a form readable by Open Journal Systems, the open-source software developed by Canada and used by INASP.

Once our journals are online and retrievable via London and Google Scholar, we can expect scholars around the world to read us and eventually to cite us. If citation is the name of the game, this is the way to compete.

Another Times criterion is the number or percentage of international teachers in a university. Having taught in various universities abroad, I know how important it is for “locals” to interact with visiting professors.

Inbreeding or having only locals teaching locals is the worst thing that can happen to a university, since it is the equivalent of marrying only within one’s clan. There are many ways, not all of them expensive, to get foreigners to teach in our universities.

The last Times criterion is the number or percentage of international students in a university. This is one way to solve two problems at the same time. The first is income. International students bring in foreign currency. The second is the use of English. The simplest way to encourage Filipino students to practice speaking English on campus is to let them have international students as classmates. (The Philippine Star, 30 November, 7 & 14 December 2007)