Tag Archives: ranking

League Tables are Over-rated

17 Oct

League Tables are over rated. Then why do we pay so much attention to them?

09/13/2013 |
Just this week the annual QS league tables were released, and as expected Indian Universities were not in the top 200 or 300 ranks. This has been the trend for years in all the league tables including the Times Higher Education Rankings and the Shanghai Rankings. The distress at not ranking high is palpable, but before we do aim to do so, it might be a thought to review what league tables can and cannot do for us.

League Tables are over rated, clearly. Then why do we pay so much attention to them? Because they are all we have as a tool to bring an objective comparison to the quality of institutions of education.

Every soul of some erudition knows that the right thing to do is to look into the middle distance, then narrow their eyes and shake their head while slowly saying, “Well, you know… league tables.. they are not really a measure”

True. They are not a measure (of what? Presumably Quality). They are a collation of proxy measures of Quality in education institutions that enable comparison.

League tables can only measure the things that can be quantified. Their criteria have to be designed in such a manner that ensures that they are able to collect data from all educational institutions in a consistent manner. This is their claim to fame – being able to bring objectivity and therefore the ability to compare across contexts.

There is a lot that cannot be captured by league tables. Some will say, especially in education, that none of the real things that matter can be captured by league tables. There is as much truth in that as saying that a photograph in two dimensions cannot capture three dimensional reality.

So, while research output can be measured by the number of papers published, one cannot really use that to judge the quality of those papers. To manage that, one tries to restrict the set to journals that are of known and accepted quality, often peer reviewed. If the papers are accepted by these journals, then they will be included in research output, else not. There are a few problems here. As every academic writer knows, journals are often accessible to a certain clique who are the guardians of tradition (another proxy for quality, possibly). Each journal has its own style of analysis or writing. So a young bright academic finds that even good work has to conform in many ways, and if rejected, they must then recast it for another journal – and that means rewrite completely. If the academic – and in our example a bright young academic – is to be judged on the quantity of papers accepted, then it would be quite unfair to that person. On the other hand, those with access to the journals would be able to churn out many more articles than their research warranted. To create three papers out of one significant piece of research is not unknown.

Then if mere quantity is not enough, one should look for research impact – that is objectively measured in citations. Well, that is the best objectivity can do, little more. One could – and I do not know any league tables that do that – include citations included in papers that supported patent applications. If a paper has been in the citation chain for an actual innovation patented, then it has impact, else the research was merely theoretical.

Before the historians and philosophers start baying for my blood, let me step up and say – I agree! Research is meant to be theoretical. It is meant to add to the body of knowledge regardless of its current usability. There are enough examples in the sciences too that point to innovations that were used decades later. When they were created, they would not have added to the count of any league table, especially if they languished merely as working papers.

Similarly for teaching and learning. How can we measure for teaching quality? Discussed here. We can measure for teacher qualifications, for student achievement in standardised tests, but who can really say objectively that excellent or quality learning happens in the classrooms of this institution and not at another. How can they be graded and ranked? There is only so much league tables can do. There is no way the impact of learning can be measured across the lifetime of the learner. Nor can any league table ever measure the value of peer networks – though this is where it gets interesting. While there is no clear metric to peer networks, the universities that top the league tables are the ones with the most useful networks.

What the league tables cannot cover is the culture and ethos of an organisation.
No league tables can capture what the HBR case study on gender and class just discussed here

Does this mean we reject league tables straight out?

Of course not.

But when we use and quote league tables, especially in education we must do so with caution and cognition.

Firstly, do not take them too seriously. They are a snapshot and serve a particular purpose. They cannot serve all institutions and countries, but can merely indicate what is important to most. For example, India may not be ready to climb up the league tables yet – it may not be the priority for the nation as discussed here.  If India wants to make a place for itself on the league tables a priority, then it should focus on that, as described here

Newspaper and magazine league tables have to be a simple composite in order to be able to have a global span. But there are other ways of constructing comparative benchmarks that may be more productive depending upon the need and the purpose of that benchmarking exercise. All benchmarking exercises do not create rankings as league tables must. Quality can be tracked through simple yet sophisticated exercises that can easily be designed according to the need of the group or individual institutions. Till we invest in creating more customised benchmarks and trackers, we will have to make do with league tables.

Advertisements

India Continues to surprise in Global University Rankings

17 Oct

India continues to surprise in global university rankings

Meeta Sengupta
03 October 2013, 09:19 AM IST

India does manage to have some interesting things happening in the education manthan – the churning that traditionally threw up the treasures of the world.

And this time the crown goes to Panjab University that has managed to rise above all other Indian higher education establishments and is ranked between 226 and 250 in the world. Still low, for a country as educated and populous as India, but a significant achievement nonetheless. Panjab University has been known for its commitment to its faculty and academic standards, and it is good to see acknowledgement of that effort in global rankings.

Of the three or four major global higher education rankings, the Times Higher Education rankings are the most comprehensive and the most popular. With heavyweight analytics by Thomson Reuters and the might of the Times group in publication, the reach and rigour of the rankings is unparalleled. The strength is in the methodology that is revised and updated and here is a link to the criteria the university managed to score more than any other Indian university http://www.timeshighereducation.co.uk/world-university-rankings/2013-14/world-ranking/region/asia/methodology The rankings have noted the global shift from the West to the East in the higher ranks. This is a steady progression seen over the past few years and the younger universities of the east have worked hard to make themselves known in the upper echelons of the rankings.

Can this be managed? Can one climb up the rankings at will? Of course this is not an easy process that can be accomplished overnight. But like any assessment system one can work towards what is being assessed, work harder at those aspects and move up league tables. This is not to say a system is rigged – it is not. It is merely recognising and adopting a set of criteria seen as essential to defining a good university. Can Indian universities do this? I had written about this before in another publication and it is linked here ( http://www.dnaindia.com/analysis/1876684/standpoint-what-will-it-take-to-get-indian-universities-into-the-league-tables) This is of course a pragmatic and narrow view of progress at any university. All good universities will do this and more, depending upon their context, needs and goals.

How did Panjab University do it? We will have to ask them of course, but they seem to have done exceedingly well in the citation indices. They clearly focused on the criteria and were able to rank higher than the prestigious IITs too.  (May I please take a self indulgent I-told-you-so moment here?) Here is what Panjab University’s scores look like on the Times Higher Education page – http://www.timeshighereducation.co.uk/world-university-rankings/2013-14/world-ranking/institution/panjab-university While the University scores under 30 points for each of the criteria for teaching, international outlook and industry income, the scores are very interesting for their research and citations. Panjab University scores a mere 14 points for their research, but a colossal 84.7 for their citations ( 30% weightage) pushing them neatly into the middle leagues – to bring their overall rank to between 226 and 250. This is a leap forward for India that struggled to get mentions even in the two hundreds. Punjab University did receive a grant of almost Rs. 35 crores for investment in research infrastructure in acknowledgement of its good work in quality research and research papers. It was among the top three universities along with Delhi University and Hyderabad University, and amongst 14 to receive such appreciation.

Does that make Punjab University the best University in India? Today, it does. They deserve congratulations for their achievements. As with everyone who achieves leadership, we now wait for them to reveal their secrets to their peers, in the spirit of collegiality that will take Indian higher education forward – for this has to be the real goal. It is not enough for one institution to do well. Each institution will do different things well, and it is only when we share and embed best practices across higher education institutions that the students and academia will gain.

 

This was published in Times of India blogs on October 3, 2013 and is linked here http://blogs.timesofindia.indiatimes.com/educable/entry/india-continues-to-surprise-in-global-university-rankings