University rankings – a perspective from one of their founding fathers

Report Post

John O’Leary is a world-leading British education journalist and consultant, and a pioneer of university rankings. Together with Nunzio Quacquarelli, director of QS, he oversaw the introduction of the QS World University Rankings® in 2004. Tony Martin asked John how his background and professional ethos brought him to innovate and publish in such a vital area of global higher education.

As an expert in world higher education, what was your personal university experience and how did your degree prepare you for your career?

I was the first person in my immediate family to go to university, apart from an uncle in Ireland who had to get permission from the parish priest to go to Trinity College Dublin. He ended up as chief executive of Ireland’s Higher Education Authority. In 1970 in the UK, only one 18-year-old in 12 went to university, so it was not surprising that my parents hadn’t been, and it meant there wasn’t much advice available on where to go. Most of mine came from my headmaster, who steered me towards politics because I wanted to be a journalist.

I went to Sheffield University, where leading political theorist Bernard Crick was the professor of politics, and future UK government minister David Blunkett was a student.

I enjoyed the degree but spent more time on the student newspaper and the students’ union. I stayed on for a year after graduating to be president of the union, which was a great experience and, through the UK’s National Union of Students, introduced me to a lot of future Labour Party politicians, some of whom were among my best contacts as a journalist. I’ve recently finished a six-year term as trustee of the Sheffield University students’ union.

What took you into journalism and what were your early journalistic experiences and subsequent progression in the profession?

I started my career as a graduate trainee at the Evening Chronicle in Newcastle-upon-Tyne. University graduates were a small minority in journalism at the time and were treated with great scepticism. After a couple of years as a general reporter, I was given the education ‘beat’, mostly writing about schools but also Newcastle’s university and polytechnic (now Northumbria University). That led fairly naturally to applying to what was then The Times Higher Education Supplement, which I joined as a reporter in 1978. Within six months, the paper (part of The Times group) had closed for a year over industrial action by the print unions. When it reopened I stayed for another 10 years, becoming foreign editor, news editor and then deputy editor.

What inspired you to rank UK Universities by publishing The Times Good Universities Guide and what was their impact and development?

In 1990, I moved to The Times newspaper as higher education correspondent and became education editor in 1991. By then, the first high school league tables had been published in the UK and the government had announced that polytechnics would become universities. With Tom Cannon, a former director of Manchester Business School, I set about producing rankings for UK universities.

We used a combination of published statistics and questionnaire returns, and produced a ranking based on 14 measures, from entry grades and staff qualifications to graduate employment levels and the proportion of international students. Oxford and Cambridge tied for top place in the first edition. We published these rankings in the first edition of The Times Good Universities Guide.

The universities were largely furious and described the ranking as worthless, putting pressure on the newspaper to drop them. But the Guide was extremely popular with readers and the circulation of the paper went up a lot on the day of publication, so it survived – and still does 25 years later, albeit with only five of the original ranking measures. It now relies entirely on published statistics and has a review group of academic planners who monitor its methodology and make suggestions on behalf of the universities.

John O’Leary (second right) presents the first copy of the 2015 QS Asian rankings to the then President of India, Pranab Mukherjee (second left) in New Delhi.

What prompted the launch of world university rankings with QS in 2004?  Why was this the right time?

By the early 2000s, some of the leading UK universities, such as Manchester and Bristol, were beginning to judge themselves against international competitors, so I was keen to have an international element to the Times guide. With relatively few statistics available internationally, I felt this would require a reputational element. I knew Nunzio Quacquarelli, who had founded QS and was publishing the MBA Career Guide, which included surveys of employers and business schools. The Times published occasional supplements on MBAs and Nunzio often contributed articles, drawing on these surveys, so we discussed whether QS could do the same for entire universities.

Before those discussions had borne fruit I had become editor of The Times Higher Education Supplement (THES) where the management was eager to pick up the project. Nunzio and I, with Ben Sowter of QS and Martin Ince, my deputy at THES, had a number of discussions about possible methodologies, and eventually agreed on one similar to today’s QS World University Rankings®. We were not the only ones having such thoughts and the Shanghai rankings (Academic Ranking of World Universities) actually came out first. They were less broad-based than ours, focusing almost exclusively on scientific research.

We wanted to sample opinion among academics and employers, as well as making use of the available statistics on citations and staffing levels. We also decided that a global ranking should take account of the scale of universities’ international activities, so we included the proportion of international students and academic staff, although only with a low weighting (of 5 per cent each).

In November 2004, I wrote in the edition of THES containing the first rankings that this was a first attempt to compare universities internationally and I was sure our (predominantly academic) readership would come up with plenty of other measures. There have been a lot of suggestions for what should be measured – mainly teaching quality and universities’ contributions to their communities and/or economies – but nothing practical that we thought would be an improvement. QS had given a lot of thought to including teaching measures, before publication and since, but has never come up with a workable solution. In my view, neither has anyone else. It is undoubtedly a flaw in rankings that are designed primarily to guide prospective students who have a vested interest in teaching quality, but, equally, to publish an unsound measure would be misleading.

Since 2004, what has changed with world university rankings? What are their main impacts on both student choice and on the universities of the world?

I parted company with THES in 2007 and have since continued to work with QS on a consultancy basis and as a member of the executive board responsible for the QS range of university rankings. Other international rankings have appeared. Times Higher Education (THE), as THES became, eventually published its own international ranking, as did US News and World Report, and several other organisations.

Global rankings, particularly those from the main players – QS, THE and Shanghai – have become much more influential that we ever expected, or wanted. I regard rankings as a signpost for students to help their own researches, rather than dominating the decision-making process, and certainly not as something that should shape a university’s strategy. That said, given the scale of international study in recent years, I think they have provided valuable information for students, who need help to put universities’ increasingly sophisticated marketing in context. There has, naturally, been plenty of criticism of the QS and other rankings, but little to suggest what would take their place if they ceased to exist.

“Overall, I am proud of the way rankings have developed and I would love to see the day that they include some element of teaching quality – but a reliable measure remains some way off at the moment.”

What are your personal wishes for the future contribution of rankings both to society and to higher education around the world?

It does concern me that rankings can be too influential, although not particularly with students, who I think probably take them with a healthy pinch of salt in most parts of the world. Surveys, for example by QS Solutions, suggest that students take most notice of the subject rankings and are not slaves to the broad institutional results. It is the overreaction of some governments that worries me most, and the knock-on effect on universities. Unrealistic targets put pressure on universities to depart from missions that might, quite reasonably, prioritise teaching or community service over research. But the rankings are now compiled in a more sophisticated way than in their early years and are often used by universities to help select international partners.

Overall, I am proud of the way rankings have developed and I would love to see the day that they include some element of teaching quality – but a reliable measure still remains some way off at the moment.

Sheffield University graduate, John O’Leary was education editor of The Times newspaper from 1991 to 2002 and editor of The Times Higher Education Supplement from 2002 to 2007. He launched, and continues to edit The Times and Sunday Times Good University Guide, which introduced the first university rankings to the UK, and he is a founding member of the executive board responsible for the QS World University Rankings®. Today, John writes regularly on education for The Times newspaper and Education journal, and is a frequent speaker at international conferences. He is a member of the UK’s Higher Education Commission and is the author of Higher Education in England, published by the Higher Education Funding Council for England. In 2011, John won the UK’s Ted Wragg Award for a sustained contribution to education journalism.