The influential European Union Committee of the British House of
Lords (upper house of Parliament) created the report "The Modernisation of Higher Education
in Europe" in which the U-Multirank project was criticized heavily.
EU money should be better spent on other priorities given that the
Times Higher Education and Shanghai rankings already did a good job
in providing "transparency and flexibility for students to make an
informed choice". News that Times Higher Education was quick to catch up
on.
The House of Lords raised concerns that the "league tables
market was already too crowded, with each ranking deploying its own
methodologies; that it would confuse applicants and be incapable of
responding to rapidly changing circumstances in institutional
profiles; that it could become a 'blunt instrument' which would
'not allow different strengths across diverse institutions to be
recognised and utilised' and end up being used as the basis for
future funding decisions; on the grounds of quality, accuracy and
lack of data; and that EU funds could be better spent on other EU
priorities."
Concerns over U-Multirank becoming official
reference
Given the doubts the House of Lords has regarding the added
value of U-Multirank, the report also urged the Commission to not
declare this new instrument as the official reference for European
universities: "The Higher Education Funding Council for England
(HEFCE) acknowledged the Commission's efforts to overcome some of
the limitations of traditional league tables and to render it more
objective but advised 'caution in providing any form of official
sanction to any one form of ranking tool given that universal
ranking systems have a history of lacking real comparability and
robustness'."
Instead, the House members pointed towards the merits of
existing rankings. The Times Higher Education Ranking was mentioned
in this context as a role model given that it recently revised its
methodology. "While Times Higher Education considered that rankings
were "relatively crude" and could never be properly
objective, they nevertheless considered that if used appropriately
they could still provide a useful role in providing
information."
Putin disappointed by Western rankings
Especially with the last assessment Russia's current Prime
Minister and President-elect Vladimir Putin would disagree. "You
must know that certain experts think that these Western ratings
are, in fact, an instrument for raising the competitiveness [of
their graduates] on the labor market," he remarked.
This came as a reaction to the recent outrage by Russian media
that none of the country's universities are ranked in THE's top
100. Now, Putin instructed the Russian Education Oversight Agency
to create an alternative ranking.
Complete report on U-Multirank
50.Current rankings-including the Times Higher Education World
University Rankings and Shanghai Jiaotong University's Academic
Ranking of World Universities-mainly focus on research-intensive
universities and only include a small proportion of European
universities. The Commission therefore believes that a wider range
of indicators and information should be made available to increase
transparency and allow more informed choices to be made, as well as
supporting policy-makers' higher education reforms. In response,
the Commission intends to launch U-Multirank in 2013, which
will allow users to profile universities using a number of
performance indicators rather than just research output.
51.Most of our witnesses were not convinced by the merits of yet
another league table, with the British Council description of
rankings as both a "blessing and a curse" capturing this
dichotomy well. The Russell Group told us that "ranking
universities is fraught with difficulties and we have many concerns
about the accuracy of any ranking. It is very difficult to capture
fully inumerical terms the performance of universities and their
contribution to knowledge, to the world economy and to society.
Making meaningful comparisons of universities both within, and
across, national borders is a tough and complex challenge, not
least because of issues relating to the robustness and
comparability of data". The EUA were critical of existing ranking
systems as favouring very large research intensive
institutions and while they praised the proposal's attempt to move
away from research outputs to look at other indicators, they
considered that this would be hard to achieve in practice,
particularly due to the lack of data in some universities and
Member States and the difficulties in collecting data more
generally, including the additional burdens that this may place on
universities.
52.Many of our other witnesses raised a series of concerns:
about the proposal's lack of clarity as to whether it would be a
ranking or transparency tool; that the league tables market was
already too crowded, with each ranking deploying its own
methodologies; that it would confuse applicants and be incapable of
responding to rapidly changing circumstances in institutional
profiles; that it could become a "blunt instrument" which would
"not allow different strengths across diverse institutions to be
recognised and utilised" and end up being used as the basis for
future funding decisions; on the grounds of quality, accuracy and
lack of data; and that EU funds could be better spent on other EU
priorities.
53.Notwithstanding these concerns, if the Commission's stated
intention of increasing transparency and providing more flexibility
for students to make an informed choice based on different criteria
proved to be possible, then many of our witnesses were prepared to
support its introduction as potentially adding value. The UK
Bologna Experts were of the same view but considered that
U-Multirank's success was "highly dependent on the extent of
institutional engagement, coverage, and accuracy of data used to
compile the rankings" and that it was "vital that the
instrument recognises the diverse character of European HEIs in so
far as direct comparisons can be iniquitous and misleading".
54.The Higher Education Funding Council for England (HEFCE)
acknowledged the Commission's efforts to overcome some of the
limitations of traditional league tables and to render it more
objective but advised "caution in providing any form of official
sanction to any one form of ranking tool given that universal
ranking systems have a history of lacking real comparability and
robustness". The NUS also welcomed the Commission's efforts but
still had doubts about how it would work in practice, believing
instead that improving the public information made available to
students could be achieved by alternative means "without the
need to introduce (yet another) potentially subjective and
confusing rankings system". The Government considered that
"it might be useful" if it genuinely provided a transparent
source of information for students wanting to study abroad but were
not convinced that it would add value if it simply resulted in an
additional European ranking system alongside the existing
international ranking systems. However, the Minister struck a less
positive tone when he told us that it could be viewed as "an
attempt by the EU Commission to fix a set of rankings in which
[European universities] do better than [they] appear to do in the
conventional rankings".
55.We were interested to note that THES have recently revised
their global rankings in 2010 in order to apply a different
methodology and include a wider range of performance indicators (up
from 6 to 13). They told us that their approach seeks to achieve
more objectivity by capturing the full range of a global
university's activities-research, teaching, knowledge transfer and
internationalisation-and allows users to rank institutions
(including 178 in Europe) against five separate criteria: teaching
(the learning environment rather than quality); international
outlook (staff, students and research); industry income
(innovation); research (volume income and reputation); and
citations (research influence). In order to inform the revision of
their rankings, their data supplier, Thomson Reuters, conducted a
global survey which found that many users distrusted the
methodology of the existing world rankings. While THES considered
that rankings were "relatively crude" and could never
be properly objective, they nevertheless considered that if used
appropriately they could still provide a useful role in providing
information.
56.We also believe that the provision of clear information and
guidance to students is important in order to assist them in making
an informed choice of university. However, we also
appreciate how difficult it can be to evaluate a wider range of
university performance indicators in an objective manner, noting
the limitations inherent in many of the existing ranking
systems.
57.Therefore, it is important that the Commission is clear about
the purpose of U-Multirank, what information will be provided and
what methodology will be used. If the perceived deficiencies in
most other ranking systems are overcome in relation to this
proposal then we could be convinced of the benefits of its
introduction. However, until these deficiencies can be overcome, we
consider that the Commission should prioritise other activities. In
the meantime, rankings such as the Times Higher Education World
University Rankings may have a valuable contribution to make.