Re: Future UK RAEs to be Metrics-Based

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Tue, 19 Sep 2006 01:07:56 +0100

Fytton Rowland is absolutely right. It would be *extremely* bad and
extremely foolish if the current RAE-reform juggernaut continued to veer
mindlessly toward prior-funding as the mother-of-all-metrics. I hope
clearer heads will prevail, and that the RAE will use its wise decision
to scrap panel review in favour of metrics in order to develop a rich
set of candidate metrics, adapted to each discipline, rather than to
abandon measurement and prediction altogether for and opt instead for
self-fulfilling prophecy and the Matthew Effect that would result from
relying solely or mainly on the prior-funding metric.

Stevan Harnad

On Tue, 19 Sep 2006, Fytton Rowland wrote:

> All the exchanges in this thread over the last day or so seem to be assuming
> that the 'metrics' to be used in future RAEs in the UK will measure
> citations or some other similar measure of use of published papers. If
> that were so, I would not be too unhappy - Charles Oppenheim's earlier
> research has shown that such measures correlate quite well with past RAE
> results achieved by much more labour-intnsive and expensive methods.
>
> But if we look at the actual proposals from the funding councils that are
> currently out for consultation, we find that their idea of 'metrics' is
> something quite different. Essentially their proposals give primacy to a
> measure of money earned from outside research grants. This is bad in two
> important ways. First, it measures resaerch inputs not outputs, and
> potentially rewards inefficiency - surely those who achieve good research
> results on a shoestring should be admired, not penalised? Second, it
> removes diversity of research funding - essentially, the only way to build
> a research-active departmenbt is to get money from research councils, etc.
> The HEFCE money would simply apply a multiplier to that from research
> funders. The current RAE system, or a hypothetical one based on citation,
> offers an alternative route ro research success for those who do good work
> without outside funding; if their outputs (published papers and books) are
> adjudged good and influential, they receive QR funding independently of any
> outside research grants they may have. The proposed money-based metrics
> offer no alternative means of support, but simply apply 'to him that hath,
> it shall be given'.
>
> So, for me, metrics in principle are not necessarily bad but the actual
> metrics being proposed are very bad indeed.
>
> Fytton Rowland, Department of Information Science, Loughborough University,
> UK.
>
Received on Tue Sep 19 2006 - 01:56:07 BST