Comment on:
UK plans research funding overhaul
by Stephen Pinfield
The Scientist, Wednesday 21 June, 2006
http://www.the-scientist.com/news/display/23683/
-----------------------------------------------------------------------
Let 1000 RAE Metric Flowers Bloom:
Avoid Matthew Effect as Self-Fulfilling Prophecy
by Stevan Harnad
The conversion of the UK Research Assessment Exercise (RAE) from the
present costly, wasteful exercise to time-saving and cost-efficient
metrics is welcome and overdue, but the worrying thing is that
the RAE planners currently seem to be focused on just one metric --
prior research funding -- instead of the full and rich spectrum of new
(and old) metrics that will become available in an Open Access world,
with all the research performance data digitally available online
for analysis and use.
Mechanically basing the future RAE rankings exclusively on prior funding
would just generate a Matthew Effect (making the rich richer and the
poor poorer), a self-fulfilling prophecy that is simply equivalent to
increasing the amount given to those who were previously funded (and
scrapping the RAE altogether, as a separate, semi-independent performance
evaluator and funding source).
What the RAE *should* be planning to do is to look at weighted
combinations of all available research performance metrics -- including
the many that are correlated, but not so tightly correlated, with prior
RAE rankings, such as author/article/book citation counts, article
download counts, co-citations (co-cited with and co-cited by, weighted
with the citation weight of the co-citer/co-citee), endogamy/exogamy
metrics (citations by self or collaborators versus others, within
and across disciplines), hub/authority counts (in-cites and out-cites,
weighted recursively by the citation's own in-cite and out-cite counts),
download and citation growth rates, semantic-web correlates, etc.
It would be both arbitrary and absurd to blunt the sensitivity, power,
predictivity and validity of metrics a priori by biasing them toward the
prior-funding counts metric alone, which should just be one out of a full
battery of weighted metrics, adjusted to each discipline and validated
against one another (and against human judgment too).
Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006) The Open
Research Web: A Preview of the Optimal and the Inevitable, in Jacobs,
N., Eds. Open Access: Key Strategic, Technical and Economic Aspects,
chapter 21. Chandos.
http://eprints.ecs.soton.ac.uk/12369/
------------------------------------------------------------------------
Stevan Harnad
AMERICAN SCIENTIST OPEN ACCESS FORUM:
A complete Hypermail archive of the ongoing discussion of providing
open access to the peer-reviewed research literature online (1998-2005)
is available at:
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/
To join or leave the Forum or change your subscription address:
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
Post discussion to:
american-scientist-open-access-forum_at_amsci.org
UNIVERSITIES: If you have adopted or plan to adopt an institutional
policy of providing Open Access to your own research article output,
please describe your policy at:
http://www.eprints.org/signup/sign.php
UNIFIED DUAL OPEN-ACCESS-PROVISION POLICY:
BOAI-1 ("green"): Publish your article in a suitable toll-access journal
http://romeo.eprints.org/
OR
BOAI-2 ("gold"): Publish your article in a open-access journal if/when
a suitable one exists.
http://www.doaj.org/
AND
in BOTH cases self-archive a supplementary version of your article
in your institutional repository.
http://www.eprints.org/self-faq/
http://archives.eprints.org/
http://openaccess.eprints.org/
Received on Wed Jun 21 2006 - 12:22:32 BST