Pertinent Prior AmSci Topic Threads:
"Future UK RAEs to be Metrics-Based"
http://users.ecs.soton.ac.uk/harnad/Hypermail/Amsci/5250.html
"Australia's RQF"
http://users.ecs.soton.ac.uk/harnad/Hypermail/Amsci/5805.html
"Academics strike back at spurious rankings" (Nature, 31 May)
http://users.ecs.soton.ac.uk/harnad/Hypermail/Amsci/6452.html
On Tue, 19 Jun 2007, Arthur Sale wrote:
> The Australian Opposition has announced that should they win office in a
> federal election to be held later this year, they will scrap the
> Government's planned RQF process (based on peer panels and the UK's RAE),
> and replace it by a metric-based quality assessment.
>
> "The RQF process is cumbersome, costly and threatens to become
> incredibly time-consuming. It is neither an efficient nor a
> transparent way to allocate valuable research dollars to universities.
> "Labor will work hand in hand with researchers, and their
> institutions, to develop a research quality assurance framework
> that is world's best practice. It will be metrics based. It will
> be transparent. It will take due account of differences between
> disciplines and discipline groups so that measures are fair, and
> funding can flow equitably."
Expensive, time-consuming panel-reviews of research performance should
certainly be phased out in favour of metrics, customised field by field.
So an important question is: Against what will Australia's metrics
be validated?
The UK RAE is conducting one last *parallel* panel/metric exercise, in
which various combinations of metrics can be systematically compared to
and validated against panel rankings, field by field.
Will the UK's RAE 2008 parallel exercise be Australia's testbed for
metrics too? That might not be a bad idea. Though with plans for
Australia's RQF already quite advanced, it might be even better to
do a parallel panel/metric validation exercise in Australia too, to
replicate and cross-validate the UK outcomes (perhaps in collaboration
or coordination with the UK).
(I certainly don't mean that panel rankings are the face-valid arbiters
of research performance quality or impact! But they do have a history,
at least in the UK, and so they provide a starting reference point.
Human judgment will also be needed to tweak the metric weights to make
sure they generate sensible rankings.)
Unbiassed Open Access Metrics for the Research Assessment Exercise
http://openaccess.eprints.org/index.php?/archives/175-guid.html
Harnad, S. (2007) Open Access Scientometrics and the UK Research
Assessment Exercise. To appear in: Proceedings of 11th Annual Meeting
of the International Society for Scientometrics and Informetrics,
25-27 June 2007, Madrid, Spain.
http://eprints.ecs.soton.ac.uk/13804/
Stevan Harnad
Received on Wed Jun 20 2007 - 02:47:37 BST