On Tue, 28 Mar 2006, Loet Leydesdorff wrote:
>> SH: The multiple regression ("metric") method is not yet in use at all.
>> It will now be tried out, in parallel with the next RAE (2008), which
>> will be conducted in the usual way, but doing the metrics alongside.
http://www.hm-treasury.gov.uk/media/1E1/5E/bud06_science_332.pdf
> Interesting! Do you expect the multiple regression model to correlate highly
> with the RAE? And if it will deviate, can you specify why?
It is already *known* that priori funding (which is explicitly counted)
correlates very highly (.98), and citations (which are not explicitly
counted) correlate highly (.7-.9) with the current RAE outcomes. So grants
alone would predict all but 4% of the variance in the RAE rankings in
many fields already. That is why the costly, time-consuming non-metric
exercise is being dropped, and replaced entirely by metrics.
But remember that the RAE itself is trying to measure/predict something,
namely, research performance. And that prediction can certainly stand
to be optimised. It is not optimised by simply relying on measures that
duplicate the current RAE outcome, such as it is. Metrics should enrich
and augment it. I cannot predict which way they will go in specific
cases: I expect that the components and weightings of the metric
multiple regression equation will vary from discipline to discipline,
and will need to be calibrated. Some disciplines are more research-grant
intensive than others, some journal-intensive, some book-intensive,
some short-latency-impact-based, some long-: Parameters will have to be
tweaked, and cross-validated against cross-years predictivity as well as
peer ratings and feedback. And the full-text OA corpus will be a rapidly
growing database, there to explore and data-mine for still further
candidate metrics.
Whilst saving a great deal of time and money, the new metric RAE can
only increase its richness, sensitivity and diversity, compared to the
old, bloated one.
Stevan Harnad
American Scientist Open Access Forum
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
Chaire de recherche du Canada Professor of Cognitive Science
Ctr. de neuroscience de la cognition Dpt. Electronics & Computer Science
Université du Québec à Montréal University of Southampton
Montréal, Québec Highfield, Southampton
Canada H3C 3P8 SO17 1BJ United Kingdom
http://www.crsc.uqam.ca/ http://www.ecs.soton.ac.uk/~harnad/
Received on Tue Mar 28 2006 - 16:56:33 BST