Re: Future UK RAEs to be Metrics-Based

From: <l.hurtado_at_ED.AC.UK>
Date: Mon, 18 Sep 2006 17:41:31 +0100

Steven and I have exchanged views on the *feasibility* of a metrics
approach to assessing research strength in the Humanities, and he's
impressed me that something such *might well* be feasible *when/if*
certain as-yet untested and undeveloped things fall into place. I note,
e.g., in Steven's addendum to Openheim's comment that a way of handling
book-based disciplines "has not yet been looked at", and that a number
of other matters are as yet "untested".
This being the case, it is certainly not so a priori to say that a
metrics approach is not now really feasible for some disciplines.

I emphasize that my point is not a philosophical one, but strictly
whether as yet a worked out scheme for handling all Humanities
disciplines rightly is in place, or capable of being mounted without
some significant further developments, or even thought out adequately.
That's not an antagonistic question, simply someone asking for the
basis for the evangelistic stance of Steven and some others.
Larry Hurtado

> Charles Oppenheim has authorised me to post this on his behalf:
>
> "Research I have done indicates that the same correlations between
> RAE scores and citation counts already noted in the sciences and
> social sciences apply just as strongly (sometimes more strongly)
> in the humanities! But you are right, Richard, that metrics are
> PERCEIVED to be inappropriate for the humanities and a lot of
> educating is needed on this topic."
>
> Professor Charles Oppenheim
> Head
> Department of Information Science
> Loughborough University
>
> I would also add the following to what Charles has said:
>
> (1) Wherever anyone has checked the correlation between journal citation
> counts and RAE outcome, the correlation has always been significant
> and sizeable, hence predictive. Humanities disciplines have not been
> exceptions.
>
> (2) In book-based fields, what has likewise not been looked at is
> supplementing the journal-article citation metric with a book-citation
> metric. (That's still metrics!)
>
> (3) And then there are all the other candidate metrics, most still
> untested: downloads, co-citations, hubs/authorities, recursive CiteRank,
> download/citation growth parameters (latency, slope, peak, longevity),
> semantic metrics, etc.
>
> An a-priori declaration, free of any supporting evidence -- by any
> discipline today -- that its work is an exception, not assessable by
> metrics, makes about as much sense as an a-priori declaration, without
> any supporting evidence, that a discipline's work is not assessable by
> any form of comparative performance evaluation at all. (Who's to say whether
> subjective evaluations have any validity either?)
>
> Stevan Harnad
>



L. W. Hurtado, Professor of New Testament Language, Literature & Theology
Director of Postgraduate Studies
School of Divinity, New College
University of Edinburgh
Mound Place
Edinburgh, UK. EH1 2LX
Office Phone: (0)131 650 8920. FAX: (0)131 650 7952
Received on Mon Sep 18 2006 - 19:20:59 BST