Re: UK Research Assessment Exercise (RAE) review

From: David Goodman <dgoodman_at_PHOENIX.PRINCETON.EDU>
Date: Thu, 28 Nov 2002 18:59:48 -0500

Jan, I want to comment on one point only:

On Thu, 28 Nov 2002, Jan Velterop wrote:
>
> Close analysis of the track record of many journals shows an enormous
> variability in rates of citation for the articles published in them. If
> my journal publishes mostly 'landfill' science, but I manage to
> persuade a few brilliant review articles (for instance by paying the
> review-author generously), I can secure a reasonable impact factor, the
> common measure of a journal's track record. This is not hypothesis, but
> widespread reality. Secondary evaluation brings this to the fore, and
> that's why secondary evaluation, in the manner of for instance Faculty
> of 1000, is so important.

The analysis of this, and the corrections for it, do not only rely on
non-quantitative reviews, but are accessible to
current bibliometric techniques -- the help pages for JCR even explain the
technique for removing review articles from the count.

Another interesting measure, which ISI currently supplies only on a custom
basis, but can certainly be calculated independently by anyone willing to
do the work, is the proportion of articles with 0, 1, 2, ..., n
citations.

I also need to emphasize that I regard Jan's work highly, and agree that
human judgements are a useful check on quantitative techniques as well as
a supplement to them. And of course reviews are useful in their own
right--if I didn't think so I would not have been reviewing reference
books and databases for CHOICE for the last 15 years or so.

 Dr. David Goodman
Princeton University Library
and
Palmer School of Library & Information Science

dgoodman_at_princeton.edu
Received on Thu Nov 28 2002 - 23:59:48 GMT