If one assesses an institute's productivity by the papers from its
researchers, and one rates those papers with the help of journal impact
factors, is it not the case that one should expect the results to be in line
with the citation counts for those papers? Is it me or is there a circular
argument here?
Jan Velterop
-----Original Message-----
From: Peter Suber [mailto:peters_at_earlham.edu]
Sent: 25 November 2002 01:59
To: AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG
Subject: Re: UK Research Assessment Exercise (RAE) review
In the recent postings on RAE ratings and scientometrics, I don't believe
I've seen anyone cite this piece of research:
Andy Smith and Mike Eysenck, "The correlation between RAE ratings and
citation counts in psychology" (June 2002)
http://psyserver.pc.rhbnc.ac.uk/citations.pdf
The authors' summary: We counted the citations received in one year (1998)
by each staff member in each of 38 university psychology departments in the
United Kingdom. We then averaged these counts across individuals within
each department and correlated the averages with the Research Assessment
Exercise (RAE) grades awarded to the same departments in 1996 and 2001. The
correlations were extremely high (up to +0.91). This suggests that whatever
the merits and demerits of the RAE process and citation counting as methods
of evaluating research quality, the two approaches measure broadly the same
thing. Since citation counting is both more costeffective and more
transparent than the present system and gives similar results, there is a
prima facie case for incorporating citation counts into the process, either
alone or in conjunction with other measures. Some of the limitations of
citation counting are discussed and some methods for minimising these are
proposed. Many of the factors that dictate caution in judging individuals
by their citations tend to average out when whole departments are compared.
Peter
----------
Peter Suber, Professor of Philosophy
Earlham College, Richmond, Indiana, 47374
Email peters_at_earlham.edu
Web
http://www.earlham.edu/~peters
Editor, Free Online Scholarship Newsletter
http://www.earlham.edu/~peters/fos/
Editor, FOS News blog
http://www.earlham.edu/~peters/fos/fosblog.html
Received on Mon Nov 25 2002 - 09:31:55 GMT