I've now read the whole report. Yes, it tilts at the
usual windmills, and rightly dismissed the use of Impact
factors for anything but crude comparisons, but it fails
to address the fundamental issue, which is: citation and
other metrics correlate superbly with subjective peer
review. Both methods have their faults, but they are
clearly measuring the same (or closely related) things.
Ergo, if you have evaluate research in some way, there is
no reason NOT to use them! It also keeps referring to
examples from the field of maths, which is a very strange
subject citation-wise.
Charles
On Wed, 11 Jun 2008 18:05:36 +0200
"Armbruster, Chris" <Chris.Armbruster_at_EUI.EU> wrote:
> It is true that Thomson is misspelled as Thompson, but
> it is so consistently. It also the case that the Leiden
> stalwarts A.J.F. van Raan (wide body of work on
> performance measurement, university ranking etc.) and
> H.F. Moed (Book: Citation analysis in research
> evaluation) are not cited.
>
> Nevertheless, after reading the report, I would caution
> against dismissing it. Science and scientists should be
> concerned about the politicisation of metrics.
> Politicisation comes from governments and research
> funders but is also going on inside academic
> institutions. Moreover, in a general sense the citation
> and usage metrics currently available are not 'fit for
> purpose'. Worse still, politicisation carries with it the
> significant risk of arresting the development of tools
> for metric research evaluation. Evaluation is often
> narrowly defined as assessment and performance of
> institutions and indivudals for the purpose of awarding
> or denying funding and employment. This is something
> entirely different from metric evaluation as research
> information service to aid scientists in reducing the
> complexity of scientific information in their daily
> research.
>
> All we have at the moment are some 'quick fix metrics'.
> And these are increasingly used to make and legitimate
> all kinds of decisions. It is thus welcome that
> mathematicians and statisticians scrutinise current
> practices and show up the lack of validity and
> reliability of many measures, technical faults as well as
> the misguided judgements of peers, university management,
> funding agencies and government.
>
> My own contribution (working paper) may be found with
> SSRN:
> Armbruster, Chris, "Access, Usage and Citation Metrics:
> What Function for Digital Libraries and Repositories in
> Research Evaluation?" (January 29, 2008).
> Available at SSRN: http://ssrn.com/abstract=1088453
>
> If the link is broken, please use a search engine *SSRN
> plus title*
>
> Chris Armbruster
>
> -----Original Message-----
> From: American Scientist Open Access Forum on behalf of
> C.Oppenheim_at_lboro.ac.uk
> Sent: Wed 11/06/2008 14:56
> To:
> AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG
> Subject: Re: Citation statistics
>
> I haven't had a chance to read the report yet, but I'd
> be suspicious of any report that fails to spell "Thomson"
> correctly and fails to cite Ton van Raan, THE expert on
> the subject.
>
> Charles
>
>
> Professor Charles Oppenheim
> Head
> Department of Information Science
> Loughborough University
> Loughborough
> Leics LE11 3TU
>
> Tel 01509-223065
> Fax 01509 223053
> e mail c.oppenheim_at_lboro.ac.uk
> -----Original Message-----
> From: American Scientist Open Access Forum
> [mailto:AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG]
> On Behalf Of Jean Kempf
> Sent: 11 June 2008 12:01
> To:
> AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG
> Subject: Citation statistics
>
> Here's a report on citation statistics written by a
> statistician
>
> http://www.mathunion.org/Publications/Report/CitationStatistics
>
> A press release that was mailed out today to journalists
> is at:
>
> http://www.mathunion.org/Publications/PressRelease/2008-06-11/CitationStat
> istics
Received on Fri Jun 13 2008 - 01:01:45 BST