Prior Amsci Topic Thread:
"How to compare research impact of toll- vs. open-access research"
(started June 2003)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/2858.html
On Mon, 3 Oct 2005, Sally Morris (ALPSP) wrote:
> The problem is, there is no evidence of correlation between citations and
> the return on research expenditure.
Citations *are* one measure of return on research expenditure. Research
is funded in order to be applied and built-upon, i.e., to be *used*;
citations are an index of that usage. Uncited, unused research may as
well not have been conducted, and represents *no* return on the research
investment. Whatever increases usage and citations, increases the return
on the research investment. Any loss of such a potential increase is a
loss of potential return on the research investment.
Self-archiving increases citations 50%-250%. Hence the failure to self-archive
loses 50%-250% of the potential return on the research investment.
> I haven't been able to trace many analyses which do look at this (Don
> King will know, if anyone does) but those I've read look at output of
> articles, registration of patents, and Gross Domestic Product.
Article counts are a measure of the return on the research investment,
but far too crude a measure, for, as noted, the articles may not be used.
Patents are pertinent only to a tiny portion of the research literature,
so have insufficient generality to be a useful general measure of research
impact. Moreover, they are often based on *unpublished* research,
whereas self-archiving and the OA movement are directed specifically
at published research. However, patent counts and citation counts are
in fact positively correlated:
"patent volume is positively correlated with paper citations,
suggesting that patent counts may be reasonable measures of research
impact"
Agarwal, A. & Henderson, R. (2002) Putting Patents in Context:
Exploring Knowledge Transfer from MIT. Management Science 48 (1), 44-60
https://dspace.mit.edu/retrieve/3871/IB_Putting%2BPatents.pdf
Gross domestic product is again too crude. Most basic research is too far from
practical applications to contribute to the GDP. But one thing is certain: If a
piece of research is to make a contribution to the GDP, it must be accessible to
its potential appliers. Self-archiving substantially increases accessibility, as
indicated by the fact that it generates substantially more citations.
I too would be interested, however, to know of studied correlating GDP with
citation counts.
> Clearly, we are a long way off being able to analyse whether or not
> self-archiving (or any other form of open access) does or does not
> contribute to these objective output measures.
I thought the question was about whether citation counts are correlated with these
measures. We already know that self-archiving is correlated with increased
citation counts.
> But to pretend that we 'know' citations are a proxy for any of them is
> not, to my mind, an argument that holds any water
The claim was not that citations are a proxy for GDP, but that citations
are a (face-valid) measure of the return on the investment of public
funds in research -- and, more particularly, that the loss of potential
citations is the loss of potential returns on the investment of public
funds in research (lost "value for money").
> Stevan, I know what you're going to say so please don't bother - frankly, I
> am more interested in hearing what other people have to say
Sally, I'd be pleased to obey your request not to reply to you, if this
were only a private conversation between you and me. But, you see, others
are involved too, in particular, researchers and their interests. You
appear to be concerned about hypothetical future losses to publishers
because of self-archiving -- losses for which there exists no evidence at
all to date. I am concerned about actual current losses to researchers for
not self-archiving -- losses for which the sizeable positive correlations
between self-archiving and citation counts, and between citations counts
and researcher revenue (in terms of both salary and research funding)
constitute strong positive evidence.
See, for example, the many studies showing the correlation between RAE rankings
and citation counts, as cited here:
http://www.ariadne.ac.uk/issue35/harnad/
In particular, Eysenck & Smith (2002) write:
"Correlation between RAE ratings and mean departmental citations +0.91
(1996) +0.86 (2001) (Psychology)"
"RAE and citation counting measure broadly the same thing"
http://cogprints.org/2749/
Stevan Harnad
Harnad, S. (2005) Maximising the Return on UK's Public Investment in Research.
http://eprints.ecs.soton.ac.uk/11220/
Received on Tue Oct 04 2005 - 04:22:55 BST