Re: How to compare research impact of toll- vs. open-access research

From: David Goodman <David.Goodman_at_LIU.EDU>
Date: Mon, 8 Sep 2003 22:20:47 -0400

I think Stevan and I are in basic agreement both about the areas in which bibliometric research is needed, and the practical measures to follow meanwhile.

I am also in agreement with him that if one is measuring purely research impact, citations are a more directly relevant figure in the typical scientific fields than total downloads (Provided that one takes account of the sometimes artificial patterns of citing behavior--which continues to be an actively studied question). But libraries --even research libraries-- support more things than the production of journal articles. In particular, they support higher education. True, in most fields of science all good theses are published (and I and others have studied and are studying the relationship between citations in theses and in the corresponding papers). But there is a good deal of preparatory work before the student reaches that point, on both the graduate and the undergraduate levels. As a rough estimate, about one-half the cost of a typical university library is attributed to research and half to education. The amount of staff and resources devoted to purposes that do not result in published papers may seem surprisingly large--but no one ever claimed that higher education is or can be an efficient activity.

One could argue that research can proceed in an environment free of any level of teaching below the postdoctoral. Such institutions are the exception. This may be a personal bias, but I think that faculty need graduate students--and ideally even undergraduates. Recall that Rockefeller and later CSHLQB did add graduate schools.

Total use, as measured by downloads/readings can be measured objectively, and techniques are developing for measurements that will be more precise than the current + or - 50 percent. For a detailed discussion of the current and evolving standards see www.projectcounter.org (yes, there is a personal interest to declare--I'm one of the executive committee, which is about half publishers and half librarians).

Doing arithmetic with figures from ISI may be tedious, but it is conceptually well understood. It is, of course, a classic mistake to assume without further analysis that what is easiest to measure is what is most relevant.
The quality of research cannot be fully understood by measuring parameters from the outside. The first book I assign my students is "Thinks..." by David Lodge.
> ! The *quality* of the weather can only be known by
> experiencing it directly, and that is what meteorology is trying to spare
> us the trouble of having to do for ourselves, in advance, in every
> case!)

------
The practical measure meanwhile, of course, is open access by any suitable means. The one thing we know does not work is the present system.
-----


Dr. David Goodman
Associate Professor
Palmer School of Library and Information Science
Long Island University
dgoodman_at_liu.edu

(and, formerly: Princeton University Library)
Received on Tue Sep 09 2003 - 03:20:47 BST