Unfortunately, too many distinct and independent factors are conflated
in this question about e-citations.
(1) Eprint Archives like CogPrints
http://cogprints.soton.ac.uk/
differ from Refereed Journals like Psycoloquy
http://www.ecs.soton.ac.uk/cgi/psyc/newpsy
Journals are the safeguards of a quality-controlled and certified
(peer-reviewed) literature. Archives are a place where the authors of
those peer-reviewed journal articles can self-archive them publicly
online, to make access to their research free.
(2) It is definitely true that, except for Physics (and some
Mathematics), all other disciplines have been slow off the mark with
self-archiving to date.
http://www.dlib.org/dlib/december99/12harnad.html
(3) What promotion and tenure committees at universities need is
objective measures of the research impact of their faculty's work.
Journal publication counts are one such measure, journal prestige and
impact factors are another, and article citations are another.
(4) No one yet knows yet what online hit-rate means, if anything.
Objective measures have to be validated by being correlated with the
criterion they are trying to measure (just as intelligence and aptitude
tests are).
(5) Citations have a face validity: It means that the reported research
is being used in further research (most published papers are not cited
at all).
(6) Hit-rates reflect other things besides research impact; it is not
even clear who is doing the hitting (with journal citation, it is
clearly publishing researchers who are doing the citing). Hit-rates may
reflect teaching use (not a research indicator) or some form of
"popularity" unrelated to research.
(7) As noted, unlike citations in publications, which are refereed,
"hit-rates" can easily be abused if they begin to be used as career
indicators. There are also no doubt ways to control for this, but first
of all, the hit-rates need to be validated.
(8) Hit-rates (and other usage indicators, such as citation navigation
trails will be increasingly used in monitoring and sign-posting both
online journal use and online archiving.
http://opcit.eprints.org/)
(9) Far more important than enhancing these not-yet-validated
hit-rates, open self-archiving online will enhance citation rates
themselves, because of the increased access and exposure, and freedom
from access tolls, thereby increasing the impact of research directly.
(10) Online journals will be increasingly covered by the classical
indexing services. (I am about to announce that Psycoloquy will now
begin to be indexed by both the Institute of Scientific Information
(ISI) Science Citation Index, and by the American psychological
Association's PsycINFO.)
(11) Open Archives will evolve their own forms of indexing and alerting
services, made possible by the interoperability guaranteed by
compliance with the Santa Fe Convention
http://www.openarchives.org/
(12) In my opinion, what will facilitate and accelerate self-archiving
will be the availability of generic, Santa-Fe compliant archiving
software that any University can mount to establish its own eprint
archive for all of its researchers
http://www.eprints.org/index.html
On Sat, 22 Apr 2000, Dr. John R. Skoyles wrote:
> Cogprints is at best only most moderately successful. Maybe, it is because
> it lacks that key component still restricted to orthodox paper journals --
> citation rankings.
No. All the refereed articles in CogPrints have citation rankings. What
authors have not yet picked up on -- but they will -- is how open
archiving will enhance those citation rankings.
> for the key measure of a paper -- its visitations. Counters...
These will come (though hit-rates will not -- and should not -- be given
the same prominence as citation rates until and unless it is shown what
they measure). But it is not such vanity-indicators that will tilt the
balance toward self-archiving -- which is, I hasten to remind everyone,
a SUPPLEMENT not a SUBSTITUTE for refereed journal publication -- but
rather the enhanced research impact provided by the incomparably
enhanced visibility and access of refereed papers that are also
self-archived online.
> Obviously, such counts could be cheated by researchers putting their
> own papers on reading lists and getting students clicking in.
Or by sending out automatic software agents to hit one's own papers
over and over. Abuses will evolve once the hit-rates start being used
as performance indicators (and counter-measures will evolve too); but
first the hit-rates have to be validated to determine whether and what
sort of performance indicator they really are.
> absence [of hit-measure] is holding back the eprint movement. Simply
> uploading papers at present has no feedback upon ones reputation -- no
> one has ever because of this put an extra line on their CV -- no, 'I
> was the author of top ten most visited uploaded papers'. Cogprints is
> silent to what gets researchers most motivated -- their ranking to
> other ones.
But not just any ranking: Ranking on measures that are demonstrably
related to quality and impact.
The way to establish a research reputation is still to publish in a
high quality peer-reviewed journal. THEN self-archive the paper to
enhance visibility and access, hence impact. Archives are not meant to
be Vanity Presses, bypassing peer review. (Virtually all of the 128,000
papers in the Physics Archive were first deposited as pre-refereeing
preprints, then they underwent refereeing, and either the refereed
finally draft was swapped for the preprint, or, if refereeing changes
were minor, the citation was updated to reflect the final published
version.
> that might change if appointment committees looked
> down CVs and checked for eprint visitation ranking in the way they do now
> days for citation impacts.
If they started to do that now, it would be foolish and arbitrary, like
using the number of emails their researchers send or receive. First it
must be determined what, if anything, those hits mean.
> the publication of such ranking
> lists of the most linked epapers will make researchers aware of the key
> advantage of being on the internet for the up and coming generation of
> students who will cite them later on in their own work.
The goal is not to be "cited" (linked?) by your students, but to have
your refereed research used to generate further research, which is in
part reflected by research citations. The first step on that path is to
have your research validated by peer review; then to self-archive it so
everyone can access it; then validated measures like citation impact
should show enhancement from the visibility and access. Last, perhaps,
new online performance indicators will evolve and be validated and used.
Eprint Archives are not meant to be new forms of Vanity Press, but freer
forms of access to the expert-validated research literature.
> suppose technology existed
> that let paper journals know the number of times a paper journal was opened
> to be read in the library or study. Do you think that knowledge would be
> ignored -- I suggest to you that it would be valued in the same manner that
> we present treat citations.
I doubt it! Journals would no doubt use it to enhance subscriptions,
but hit-rates would no more become a performance indicator on paper
(unless first tested and validated) than they will be online.
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 23-80 592-582
Computer Science fax: +44 23-80 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
Received on Mon Jan 24 2000 - 19:17:43 GMT