On Tue, 28 Sep 2004, Richard Poynder wrote:
> Mark McCabe argues that peer review is not endangered by a shortage of eyeballs.
> [See, http://groups.undp.org/read/messages?id=97687 ]
Peer-reviewers (and their eye-balls) are a scarce, over-harvested resource.
But Mark McCabe is right that there are probably enough qualified reviewers
to keep up with the growth in the number of research articles. (The reviewers,
after all, are just the authors -- the peers -- wearing other hats.) The
Web, and online article indices and databases (including Open Access full-texts
and reference lists) will help distribute the reviewing load more evenly,
however. See:
"Classical peer review will not change in its essentials in
the online/open-access age, but it will become faster, cheaper
and more efficient to implement. Papers will be submitted
by depositing them in the journal's website or the author's
institutional website. Many authors will elect to make their
preprints publicly accessible, either on the journal's website or
on their institutional website. Referee selection will be aided
by online searches of the open-access literature and sometimes by
calls-for-referees to targetted specialist lists. Referees will
access the manuscripts online and submit their reports online,
and editorial dispositions will be done online. Self-selected
commentary may sometimes supplement (but not substitute for) the
editor-appointed refereed. After publication, a similar system can
be used for open peer commentary, and authors' responses. Article
download and citation impact will be tracked from the preprint
onward by online scientometric engines."
http://www.ecs.soton.ac.uk/~harnad/Temp/peerevabs-amchem.html
"PostGutenberg Peer Review:
The Invariant Essentials and the Newfound Efficiencies"
http://www.jpgmonline.com/documents/editor/25/10_Harnad_peerreview_3.pdf
http://www.ecs.soton.ac.uk/~harnad/Temp/peerev.ppt
> If, however, there were
> effective alternative quality-control mechanisms available to do the first-level
> work of assessing scholarly papers might not those eyeballs be better occupied on
> something other than reviewing every paper written?
Qualified experts provide corrective and evaluative feedback on
specialized work in advance as a service to their busy peer user community
-- so researchers know in advance which papers (1) justify the investment
of their scarce reading time, and (2) which papers justify the further
investment and risk of their even scarcer and more precious research
time and effort, the risk of trying to use, apply and build upon them.
This is peer-to-peer service provided for free by referees when they
are wearing their referees' hats. Essential to the service is that it is
(1) done in advance (i.e. *before* their fellow-researchers invest their
time and effort in the paper), that it is (2) dynamic and interactive
(i.e., it is not just a green-light/red-light tagging of papers as
"publishable" or "unpublishable" [in a journal with a certain known
track-record for quality] but a process of revision in which the quality
of the paper is meant to improve] and it is (3) answerable (the referees
are selected for their expertise by a qualified expert editor, the author
is answerable to the referees and editor, the referees are answerable
to the editor, and the editor and journal are answerable to the journal's users).
There are potential ways to make this answerable system of human
evaluation and feedback more efficient and equitable (see URLs above), but
as far as I know, no *alternative* to it has yet been tested and shown to be
successful, sustainable and scaleable, and to yield articles that are
of a quality and reliability level at least equal to the one yielded
by peer review.
Alternatives, in other words, are merely in the mind of the speculator,
until and unless they have been tested and demonstrated to succeed,
sustain and scale. No such alternatives exist yet -- and open peer
commentary in particular (post-hoc vetting by self-appointed "peers,"
with time on their hands and answerable only to the anarchic court of
online opinion) is certainly no such alternative. See:
Harnad, Stevan (1998/2000) The invisible hand
of peer review. Nature [online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible/invisible.html
Longer version in Exploit Interactive 5 (2000):
http://www.exploit-lib.org/issue5/peer-review/
"Self-Selected Vetting vs. Peer Review: Supplement or Substitute?" (2002)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/2340.html
Harnad, Stevan (1997) Learned Inquiry and the Net:
The Role of Peer Review, Peer Commentary and Copyright.
Learned Publishing 11(4) 283-292. Short version appeared
in 1997 in Antiquity 71: 1042-1048. Excerpts also appeared
in the University of Toronto Bulletin: 51(6) P. 12.
http://citd.scar.utoronto.ca/EPub/talks/Harnad_Snider.html
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/94/index.html
> Concerning the larger question of whether peer review is relevant to a
> discussion of OA: it might be easier to agree that it was not relevant if
> all that was being archived along the green road were the "final,
> peer-reviewed, revised, corrected, accepted" papers that Stevan Harnad
> recommends. The reality, however, is somewhat different. As Stevan has
> himself elsewhere pointed out (http://www.infotoday.com/it/apr04/poynder.shtml)
> what is currently being archived is "a good deal of the target content - peer-reviewed
> journal articles - but also preprints, unpublished papers, non-papers, and
> metadata without the full-text papers". Anyone sifting for gold through this
> plethora of material will surely be seriously challenged? While the traditional
> peer review process may be good enough when it comes to assessing what is and is
> not of relevance/good quality in the print environment, it may not be good enough
> in a self-archiving universe.
Richard forgets that the journal-name -- the certificate that the paper has met
that journal's established quality standards -- is still there to serve as the gold
standard, as it always did, whether on-paper or on-line, whether for toll-based
access or open access. Researchers are well aware of the difference between an
unrefereed preprint and a refereed journal article. They have been dealing with
it for years. But one *unwelcome* filter they have also had to deal with for years
-- namely, whether or not their institution happens to be able to afford to pay
for access to the journal in which a particular paper appears -- is no longer a
constraint once all peer-reviewed articles are self-archived and made Open
Access.
As to whether the minuscule additional element of uncertainty as to whether an OA
article is indeed what it purports to be (by the self-archiving author), please see
the relevant entries on authentication, corruption and version-control in the BOAI
self-archiving FAQ:
http://www.eprints.org/self-faq/#2.Authentication
http://www.eprints.org/self-faq/#3.Corruption
http://www.eprints.org/self-faq/#23.Version
None of these call for an alternative to peer review. The relevant role of
peer-vetting in all this is not as a substitute for peer review, but as a detector
of occasional scholarly lapses in self-archiving practises!
http://www.eprints.org/self-faq/#7.Peer
Yes, peer-review is relevant to a discussion of OA -- but only inasmuch as it is
the *peer-reviewed* literature we are seeking Open Access *to*: This is done by
providing a version of each article that is free of access-tolls, not one that is
free of peer review!
> Indeed, if one talks to researchers in the developing world it is clear that many
> view self-archiving not as a supplement to peer review, but as an alternative.
You'll see the same untested view in the developed world too (and
even among some of the physicists and mathematicians who have been
self-archiving pre-refereeing preprints for years).
Peer Review Reform Hypothesis-Testing (started 1999)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#480
A Note of Caution About "Reforming the System" (2001)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#1170
Self-Selected Vetting vs. Peer Review: Supplement or Substitute? (2002)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#2341
The important thing in OA is to look at what "Simon" is *doing*, not what "Simon
Says" he is doing: Peer review proceeds apace, and all the literature (and its
quality) remains answerable to it; the self-archiving of pre-refereeing preprints
has not changed that in the least. Pre-refereeing preprints are not so much a
matter of Open Access (OA) as of "Early Access" (EA), as Michael Kurtz
has noted. Preprints are to be used with caution, but they are certainly
a useful supplement to Open Access to the later peer-reviewed versions
in many fields and many cases:
"OA advantage = EA + AA + QB + OA + UA"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/3977.html
But for most of the not yet self-archiving world, even OA is EA, and will continue
to be until all peer-reviewed journal articles are 100% OA!
> Unable to obtain a place at the peer-review table they believe self-archiving
> offers the best chance they have of getting their work before the eyeballs of
> their peers.
Open Access is about Open Access to the 2.5 million articles published
in the 24,000 peer-reviewed journals worldwide. Anything more (be it
early access or books) is merely a bonus. If it is work that cannot meet
the standards of peer review, it is not even that. Access to that is certainly
not OA (for the same reason that access to anything else someone may decide to
put on the Web -- be it an advert, a polemic, or a piece of porn -- is not OA).
Please let's (continue to) distinguish web access in general from OA
and the OA movement in particular!
"The literature that should be freely accessible online is that which
scholars give to the world without expectation of payment. Primarily,
this category encompasses their peer-reviewed journal articles, but
it also includes any unreviewed preprints that they might wish to
put online for comment or to alert colleagues to important research
findings."
http://www.soros.org/openaccess/read.shtml
> Perhaps the greatest mistake the OA movement could make is to assume that the only
> issue is one of access, without considering that in the process other things may
> have to change as well.
Access is certainly not the only issue. There are many other worthy
issues (poverty, illness, injustice) that have nothing to do with OA. But
the only OA issue is access, and only access to the target literature
(which is, to a first approximation, one and the same as the 2.5 million
articles published in the planet's c. 24,000 peer-reviewed journals). The
OA movement is not the "Open Information" movement. Human beings, with
mouths to feed, may still want to sell their books, magazine articles,
music, software, rather than give it away, even if it is all digital. But
every single author of every one of the 2.5 million yearly articles in the
24,000 peer-reviewed journals wants to give them away so as to maximize their
uptake, usage and impact. That is how (indirectly) the researcher manages to
feed the mouths to feed:
http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.2
http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.1
Napster
http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#9.1
But OA and self-archiving are not about *self-publishing* either:
http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.4
They are about providing access to peer-reviewed research articles,
both before and after peer-review.
"What should be self-archived?"
http://www.eprints.org/self-faq/#What-self-archive
Nor are they about reforming or replacing peer review:
Peer Review Reform Hypothesis-Testing (started 1999)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#480
A Note of Caution About "Reforming the System" (2001)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#1170
Self-Selected Vetting vs. Peer Review: Supplement or Substitute? (2002)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#2341
Stevan Harnad
UNIVERSITIES: If you have adopted or plan to adopt an institutional
policy of providing Open Access to your own research article output,
please describe your policy at:
http://www.eprints.org/signup/sign.php
UNIFIED DUAL OPEN-ACCESS-PROVISION POLICY:
BOAI-2 ("gold"): Publish your article in a suitable open-access
journal whenever one exists.
http://www.earlham.edu/~peters/fos/boaifaq.htm#journals
BOAI-1 ("green"): Otherwise, publish your article in a suitable
toll-access journal and also self-archive it.
http://www.eprints.org/self-faq/
http://www.soros.org/openaccess/read.shtml
AMERICAN SCIENTIST OPEN ACCESS FORUM:
A complete Hypermail archive of the ongoing discussion of providing
open access to the peer-reviewed research literature online (1998-2004)
is available at:
http://www.cogsci.soton.ac.uk/~harnad/Hypermail/Amsci/index.html
To join the Forum:
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
Post discussion to:
american-scientist-open-access-forum_at_amsci.org
Received on Tue Sep 28 2004 - 13:08:34 BST