Re:
http://text-e.org/debats/
A-Priori Peer Review vs. A-Posteriori Review and Commentary (on
Peer-Reviewed Research)
Stevan Harnad
The function of peer review (refereeing) in research publication is
both to shape and to label the quality of published research. It is
a dynamic process, involving closed review or raw manuscripts by
selected, qualified expert referees and answerability to a qualified
expert editor, who is in turn answerable for the reliability of his
established journal label.
There may be several rounds of revision and re-refereeing before a
paper earns the journal-label (it may never: the highest quality
journals are by definition also the most selective and exclusive, which
usually means having the highest rejection rates). The research
community then uses these labels (there is a hierarchy of them, as
there is a hierarchy of quality among refereed journals) to filter
their reading in what would otherwise be not only an un-navigable
information glut, but also a wash of raw materials in indeterminate
stages of refinement, reliability and useability.
So far, what I have described is all PRE-labelling. Post-labelling
reviews, commentaries and other forms of evaluations are very valuable
SUPPLEMENTS to this quality-control, but they are not SUBSTITUTES for
it.
I state this with some confidence. After nearly a quarter-century's
experience in implementing both pre-publication peer-review and
post-publication peer commentary --
http://www.bbsonline.org -- I know
the difference! None of you wants to have to have to contend with the
raw drafts that make their way to the editor's desk. Referees (unpaid)
will do so, heroically, for a small, targeted number of raw drafts
annually, but only if explicitly selected by the editor of a journal
they regard as representing a sufficiently high quality standard, and
knowing that the peer review process will be answerable to the referees,
editor, and the journal's established standards. Referees are a rare,
precious, over-utilized resource. They offer their services out of a
peculiar combination of responsibility, magnanimity, and superstition
("referee unto others as you would have them referee unto you").
And there is not enough money in the world to begin to pay them (or the
authors, for that matter) fairly for their give-away services.
Peer-review implemented on-line is merely peer review, implemented
on-line. And whoever implements it, and certifies its implementation
to their quality-standards with a label, is a refereed journal
publisher. Most of the more important journals today, are implementing
their peer review on-line.
So as long as you have classical peer review, you have journals. The
established journals (of which there are over 20,000 already) have the
advantage of experience with peer review, an established authorship,
refereeship, editorship and readership, and a known, established label.
New journals are a pig-in-a-poke but occasionally new ones do appear,
and very occasionally they even survive, and pass into the canon.
Occasionally also journal labels migrate from publisher to publisher, or
die.
BioMedCentral
http://www.biomedcentral.com/ is one of these new
journals -- a mega-journal, actually. It is online-only, peer-reviewed,
and all of its full-text contents are accessible on-line for free.
It is also -- relative to the canon of perhaps of perhaps 10,000
established biomedical journals -- virtually empty. It faces the usual
uphill battle of new journals trying to establish themselves. It has
some advantages -- some stellar names, a large promotional system, and
of course the promise of free online accessibility to everyone at a time
of growing concern about access. But it also faces the extra handicap
of not only being unestablished, and competing with the many
established journals, but also of being online-only (a feature that the
scientific community is still -- irrationally -- leery about).
But let us wish BioMedCentral every success; for if, mirabile dictu,
they do capture a significant portion of the biomedical authorship, and
successfully maintain the peer-review standards of the journals from
which those authors have defected, then BioMedCentral will have brought
biomedical research that much closer to the optimal and inevitable: the
freeing of access to the entire 20,000 online.
Now what is "Faculty of 1000"? It appears to be a spin-off of
BioMedCentral. Recall that BioMedCentral gets no subscription/license
revenues because it gives away its contents free online. How does it pay
the costs of implementing peer review? Those costs are substantially
lower than what the planet is currently paying collectively in
subscription/license fees (an average of $2000 per article) in exchange
for the limited access that that buys for the lucky few who
can afford to pay (peer review costs account for 10-25% of that, or
$200-$500 per article).
My guess is that one of the purposes of the Faculty-of-1000 service
is to pay for the peer review. Another is to try to attract more
authors to BioMedCentral.
But what is certain is that Faculty-of-1000's post-publication
review service is no substitute for peer review: It presupposes it!
DAN SPERBER: "I well understand that existing journals are at present
best placed to go on organizing the peer review process. But if their
other traditional functions linked to printed paper go, or become
marginalized, new ways of providing peer review may emerge, based on
new and different types of organizations."
Perhaps. But so far no coherent candidates have been proposed (let
alone tested!) that are not either (1) just notational variants of peer
review (on-line peer review), (2) untested (and unrealistic)
speculations about substituting a-posteriori commentary for a-priori
peer review, or (3) even more unrealistic notions that the power and
success of unrefereed preprints have demonstrated that peer review was
never needed in the first place!
So the only thing really being contemplated here is title-migration
(along with editorial board, refereeship and authorship).
See:
http://www.eprints.org/results/
RANSDELL: "Publication at LANL as involving peer review"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/0333.html
JTW SMITH: "Re: The True Cost of the Essentials (Implementing Peer Review)"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/0310.html
KUPERBERG: "Central vs. Distributed Archives"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/0950.html
DAN SPERBER: "For instance, I have just come across "Faculty of 1000" -
http://www.facultyof1000.com/start.asp - described as 'an innovative
online research tool from BioMed Central that highlights the most
interesting papers in biology, based on the recommendations of over
1000 leading scientists'."
DAN SPERBER: "Here is the short description provided by the (paying but
testable for free) service... I have not tested it (it is not my field)
but the idea looks very promising indeed, and quite different from the
journal format. Would services of this kind complement journals?
Replace them? Or interact with journals in yet other ways?"
A-posteriori commentary complements, but cannot substitute for
a-priori peer review.
The excerpts below are for a for-fee service, so I have no comment,
except to say that there will be a-posteriori reviews and commentaries
for-free too, and more and more of them as more and more of the primary
refereed (sic) research on which the reviews are based becomes accessible
on line for free. Ceterum censio: To free it all, we don`t need to
sacrifice submitting our papers to our preferred, established refereed
journals; we need only self-archive them, now!
Harnad, S. (2001) Six Proposals for Freeing the Refereed Literature
Ariadne28 June 2001.
http://www.ariadne.ac.uk/issue28/minotaur/#1
BIOMEDCENTRAL/Fo1000:
http://www.biomedcentral.com/
Faculty of 1000 is the next generation literature awareness tool. It is
a revolutionary new online research service that will comprehensively
and systematically highlight and review the most interesting papers
published in the biological sciences, based on the recommendations of a
faculty of well over 1000 selected leading researchers.
Faculty of 1000 will be run by scientists for scientists, and will
provide a rapidly updated consensus map of the important papers and
trends across biology.
Faculty of 1000: ? Provides scientists with a continuously updated
insider's guide to the most important papers within any given field of
research
? Highlights papers on the basis of their scientific merit rather than
the journal in which they appear
? Offers the researcher a consensus of recommendations from well over
1000 leading scientists
? Systematically organizes and evaluates the mass of information within
scientific literature
? Offers an immediate rating of individual papers by the authors'
peers, and a first real alternative to the indirect assessment provided
by the journal impact factor.
Received on Mon Nov 26 2001 - 16:42:16 GMT