At 12:16 pm +0000 26/1/01, Stevan Harnad wrote:
>Simon's online peer review system is very interesting and welcome, but
>make no mistake about it: It is just classical peer review
Yes and no. Some distinctions need to be sharpened up, that also
inform the notion of eprint archives with quality control.
Yes, we place great importance on review by qualified peers who have
agreed to review an article. We have argued that without co-evolving
agreed Policies and Practices for using the technology, facilities
for online peer review will fail
[
http://kmi.open.ac.uk/tr/abstracts/kmi-tr-96.html]
One consequence of this view is that hoping that experts will
volunteer their reviews of eprint articles on any but the highest
profile articles may be optimistic. The social contract that exists
when a journal engages a reviewer is important.
However, I disagree with your terminology. Ask any academic what
"classical peer review" is, and they will tell you about the
"anonymous reviewer, no right of reply for author" process.
What you refer to as "classical peer review, implemented on-line,
with post-hoc commentary as a SUPPLEMENT" is what conventional
journals do when they 'go online': allow email/web upload submission
of articles and reviews, and *when published*, bolt on an "Add
Comment" button to the article, already cast in concrete. It's the
first, weakest use of a new medium, inheriting the assumptions of the
old medium.
Our conversational model places author-reviewer interaction at the
centre right from the start, makes this 'intellectual trace'
accessible to wider peers, and preserves an edited, open version of
it with the publication.
>It is just classical peer review (implemented
>online -- but that's irrelevant, because peer-review is
>medium-independent anyway,
Medium-independent? I think there's a synergy between process and
medium that can't be ignored. BBS has implemented commentary and
author response as best it could in the paper medium, but as Stevan
himself has argued, the net enables a more rapid tempo for scholarly
discourse. This is simply not practical in paper. CogPrints restricts
this benefit to "pre-journal" status contributions. JIME gives it
first class status as a mode of discourse suitable for journal
submissions, particularly in a new interdisciplinary context. The
context of use (disciplinary factors and user expertise may be
variables that determine success of more radical models).
>and even on-paper journals are implementing
>their peer review on-line these days) plus a subsequent phase of open peer
>commentary.
Hmmm -- not sure how many are *changing* their conventional 'batch
processing model' of review (1. submission received; 2. reviews sent;
3. revise if required) to offer something more productive for
improving submissions. They may allow post hoc comments to be posted,
but our sense from anecdotal reports is that these are rarely used.
Everyone's too busy. Why should I bother when I'll be rewarded for
doing other things?
>But these experiments on implementing peer-review on-line, and
>supplementing it
>with open peer commentary, though they are eminently worth performing and will
>be extremely useful, have nothing whatsoever
"nothing whatsoever" - I'm trying to suggest that there are lessons
to be learned from experiments in online peer review
>to do with the question of either
>"self-publication" or the untested merits of "open self-appointed,
>post-hoc review," as putative SUBSTITUTES for classical peer review.
We are certainly agreed that there are no proven substitutes for
organised peer review. (Future technologies may appear to provide
automatic analyses of quality, but you can't beat humans.) To make
review happen, you need to engage reviewers in a social contract. To
make this happen requires some form of coordination, up until now the
role of the journal.
However, it is not inconceivable that the net will enable
decentralised coordination of some sort, which is able to create the
social contract to review (or provide some other kind of incentive?),
without any notion of a Journal or other Learned Body to establish
the contract. Sort of Napster peer review model. But what's the
payback for the reviewer? Short of paying reviewers, I don't know of
anyone who's successfully devised other motivations yet.
Simon
Received on Wed Jan 03 2001 - 19:17:43 GMT