> From: Bruce Edmonds <b.edmonds_at_MMU.AC.UK>
>
> Since what readers need is:
>
> 1) assurance of quality and readability
> 2) assurance of permenance
>
> ... all you need is a system of review and an archiving service. You do
> need a journal as such.
That is all a journal is. The "journal" part is simply the quality
controller and the provider of the quality-control tag.
> Authors would put their paper on the web and give notification to the
> relevant subject review board that they wish their paper to be reviewed.
This is tantamount to submitting a paper to a peer reviewed journal.
But why should there be only one "relevant subject review board" for
each subject (and what is a "subject"? Biology, Microbiology,
Oncogenes, Iontophoresis?)? There are many journals, often even in the
same specialty, because research varies in quality, and this is
reflected in the level of the journal quality hierarchy in which the
paper appears.
Covarying with the subject matter and the quality level of the journal is
the composition of their Editorial Boards, and the corresponding
composition of their referee population. The very best referees only have
time to referee the very best papers for the very best journals.
The reader benefits from being able to use this known hierarchy of
quality-control tags.
> It is then reviewed (either by traditional blind review or open) and given
> star ratings (zero to 5) for content and for presentation and entered on
> database.
Refereeing is not like school grading of essays. Peer review does not
consist of giving stars for content and presentation. Referee reports
give feedback to the Editor and author about what is wrong and right
about the paper, and what needs to be done to make it publishable
(if it is potentially publishable at all: in the highest quality
journals, sometimes only 10 of submitted papers reach threshold for
potential publishability).
Should all those reports be posted openly on the Web? Would all (most?)
authors want that? Assuming that the answer is no, then we are back to
classical peer review again, whether you call the administering body a
"journal" or a "subject board."
But besides keeping the reports confidential, if the author doesn't want
them publicised, what provision is there for seeing that they are heeded
by the author?
> Readers could browse this at their own choice of level (only 5
> star on both content and presentation, above 2 for presentation and above
> star for content) as well as on subject codes and keywords.
First drafts together with stars for content and presentation (even
assuming that the referees are well chosen and competent) is not a
quality-tagged literature. It simply sorts out the first-drafts that
happen to be near publishable standards. What happens next? Who sees to
it that the rest (99% of submitted papers are NOT publishable without
varying degrees of revision, depending on the referee reports) are
revised in accordance with the expert advice?
If Editors state and administer the conditions for revision and
re-refereeing (based on the referee reports), this is again classical
peer review (call it what you like). If no one is adjudicating the
revision, then it is a free-for-all, and the question becomes: What
referees would devote their limited time and expertise to such a free
for all, where every draft is "published" with "stars," and there is no
answerability to referees' recommendations -- except endless rounds of
further star-seeking: What referees have the time to contribute to an
open-ended free-for-all like that?
[Note: I am all for public archiving of unrefereed drafts; but who
benefits from calling these anything other than what they are:
unrefereed drafts, with or without the "stars."]
> Additionally
> they could subscribe to a mailing list to be notified of new papers at their
> chosen levels of accepatability.
The star-tagging system for first drafts is of almost no use at all; for
multiple iterations of subsequent drafts, all straining for 5-star status,
it is a completely unrealistic drain on referee time, and hence could
not lead to competent refereeing, which requires answerability and
convergence, administered by a competent and likewise answerable umpire
(the editor). The multiple drafts archive would also be un-navigable,
with the meaning of generic 5-star status completely uninterpretable as
the archive gets saturated with successive drafts and referee
willingness and quality are unknown.
> There would be no revision of papers as such, but authors could re-submit
> papers (subject to some limits) to obtain higher ratings in presenation and
> content by addressing the previous (and others') comments.
Not clear what this means. Sounds like multiple revisions to me. What
you mean is all prior drafts remain public. Fine, if the author so
wishes, but that has nothing to do with peer review.
> Every five years the archive of papers (or the top x% of it) would be issued
> on a CD_ROM as a subscription to libraries (who would pay for this and thus
> cover the work need to copy papers onto the archive).
Completely unrealistic. Libraries don't want 5-year-old work. No one
wants or needs CD-ROMs. The literature is fine on the Web, mirrored the
world over, in central archives and author archives. This part is all
a red herring.
The real cost (though small) is in administering the peer review
(because referees referee for free in an answerable quality-control
system) and editing, and this is the only way (so far proposed) to
generate a quality-tagged literature that a reader can trust and
navigate.
> A web robot would periodically check that the papers were at the url and
> delete them from the review database if this is relevant.
These are irrelevant and trivial matters; quality control is the serious
one, and schemes like this misunderstand it completely.
> In this way the review service could be seen as an add-on to existing open
> paper archives such as xxx.lanl.gov or cogprints. Since academics already do
> the reviewing and much of the mark-up (or a secretary in the department
> does it) there would be almost no running cost for this
> (apart from the archve service
> which could cover itself with a very low subscription from libraries).
The fatal flaw of schemes like this is in the administration of the
peer review. Referees are generous with their time under the right
conditions (classical peer review), but not in an open-ended public
free-for-all like this: Why should they be?
The archiving costs are irrelevant and trivial: they are coverable. The
problem is the implementation of quality control of at least the same
level of effectiveness as classical peer review in paper has been to date.
> There seems to be *no* need for a publisher any more.
Call the "publisher" the implementer of peer review and call the
"journal" the quality-control tag. We still need both, by any other
name, and schemes like this, usually dreamt up by authors who have had
little hand in the quality control process, haven't a chance of
providing them.
Stevan Harnad psyc_at_pucc.princeton.edu
Editor, Psycoloquy phone: +44 1703 592-582
Department of Electronics fax: +44 1703 593-281
& Computer Science
http://www.ecs.soton.ac.uk/psyc
University of Southampton
http://www.princeton.edu/~harnad/psyc.html
Highfield, Southampton
ftp://ftp.princeton.edu/pub/harnad/Psycoloquy
SO17 1BJ UNITED KINGDOM
ftp://cogsci.soton.ac.uk/pub/harnad/Psycoloquy
news:sci.psychology.journals.psycoloquy
Sponsored by the American Psychological Association (APA)
Harnad, S. (ed.) (1982d) Peer commentary on peer review: A case study in
scientific quality control, New York: Cambridge University Press.
Harnad, S. (1984d) Commentaries, opinions and the growth of scientific
knowledge. American Psychologist 39: 1497 - 1498.
Harnad, S. (1985f) Rational disagreement in peer review. Science,
Technology and Human Values 10: 55 - 62.
Harnad, S. (1986) Policing the Paper Chase. (Review of S. Lock, A
difficult balance: Peer review in biomedical publication.)
Nature 322: 24 - 5.
Harnad, S. (1990d) Scholarly Skywriting and the Prepublication Continuum
of Scientific Inquiry. Psychological Science 1: 342 - 343 (reprinted in
Current Contents 45: 9-13, November 11 1991).
ftp://ftp.princeton.edu/pub/harnad/Harnad/HTML/harnad90.skywriting.html
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad90.skywriting.html
Harnad, S. (1995h) Universal FTP Archives for Esoteric Science and
Scholarship: A Subversive Proposal. In: Ann Okerson & James O'Donnell
(Eds.) Scholarly Journals at the Crossroads; A Subversive Proposal for
Electronic Publishing. Washington, DC., Association of Research
Libraries, June 1995.
http://www.library.yale.edu/~okerson/subversive.html
http://cogsci.soton.ac.uk/~harnad/subvert.html
ftp://ftp.cogsci.soton.ac.uk/pub/psycoloquy/Subversive.Proposal/
Harnad, S. (1996a) Implementing Peer Review on the Net:
Scientific Quality Control in Scholarly Electronic Journals. In:
Peek, R. & Newby, G. (Eds.) Scholarly Publishing: The Electronic
Frontier. Cambridge MA: MIT Press. Pp. 103-118.
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html
ftp://ftp.princeton.edu/pub/harnad/Harnad/HTML/harnad96.peer.review.html
Harnad, S. (1998b) Learned Inquiry and the Net:
The Role of Peer Review, Peer Commentary and Copyright.
Learned Publishing 4(11): 283-292
Shorter version in 1997: Antiquity 71: 1042-1048
Excerpts also appeared in the University
of Toronto Bulletin: 51(6) P. 12.
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad98.toronto.learnedpub.html
ftp://ftp.princeton.edu/pub/harnad/Harnad/HTML/harnad97.antiquity.html
http://citd.scar.utoronto.ca/EPub/talks/Harnad_Snider.html
Harnad, S. (1998h) The invisible hand of peer review. Nature [online]
(c. 5 Nov. 1998)
http://helix.nature.com/webmatters/invisible.html
Longer version:
http://www.ecs.soton.ac.uk/~harnad/nature2.html
ftp://ftp.princeton.edu/pub/harnad/Harnad/HTML/nature2.html
Received on Wed Feb 10 1999 - 19:17:43 GMT