Greg Kuperberg raises a point that has been raised (and answered) in
this Forum many times before.
I will repeat the point and the reply in summary, then
quote/comment some specifics:
Greg believes that the "preprint is [or is becoming] the postprint,"
that is, researchers are treating the unrefereed preprint as the
definitive report of the research, and that there is, or soon will
be, no more need for journals or peer review.
I will now list the empirical and logical evidence that strongly
contradicts this belief, and close with an indication of why the belief
is not only wrong, but why airing it as if it were right can actually
serve to hold back the freeing of the research literature through
self-archiving, which Greg, like me, wishes to promote.
I. The empirical evidence against the thesis that
"preprint = postprint"
Since the advent of public online self-archiving of research over a
decade ago, there has been NO SIGN WHATSOEVER of any change in
researchers' standard practise of continuing to submit all their
preprints to refereed journals (or refereed conference proceedings)
for peer review and publication. Self-archiving, as I have pointed
out many times, is a SUPPLEMENT TO, not a SUBSTITUTE FOR
peer-reviewed publication. For example, virtually every one of the
130,000 papers in the Physics Archive eventually appears in a
refereed journal, and this has been true from the very first days
of the archive onward: The preprint appears first, and then (within
about 8-12 months) either the postprint is archived too, or the
preprint is updated to include the reference to the postprint
(presumably as a function of whether or not the author judges the
preprint/postprint differences important enough to archive, which
varies from paper to paper):
http://opcit.eprints.org/tdb198/opcit/q2/
http://opcit.eprints.org/tdb198/opcit/q3/
http://opcit.eprints.org/tdb198/opcit/q4/
http://opcit.eprints.org/tdb198/opcit/publication/
http://opcit.eprints.org/tdb198/opcit/frequency/
http://opcit.eprints.org/tdb198/opcit/typical/
Harnad, S. & Carr, L. (2000) Integrating, Navigating and Analyzing
Eprint Archives Through Open Citation Linking (the OpCit Project).
Current Science 79(5): 629-638.
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad00.citation.htm
II. The logical evidence against the thesis that
"preprint = postprint"
As all preprints are eventually destined for refereeing, they are
knowingly written in anticipation of being answerable to peer
review (the "invisible hand" of peer review). This means that the
current quality of unrefereed preprints, and the use to which it is
safe (in some fields) to put them, are NOT indicators of what the
literature would be like if it were no longer written with the
expectation and eventuality of answering to peer review. It is pure
speculation to assert either (i) that preprint quality and usability
would stay at its current level if there were no peer review OR
(ii) that a hypothetical post-hoc "open 'peer' feedback" (by the
readers and users of the Archive) could substitute for the
quality-control function currently exerted by peer review. The
first (i), is just an untested conjecture (with a lot of prima
facie evidence about human nature and answerability in other
domains going against it); the second (ii) is even worse, being
compounded by further untested assumptions about the feasibility of
navigating, let alone systematically evaluating and sign-posting a
an unfiltered and unconstrained literature in which post hoc
public opinion (rather than advanced peer filtering and
sign-posting) is both the quality-controller and the guide.
III. The counter-productivity of promoting such untested and improbable
speculations
It is a historic fact that, apart from 30-40% of Physicists (and
fewer mathematicians), the research community has been much slower
to realize and act upon the possibility of at last freeing its
refereed research literature online through self-archiving than it
might have been (and even in physics/maths the liberation process is
only growing linearly, which means it would take another decade to
free it all at the current rate). Yet it is also clear (without any
hypothesizing or speculation) that freeing the entire refereed
research literature online TODAY would not only be optimal
(maximizing the access to and the impact of this give-away corpus),
but that it is already feasible TODAY. So the real challenge is to
facilitate this sure thing, and not to swap uncertain speculations
in its place. Freeing the refereed literature now, through
self-archiving, is the sure thing; trying to eliminate refereed
journals and/or refereeing, and/or trying to substitute untested
and unlikely alternatives for them can only serve to generate more
confusion and further delay in research's already overdue passage
to the optimal and inevitable.
Harnad, S. (1998/2000) The invisible hand of peer review. Nature
[online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible/invisible.html
Longer version in Exploit Interactive 5 (2000):
http://www.exploit-lib.org/issue5/peer-review/
http://www.ecs.soton.ac.uk/~harnad/nature2.html
On Mon, 27 Nov 2000, Greg Kuperberg wrote:
> Subject: The preprint is the postprint
>
> As I understand it, Stevan Harnad envisions the transition to open
> archives as taking place in a specific order: First we shall free the
> literature, preserving the dichotomy between unrefereed preprints and
> refereed postprints, and then we will consider whether or not peer review
> needs to be restructured. I think that the mathematical literature,
> at least, should go and is going in a different direction.
Not consider: test. And having tested and found a better alternative
(if any), implement it. But for now, the only sure road to take is to free the
refereed literature SUCH AS IT IS, immediately. That is urgent, and its
benefits certain. The rest is hypothetical (and very possibly
wrong-headed), and conditional on the outcome of a prior period of
systematic empirical testing, with unknown outcome. Hence it is merely
another needless obstacle to freeing the literature right now.
> TeX, the Internet, and the math arXiv have been steadily erasing the
> difference between the preprint and the postprint.
Nothing of the sort. They have simply made the preprint more similar to the
postprint in form. The substantive difference between the preprint and the
postprint is the changes resulting from peer review. Moreover, the
eventual answerability to peer review even makes its pre-emptive mark
on the preprints (the "invisible hand").
> There was a time
> when postprints were better edited, better typeset, better distributed,
> and more permanent than preprints.
But that is the trivial part of the preprint/postprint difference. The
substantive part is the changes resulting from the refereeing (and the
"silent" effects of writing a paper from the outset with the
foreknowledge that it will have to answer to peer review).
> These days most published math
> papers are only slightly better edited than preprints, and sometimes
> not at all; only slightly better typeset, and sometimes not at all.
Most journal editing is nonsubstantive; when it is substantive, it
should count as part of peer review, hence a significant
preprint/postprint difference.
> If a preprint is in the arXiv, it is at least as permanent and at least
> as widely distributed as any published paper. So the main remaining
> difference between preprints and postprints is that postprints are
> "peer reviewed", meaning that they have been anonymously refereed.
Correct.
> Sometimes the referees suggest important changes, but such suggestions
> are equivalent to those that from other colleagues. Peer review of
> math papers, then, is little more than a state of mind.
Alas you lost me completely here!
How often do referees suggest "important" changes? What data is your reply based
on?
What evidence is there that that quality-control service -- currently
implemented by a qualified Editor who chooses the qualified referees
based on their expertise, and to whom the author is answerable for
making the improvements they request -- could be performed by (soliciting?)
suggestions from (self-chosen?) colleagues (and picking and choosing
which to follow and which to ignore?)?
This all sounds quite speculative, not to mention subjective, to me...
> By analogy,
> book reviews are certainly useful, but they do not change the books
> themselves, only the readers' impression of them.
Book reviews are post hoc. (Book publishers usually also ask reviewers
to read the manuscript, recommend changes, and advise them about
whether or not to accept it for publication, but even that does not in
general make the book literature a refereed literature, as the journal
literature is.) Nor is unrefereed research, if it is reported in a
book, in general assigned the same weight by its users as refereed
research in a peer-reviewed journal (although this varies somewhat from
field to field).
So what is your point, actually? Post hoc book reviews certainly do not
make the book literature into a peer reviewed literature. Are you
recommending that the peer-reviewed journal literature be un-made, to
become equivalent in quality to the book literature, by dropping the
peer review and substituting post hoc review? What evidence do you have
for what the quality and useability of that hypothetical future
literature would be, compared to the current actual one (the one that
we are talking about freeing online)?
Wouldn't it be better to just free the refereed literature from
fee-barriers, a sure benefit, before going on to "free" it from peer
review, a rather less sure one?
> The arXiv inevitably
> relegates anonymous journal refereeing to the same role.
Not arXiv, I must repeat (as arXiv contains BOTH the pre- and
post-refereeing literature), but some hypothetical future archive, in
which there is no more peer review, and the entire literature is not
only unrefereed, but freed of the "invisible hand" too.
Nor, as I said, is the pre-emptive refereeing of refereed-journal submissions
anything like the post-hoc reviewing of unrefereed (or refereed) books!
> To be sure, many mathematicians still see a dichotomy between "preprints"
> and "published papers". For example many authors omit the arXiv numbers
> for papers that are also published. However even these people contradict
> their habits as bibliographers with their habits as readers. Everyone
> assumes that different versions of a paper are usually roughly the same.
Everyone? And are they indeed roughly the same?
http://opcit.eprints.org/tdb198/opcit/q2/
(and see earlier URLs too).
> And if there are important differences, a crucial correction is almost
> as likely to show up in a post-published e-print, or in a future paper,
> as in the original published version.
Do you have any data to support these contentions?
> So in my view the dichotomy between preprints and postprints is
> somewhere between fictitious and artificial. And I think that instead
> of postponing the peer review question, the right approach is to try
> to restructure peer review around the math arXiv. So far we have only
> succeeded in taking it half way, and only on a small scale. There are
> now three journals, Advances in Theoretical and Mathematical Physics,
> Geometry and Topology, and Algebraic and Geometric Topology, that are
> arXiv "overlays"; they systematically contribute all of their papers to
> the arXiv, and they systematically contribute the typeset copy as a new
> version of papers already in the arXiv. See
>
> http://www.intlpress.com/journals/ATMP/
> http://www.maths.warwick.ac.uk/gt/
> http://www.maths.warwick.ac.uk/agt/
But are these journals peer-reviewed? If so, you are just discussing
peer-review implementation details with no bearing on what we were discussing
above. Peer review itself is medium-independent.
And if they are not peer-reviewed, what is their quality and impact,
relative to the peer-reviewed literature? (And what would it all look
like if there were no peer review at all, just journeyman post-hoc
commentaries everywhere?)
> A more radical approach is to review arXiv articles without distributing
> them at all or claiming possession in any way, indeed without even
> consulting the authors. This is what Quick Reviews does in the field
> of quantum computation:
>
> http://quickreviews.org/
>
> Admittedly Quick Reviews is a primitive effort, but it may well
> represent the future.
If these are reviews of articles in the arXiv, see above; nothing is at issue.
These are just ad hoc reviews of preprints and of postprints -- welcome
supplements to, but no substitute for, peer review.
> Regardless, the steps towards integrating peer review with the arXiv
> (both the conservative experiments and the more radical ones) are not by
> any means a distraction from the ultimate goal. They add to the arXiv's
> reputation and they help attract new submissions. We don't want to pull
> the rug out from subscription journals and leave peer review suspended
> in mid-air. Rather we would like to work with journals and reviewers
> to build the new system.
The picture you describe sounds incoherent to me. The arXiv contains both
unrefereed preprints and refereed postprints; virtually everything in it
eventually becomes a refereed postprint (or tries to).
You also describe (refereed?) online-only journals that seem to use the arXiv as
a front-end ("overlay"); that's fine too, but so what?
And you describe ad hoc reviews (of either the unrefereed preprints or the
refereed postprints in the arXiv, or both). Fine again, but so what?
Then you speak of "submissions." But "submissions" to what? To refereed
journals? That has nothing to do with the arXiv (which is just a
front-end, a submissions conduit to the journals). Or do you mean
"deposits" in the arXiv? But that's not a submission! Apart from being
vetted for wackos, the arXiv accepts all comers. The selective process
comes from the peer review, which is not being provided by the arXiv
but by the journals. And the post-hoc reviews have nothing to do with
any of this.
So who/what is trying to attract "new submissions" to whom/what?
Just what new system do you think you are building out of these
components, in which the journals keep doing the peer-reviewing, the
archive keeps doing the freeing, and you keep imagining that peer
review is somehow on the way out? (Could you perhaps be confusing
archiving with refereed publication, and the provision of the essential
SERVICE of peer review, with the provision of an optional PRODUCT, the
text, on-line or on-paper -- optional, because it is already available
free from the archive?)
The real part of this is actually the same goal I'm headed for, but I
think I am calling a spade a spade: This is the freeing of the
peer-reviewed literature, online, through author self-archiving. All
the rest stays the same (except perhaps the vendors' S/L/P revenues,
eventually, at which time the modest but essential peer review
implementation costs per paper can be paid for out of the
author/institution's S/L/P savings, so nothing need remain suspended in
mid-air -- except add-on products and services, on-paper or on-line,
over and above the essential one of refereeing -- products and services
for which there may no longer be a market eventually, once the refereed
literature is all available online for free through author/institution
saelf-archiving).
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 23-80 592-582
Computer Science fax: +44 23-80 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
NOTE: A complete archive of the ongoing discussion of providing free
access to the refereed journal literature online is available at the
American Scientist September Forum (98 & 99 & 00):
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
You may join the list at the site above.
Discussion can be posted to:
american-scientist-open-access-forum_at_amsci.org
Received on Mon Jan 24 2000 - 19:17:43 GMT