Re: The preprint is the postprint

From: Stevan Harnad <harnad_at_coglit.ecs.soton.ac.uk>
Date: Wed, 6 Dec 2000 18:53:31 +0000

On Wed, 6 Dec 2000, Les Carr wrote:

> Hypothesis: in some research cultures, for some journals, the practical
> DIFF is (to all intents and purposes) zero.
>
> Let us make some measurements. Go to the library and eyeball some physics
> papers and maths papers to measure the DIFF against the original LANL
> submission. Conduct some (anonymous) interviews.
>
> We know that different research cultures have different archiving,
> preprint and publication habits.
>
> Perhaps the hypothesis is true: DIFF is ZERO for a significant part of the
> research community. In which case skywriting, self-archiving and the
> internet provide the means for increasing collaboration and improving the
> effect of peer-review.
>
> Perhaps the hypothesis is false: DIFF is NON-ZERO for most of the research
> community.

I completely agree: We need to know the size of DIFF, and that size may
well prove to differ (sic) from discipline to discipline.

But what is the DEFAULT assumption, meanwhile?

That DIFF = 0
or that
DIFF > 0 ?

I think it is the latter. So the data you call for (and someone [us?]
should gather) would be for the purpose of testing the default
hypothesis that DIFF > 0 (i.e., peer review makes a difference), to see
whether it is false, or to see how true it is, for which disciplines.

The default hypothesis (before we go ahead and test it) is, in other
words, NOT: DIFF = 0 (until evidence shows it to be otherwise).
Rather, it is the reverse:

One cannot, as Greg and others do, simply ASSUME that the default
hypothesis is false. (Default means: assuming that there is indeed a
causal connection between current quality and the current
quality-control system.)

(And all of this is without taking into account the "invisible hand"
effects, which in fact work to reduce DIFF in the current system, but
only because the current system of peer review is what it really is,
currently: Preprints are written to a higher quality standard in the
first place, because the authors know they are submitting them for
peer review, when they will have to answer to referees before being
certified as accepted for publication. If everything went post-hoc
instead (i.e., if there were no contingency on acceptance/certification
at all), the quality of "p-prints" (no longer "preprints," because
there would be no "post" relative to peer review) would (I would
predict, based on human nature), plummet to the level of all other human
endeavours that are not answerable to quality-control.)

But the invisible-hand stuff is hypothetical, so the first thing that
needs to be tested is the current size of DIFF (not what it WOULD be
if there were no peer review).

Does that make sense?

Anyway, can we estimate DIFF in some computational way, based on
comparisons between digital prepints and digital postprints in LANL?

Stevan

--------------------------

Les Carr replied:

    The default hypothesis surely will depend on what you expect. It is
    culture dependent! As a computer scientist (no, as a member of a
    particular subcommunity of computer scientists) I expect DIFF to be
    very small. -- Les
Received on Mon Jan 24 2000 - 19:17:43 GMT