Re: A Note of Caution About "Reforming the System"

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Sun, 15 Dec 2002 13:13:22 +0000

On Sat, 14 Dec 2002, [identity deleted] wrote:

> I agree totally about self-archiving, but am convinced its inadequate as
> a total strategy. Its a while since I read your stuff. But after reading
> it my feeling was: I agree with the goals, but I dont think the
> self-archiving program will get us there, for numerous reasons.
> (Can elaborate later)

Since things are still going too slowly, and since misunderstandings are
rife (and keep replicating themselves), I'll try to see what the source
of this misunderstanding is:

You say you agree with the goals. There is one primary goal and one
secondary goal. The primary goal is open access to the entire
peer-reviewed research literature (20,000 journals, 2,000,000 articles
annually, across disciplines, languages, globally), such as it is, as
soon as possible.

If and when the (500,000?) authors of the annual 2,000,000 articles
self-archive their peer-reviewed final drafts ("postprints") in their
own institutional OAI-compliant Eprint Archives (all interoperable with one
another, harvestable, hence seamlessly internavigable), that primary goal
will be reached. (You may have doubts about whether authors will indeed
go ahead and self-archive, but I hope you don't disagree that if/when
they do, self-archiving *does* get us to the goal of open access to the
entire peer-reviewed literature.)

The secondary goal is open access to as much as possible of the
pre-peer-review preprint literature. (I say "as much as possible,"
because of course there is not, and there should not be, a desire to
coerce authors into making their papers public any earlier than they
wish to, and the only *sure* point -- at which all authors, with 100%
certainty, want to make their papers public -- is in the form of the
final, peer-reviewed accepted draft, the postprint, that eventually
appears in the journal. But the secondary goal is still open access to
as much as possible of the preprint literature, and here too, the means
of reaching that goal is self-archiving.

So if we are speaking of the same goals, it is hard to see why you have
doubts about self-archiving's reaching them!

There are other goals that some have, however, goals that logically
and practically have nothing whatsoever to do with the primary goal of
open access to the current peer-reviewed literature (such as it is)
nor the secondary goal of open access to as much as possible of its
pre-peer-review preprint phase.

One of these other goals is an amalgam of open-access plus a new form of
literature, its quality controlled by a new form of "peer review." The
new form recommended has varied, from (1) no peer-review at all, only
self-selected vetting by whoever reads and posts comments on whatever
they read and comment on, to (2) more organized peer-review proposals,
borrowing or re-inventing more and more of classical peer review,
either in the form of (3) multiple "grading authorities," reading and
providing "marks" for the preprints literature, or else (4) post-hoc
review journals (of the kind you mention), doing the same sort of
thing, but not just giving "marks" but giving reasons -- all the way to
the complete reinvention of classical peer-review by simply creating
(5) new, open-access peer-reviewed journals, and trying to persuade
the authors of the 2,000,000 papers currently appearing in the 20,000
toll-access peer-reviewed journals to submit to these new open-access
journals instead.

Apart from (5), which is equivalent to BOAI-2, all these proposals are
hybrid: They include both the goal of open access *and* the goal of
peer-review modification or reform of some sort. And -- this is the most
important thing -- without exception, the peer-review reform component is
based on untested speculation, not on empirical evidence demonstrating
that the hypothetical system is indeed feasible, sustainable, and
scaleable to the refereed literature as a whole, and that it will indeed
deliver a research literature of a quality at least comparable to the
quality it has now, as constrained by classical peer review.

In contrast, the open-access component (both BOAI-1 and BOAI-2)
has indeed been tested, and demonstrated to be feasible, highly used,
and highly beneficial to research visibility and impact.

In short, a sure thing (the feasibility and benefits of open access to
the peer-reviewed literature) has been bundled together with a very
unsure (because untested) thing (the feasibility and benefits of a
hypothetical alternative to peer review, and whatever literature it
might yield) and added to the list of paths to open access:

(1) BOAI-1 (self-archiving the peer-reviewed literature), which has
already been shown to work, many times over, and which can in principle
be done overnight, and faces only authors' hard-to-explain
sluggishness in just going ahead and doing it!

(2) BOAI-2 (creating new open-access journals and/or converting
toll-access journals to open-access), which would also clearly work, but
depends not only on (i) authors' willingness to relinquish submitting
to their current established journals and submitting instead to new,
open-access journals, but also on (ii) the *creation* of alternative
(or converted) counterparts for the existing 20,000 journals, before
authors can switch their submission allegiances to them. A sure path to
open access, but a rather longer and slower one than self-archiving.

(3) Alternative-System approaches to open access: This requires not only
(a) self-archiving (i.e., depositing the unrefereed preprints in an OAI
Archive), as in BOAI-1, but it *also* requires authors to (b) relinquish
submitting to their established journals and it requires the alternative
journals to be first created, as in BOAI-2, *and* it further requires
that this all be done while consciously (c) relinquishing not only
the establshed journals, but the classical peer review system itself,
with everyone switching allegiance instead to a completely untested
alternative system, accepting -- on faith and in advance -- that it will
work, and will deliver a literature of quality at least comparable to
the refereed literature they have now, the literature they would now
like to see freed from toll-access barriers.

I think you can't help but see that option (3) is not only the longest
and most circuitous path of all, but that it is also the only one that
adds untested speculation as a liability to the tested and sure benefits
of BOAI-1 and BOAI-2.

I suspect that, like Greg Kuperberg and others,

    'A Note of Caution About "Reforming the System"'
    http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/1169.html
    "Self-Selected Vetting vs. Peer Review: Supplement or Substitute?"
    http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/2340.html
    http://www.eprints.org/results/

you may be interpreting the heavy and fruitful usage of pre-refereeing
preprints by the physics and mathematics community as positive evidence
that your proposed system will work. But in reality not only is it not
evidence that your system will work, it is not even a *test* of your
hypothesis: For, virtually without exception, all Physics/Maths ArXiv
authors are doing now *exactly* the same thing they always did, even
before the online era, namely, submitting their preprints to (classical)
peer-reviewed journals for refereeing and publication. They continue to
prepare their preprints in anticipation of answerability to classical
review, they continue to revise in response to the referees'
recommendations until the editor indicates the draft is acceptable.

Nothing has changed. And alternative (3) has not been tested.

Hence that valuable use to which the physics/maths community puts its
pre-peer-review preprints is in no way evidence for what that literature
would look like if there were an alternative form of peer review, or
no peer review at all; it is merely evidence of the great value there
is in self-archiving not only one's peer-reviewed postprints (which these
authors do also self-archive, of course, whenever there is a nontrivial
difference between the preprint and the postprint), plus evidence of
the enhanced value of also self-archiving the pre-peer-review preprints
(hence the secondary goal of BOAI-1).

In other words, the self-archiving physics/mathematics community gets
all the historic credit for being the first research community on the
planet to discover and derive the benefits of open access through the
self-archiving of both preprints and postprints, but (in the familiar
old caveat about needing to distinguish between what Simon-says and
what Simon-does) these early self-archivers have not always been
the best or most objective interpreters of what it is that they are
actually doing! Their interpretations are highly speculation-laden,
whereas the objective data are quite clearcut: Nothing changed, except
that the preprints and postprints were made openly accessible online
through self-archiving.

> Basically, I'm working on BOAI 2. I dont think BOAI 1 can be
> effective without BOAI 2.

I think you are working on a third option, rather more complicated (and
hypothetical) than BOAI-2. But you have not yet explained why you think
BOAI-1 cannot be effective without your option 3: Is it not the case that
universal self-archiving would provide immediate universal open access?

(This is not the place to speculate about what it is about human nature
that is making authors so slow about doing self-archiving, especially
because they are at least as slow about BOAI-2, and there the further
complication of first having to create the journals for them to switch
allegiance to slows things still further.)

Of course I expect that in the end, it will all converge on BOAI-2 --
i.e., all journals will merely be peer-review service-providers,
certifying the outcome with the journal-name, an authenticated meta-data
tag on the self-archived draft (the postprint) in an OAI Archive. But
we have to get there from here, and it seems obvious that self-archiving,
already within reach if the research community will just go ahead and
do it, is the nearest, most direct, shortest and surest path to get
there. It is BOAI-1 that will eventually induce the 20,000 toll-access
journals to downsize to BOAI-2 (or risk losing their titles and authors
to leaner BOAI-2 competitors).

> My strategy for BOAI 2 is to persuade people
> to put the digital representations of what we commonly regard as public
> domain knowledge in intelligently organized open archives that people have
> confidence in and think are worth copying over.

Note that a necessary component in what it is that you hope to persuade
people to do includes persuading them to self-archive in the first place
(i.e., BOAI-1), both their preprints and their peer-reviewed postprints
(the latter already being a body of "domain knowledge that people have
confidence in," but alas many lack access to, because of access-tolls).

But are you not merely increasing the burden of persuasion by trying to
persuade them of even more -- and not just the self-archivers, but those
who need to create all the other resources on which your option 3 depends?

> Then you can get
> tremendous usage of the open archives from the intelligent linking which
> adds great value. Problem with just having the research journals
> open archived is they are not well connected. They are the leaves on the
> forest of knowledge. The stems branches and trunks are currently missing
> in cyberspace, though being appropriated by commercials making online
> encyclopedias. I say, make a huge high quality open access encyclopedia
> of knowledge in a field, then everyone will start using it, and there
> will be big pressure to free the research journals.

I'm all for your enriched links! But don't you think that having
a citation-interlinked open-access version of the existing 20,000
peer-reviewed journal literature (and the enhanced visibility and impact
it brings) should be benefit enough, to persuade people to self-archive
in the first place? You can't go ahead and create any of your proposed
added-values without first having a sizeable chunk of that literature
up there, self-archived, in the first place! Without that it's
boot-strapping on a kiss and a promise!

And getting *that* chunk up there is what this is all about!

Cheers, Stevan

Harnad, S. (1998/2000) The invisible hand of peer review. Nature
[online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible/invisible.html
Exploit Interactive 5 (2000):
http://www.exploit-lib.org/issue5/peer-review/

Lawrence, S. (2001) Free online availability substantially increases a
paper's impact. Nature Web Debates.
http://www.nature.com/nature/debates/e-access/Articles/lawrence.html

NOTE: A complete archive of the ongoing discussion of providing open
access to the peer-reviewed research literature online is available at
the American Scientist September Forum (98 & 99 & 00 & 01 & 02):

    http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
                            or
    http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/index.html

Discussion can be posted to: american-scientist-open-access-forum_at_amsci.org

See also the Budapest Open Access Initiative:
    http://www.soros.org/openaccess

the Free Online Scholarship Movement:
    http://www.earlham.edu/~peters/fos/timeline.htm

the OAI site:
    http://www.openarchives.org

and the free OAI institutional archiving software site:
    http://www.eprints.org/
Received on Sun Dec 15 2002 - 13:13:22 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:46 GMT