On Fri, 30 Jul 1999, Ransdell, Joseph M. wrote:
> The responses to the E-biomed proposal are preponderantly affirmative
> and strongly enough so that if Varmus has been testing the waters for
> support he would seem to have no reason to hesitate in implementing a
> revised version whenever he thinks enough time has passed to do so.
> Hopefully, it will be modified in light of the flaws Stevan pointed out
> in his critique of it, chiefly by a correction of its mistaken aim of
> undertaking journal reform, as distinct from providing facilities
> supporting journals in going on-line properly.
This is incorrect. The correction was not just to drop journal (peer
review) reform and "instead provide facilities for journals to go on-line
properly"! The former is correct, but the latter is almost as garbled as
the (remediably) garbled portions of the first E-biomed draft.
http://www.nih.gov/welcome/director/ebiomed/com0509.htm#harn45
My recommended correction was to drop peer review reform AND to make it
explicit that a SELF-ARCHIVE was precisely what E-biomed was to be (in
the first instance), exactly as LANL is a self-archive. AND, most
important of all (and systematically not taken into account in
any of Joseph's comments), a self-archive not only for the unrefereed
preprint literature but for the REFEREED reprint literature (exactly as
LANL is, and has been from its very inception, as Paul Ginsparg's
recent posting has reminded us).
The effect of the latter is that the self-archive frees the refereed
journal literature (i.e., makes it accessible to everyone online without
they or their institutions having to pay
Subscription/Site-License/Pay-Per-View [S/L/P/] access tolls to get it).
This effect is certainly not correctly described as "providing facilities
supporting journals in going on-line properly"! It does, however,
prepare the way for a later, collaborative stage of journal overlays,
where the journals can officially authenticate the refereed drafts as
such. I explicitly stressed, however, that this collaboration cannot be
presupposed or even expected before the "subversive" effects of
self-archiving have prepared the way (by bypassing S/L/P for this
special literature, which is, and always has been, intended by its
authors as a give-away literature, and not a fee- or royalty-bearing
one).
http://www.arl.org/sc/subversive/
Why does Joseph mis-state this? I think it is because of our
disagreement about the catastrophic drop in quality standards that
I believe would result if peer review were abandoned in favour of a
self-archived "vanity press." This is not, and never has been, what I
think self-archiving is all about. The simple proof is the self-archived
REFEREED literature itself, which is simply a free give-away of the current
S/L/P-based journal literature by its authors. This is NOT a vanity
press! It is simply the journal literature, online, without a price tag.
Joseph's imagination is taken up with the OTHER side of self-archiving,
the unrefereed preprints. These are a wonderful, indeed revolutionary
supplement to the classical peer-reviewed canon -- and, as I have argued
elsewhere, even THEY are not quite a vanity-press either, because of the
"invisible hand" of peer review: it is in the expectation of being
answerable to the peer review that virtually all of these unrefereed
preprints have been drafted, and indeed most of them are
formally submitted to journals simultaneously with being self-archived as
preprints (and the refereed, accepted final drafts are swapped or added
as soon as they are available in most cases).
http://www.ecs.soton.ac.uk/~harnad/nature2.html
Nevertheless, a self-archive's unrefereed sector alone can be correctly
described as a kind of "interim vanity press" -- but with the knowledge
and expectation that it is only an embryonic stage along a continuum
which will eventually be clearly marked by the
quality-controlled/certified milestone of the accepted, refereed draft.
(The current average latency in LANL is about 11 months, between the
preprint and the reprint, as Les Carr <lac_at_ecs.soton.ac.uk> will soon
be reporting in a paper.) Nor does the continuous, interactive, and
self-corrective process of learned inquiry come to an end with the
certified refereed version, for there is still the possibility of
self-archiving updated/corrected revised drafts, as well as critical
commentaries and responses, all linked to that certified version.
http://cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad90.skywriting.html
Not to mention subsequent self-archived refereed articles (by the author
and others), citation-linked to the original one. All this is the world
opened up by self-archiving (and not just the "vanity press" that Joseph
mis-describes me as calling it!)
http://www.ecs.soton.ac.uk/~harnad/citation.html
> But supposing that
> happens, will the journals actually take advantage of such facilities?
> Perhaps some will, but I doubt that this will happen to anything like
> the extent wanted if nothing more is done than to provide archival
> support for that as well as for self-archiving by authors.
As stated in the original critique, there will initially not be much
incentive for journals (especially those published by commercial
publishers) to collaborate with E-biomed (other than the need to plan
ahead and face reality in the form of what is clearly the optimal and
inevitable solution for science and scientists).
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad97.learned.serials.html
But if/when the subversive effects of freeing the literature through
self-archiving begin to make themselves felt as declining S/L/P demand,
then publishers will certainly want to adapt in such a way as to retain
a niche (which I have predicted will be through downsizing to provide
only the service of quality-control/certification, QC/C, funded not
through access-blocking reader-institution end S/L/P charges but
through up-front author-institution end publication charges -- leaving
the archiving to E-biomed, and merely providing an official overlay to
authenticate the archive's refereed sector).
So Joseph has missed the point here.
> More generally, in spite of increasing evidence of popular support
> across an impressive range of interested parties for what I will call
> "The Harnad Initiative" to free the professional literature, there seems
> to me to be little reason to think that the attempts to implement it by
> providing the archives for it will have the success hoped for.
Does the following evidence of the spectacular success of LANL not
count as reason to expect success (if the desirability of a free online
journal literature for science and scientists is not reason enough)?
http://xxx.lanl.gov/cgi-bin/show_monthly_submissions
http://xxx.lanl.gov/cgi-bin/show_weekly_graph
> The
> archives should be built, in any case, as they will have some use and
> significant change will eventually come about, no doubt, but I wouldn't
> bet on much immediate use of it that betokens a change in publication
> practices. Why? Because the migration on-line is conceived thus far as
> depending on self-archiving, and there is no reason to think that people
> are presently motivated to do that, nor has anything been suggested or
> planned that might provide some incentive.
If researchers are motivated to have their research read by all who
want to read it, if they are motivated by the desire to make an impact
with their research (on both subsequent research and on citations), and if
their long-standing willingness to self-supply paper offprints to all
who ask is still in vigour, then all disciplines will take to
self-archiving, just as Physicists have done.
Of course, there's no second-guessing how quickly the rest of the
scholarly/scientific thoroughbreds will stoop to drink from the waters
of self-archiving: but that's no reason not to lead them there.
> The idea that if an archive is provided then it will be used has no
> evidence in its favor, as far as I know, and if there really was some
> general propensity for people to self-archive whenever the opportunity
> presented itself that would surely have shown itself by now.
Vide supra. I hope you are wrong about the scholarly/scientific
community's agnosia about what is optimal for it, but of course there is
a logical possibility that, historically speaking, Physicists will be
the only ones who ever twig on this! (I rather doubt this, though; they
may be smarter than the rest of us, but not THAT much smarter...)
> There are
> too many special considerations in connection with the LANL archive to
> make the continuing increase in use there evidential for some general
> trend toward going on-line across the board in academia, and I don't see
> any real indications of this happening elsewhere, with possibly some
> spotty exceptions here and there.
This sounds like a classical hedge; let us solemnly hope that those
subversive spots keep growing!
http://www.acm.org/repository/
http://cogprints.soton.ac.uk/
> Of course, it doesn't help the cause of self-archiving for the chief
> proponent of the practice to label it as resorting to the "vanity
> press", but although Stevan keeps shooting himself in the foot with
> that, I don't think that is at the root of the problem.
Not only is it not at the root of the problem, it isn't even true! It
is Joseph who is here calling self-archiving "vanity press," whereas I
call the self-archiving of refereed papers "freeing the refereed
literature" -- which is the antithesis and antipode of vanity press!
(Only the self-archiving of unrefereed papers is vanity press, and only
if it stops there, rather than going on, as most papers do, to pass
through peer review into the journal canon.)
> What is at the
> root of it is, I think, a failure to understand the role of EDITORS in
> the publication process, which has been obscured by the mistaken
> conflation of the editorial function with the function of peer review.
It would be an odd circumstance indeed if I, who have been editing
a major refereed journal for over two decades now, suddenly became
agnosic to that fact (or confused refereeing with editing).
http://www.princeton.edu/~harnad/bbs/index.html
http://www.princeton.edu/~harnad/psyc.html
> Editors tend to be self-effacing, and there seems to be a common
> (mis)understanding that because editors only have a "service" function
> as mediators they are not important. But it would be much closer to the
> truth to say that editors are the true rulers of academic life because
> they are found everywhere, at all of the gates of communication, opening
> or closing them according to judgments which hardly anyone ever thinks
> to question. Not important? Hey, think again! But why, in all of the
> discussion of "decoupling" of functions are editors not discussed?
Because this discussion is not about peer review reform but about
freeing the peer reviewed literature, such as it is!
Peer review and the role of editors is eminently ripe for empirical
investigation, but that is a different topic:
"Neither the editor nor the referees is infallible. Editors can err
in the choice of specialists (indeed, it is well-known among
editors that a deliberate bad choice of referees can always ensure
that a paper is either accepted or rejected, as preferred); or
editors can misinterpret or misapply referees' advice. The referees
themselves can fail to be sufficiently expert, informed,
conscientious or fair." [Harnad 1998h: see references at bottom of
this comment]
Joseph's commentary goes on to a display of animus against both peer
review and university administrators:
> ... pseudo-glorification of the peer reviewer... Validating and
> certifying and putting stamps of approval on documents is the sort of
> thing they used to do at the Vatican -- or maybe they still do, since
> the Pope is still officially infallible -- and in Moscow, too, up to a
> decade or so ago. But in the secular sciences of the free world?...
> ... the administrative view that the research universities are knowledge
> factories, producing and selling knowledge, with the faculty regarded as
> workers on the production line.
I won't comment on any of this. I do think I recognize (from 20 years'
of editing) the core of Joseph's grievance. It is the single aggrieved
author's viewpoint (analogous to the single aggrieved student's
viewpoint, when he feels that a test has not been a proper measure of
his proficiency or performance).
Such grievances are not to be taken lightly, because the system (both
peer review and tests/exams/marking) are indeed fallible and imperfect.
But the real question is one of scale. Every student hopes to have his
every thought and action weighed in a blind, omniscient and infinitely
fair absolute balance. In reality, all population-based measures are
approximate and have a margin of error and even bias. The objective is
to minimize that error and bias, within available resources, and to
remedy detected cases of error where possible.
The rest is down to testing and designing systems that work at least as
well as the current ones (for a population at least as large). It is
certainly no solution to focus on known or perceived cases of
misevaluation, and to simply propose scrapping the evaluative system on
their basis!
> A peer reviewer in any field is a presumptive equal of the person
> whose work is being reviewed, and that means that there is no
> presumptive superiority in status that makes the peer reviewer's view
> right and the reviewee's view wrong when they disagree. If the
> disagreement is a simple contradiction one is perhaps right and the
> other wrong; but there is nothing in the conception of a peer or of a
> reviewer that can justify regarding the peer reviewer as being in a
> favored position when disagreement occurs or which would turn an
> agreement in opinion of the two into a validation of the one by the
> other. The conclusion of a peer review is just a second opinion, that's
> all.
A competent editor knows all this, and is dealing with it in every case.
It is only the aggrieved author who sometimes feels (and sometimes with
justification) that he has been ill-used, and that an incompetent
referee's judgment has blocked his submission.
Competent, conscientious editors (who cannot be specialists in all
areas) are responsive to authors' rebuttals of referee reports. They
may betoken a faulty choice of referees, or shortcomings in the referee
reports. But they may also betoken defensiveness on the part of the
author, or unwillingness to do the work required to make a paper
ship-shape. I know of no way to automate or replace editorial judgment
here, but we should certainly keep testing new ways of strengthening
it. What is certain is that no concrete or practical alternative (let
alone one that has been tested and shown to do at least as well as the
present system) has been proposed by Joseph Ransdell here!
> why do we
> keep talking as if formal peer review is the key to legitimization in
> the sciences and elsewhere? Willingness of peers to criticize and
> openness and responsiveness to peer criticism is what provides the
> critical self-control of the process of inquiry through corrective
> negative feedback -- it is this process itself, not individual persons
> and judgments, that regulates inquiry overall and legitimates it as
> science or scholarship -- but there are many different ways in which
> corrective feedback loops can and do occur in the course of inquiry.
> Formal peer review, set up for certain special purposes, including
> journal publication, is one of them but should not be fixed upon so
> exclusively as to blind us to the other ways critical self-control
> functions in the professional communication of scientists, and should
> not be allowed to mislead us into thinking that the sciences depend for
> their validity on anybody wielding stamps of approval.
It is hard to extract the substantive point in all this: Peer review is
certainly not the only self-corrective mechanism of Learned Inquiry.
Informal peer feedback before publication, formal and informal peer
feedback after publication, and the further march of Learned Inquiry
itself, attempting to build upon published ideas and findings
(especially in empirical science), are all parts of this collective,
cumulative, self-corrective, and, one hopes (where appropriate),
convergent system too.
But at the present scale of publication, if the classical peer review
at its core were removed (at the point of formal -- i.e.
refereed-journal -- "publication") and only the rest were left in
place, I believe it would all soon devolve into anarchy, human nature
being what it is (at the population level), when it is not directly
answerable to quality standards -- until peer review was simply
rediscovered or re-invented as the simple solution for the triage of
all that growing body of unregulated and unnavigable human noise!
I could be wrong, but surely this all has to be tested first, locally,
and in a controlled way! So, for the moment, as mentioned before,
classical peer review should proceed apace, until further empirical
notice, and we should focus instead on freeing the peer-reviewed
literature, such as it is.
> If you re-read the Phelps document
> which is ancestral to the Caltech proposal, for example, you will find
> that Phelps' understanding of what is wanted in promoting on-line
> publication practices supposes that the "certification" function of
> publication (which he also regards as the "credentialing" of the author)
> can be cleanly decoupled from the distribution function, so that one
> need merely set up a pool of peer reviewers -- the first generation of
> which are Unreviewed Reviewers whose superior quality is guaranteed by
> The Self-chosen Chooser, apparently -- who can be called upon to perform
> the operation of quality control on documents submitted to them,
> assessing them as fit or not fit to print.
I don't wish to defend any untested armchair schemes for reforming peer
review, whether Ransdell's or Phelps's (indeed I made the same
recommendation for the Scholar's Forum archiving initiative as for the
E-biomed archiving initiative: Dissociate them completely from peer
review reform schemes). Joseph is, I think, just lapsing here into the
conspiratorial view about both administrators and peer reviewers that
he has repeatedly expressed in September-Forum. Again, he may have some
legitimate grievances, but he has no realistic or relevant alternative
to offer, let alone a tested one.
http://library.caltech.edu/publications/ScholarsForum/042399sharnad.htm
http://amsci-forum.amsci.org/archives/september-forum.html
> The editorial function was forgotten
> just as it has been forgotten in the ongoing discussion here by being
> reduced, in effect, to a stamp of authority wielded by the peer
> reviewers, who have been assigned the bogus task of being the official
> validators of the entire scientific process.
Nonsense. The function of a competent, conscientious, answerable editor
is part of the very meaning of the classical peer review system! It
never was a disembodied stable of "peers" to which papers were
dispatched willy-nilly for their box-score votes.
> I suggest, then, that the reason nothing is happening in academia as
> regards the migration on-line...
The premise is false. A good deal is happening because of the LANL
initiative, and, one hopes, a good deal more will be happening thanks
to the E-biomed and Scholar's Forum initiatives.
> is that editors see no future for
> themselves in it and therefore are not about to change the
> communicational arrangements researchers live by; and nothing will be
> happening until the editors do see something worth running the risk they
> run in going on-line, which may very well result in a diminishment of
> their importance if attention is not paid to it.
I am afraid that this too makes no sense. It is up to AUTHORS, not
editors, to self-archive; not even the journal's copyright policy on
self-archiving is in the editor's jurisdiction -- although they can take
a position on it, and have: cf. Editors Blume and Bloom of APS and AAAS,
respectively:
http://www.ecs.soton.ac.uk/~harnad/science.html
To repeat, journals are almost all "going online" already. That is
trivial and a foregone conclusion. What is at issue here is whether the
only online version should be the one held hostage behind the
access-denying financial firewalls of S/L/P, or there should be an
alternative give-away, author self-archive:
http://www.ecs.soton.ac.uk/~harnad/nature.html
> That their fear of
> being diminished or even eliminated is a reasonable one is evident from
> the following remark of Stevan's in his response of June 27th to the
> immunologists:
>
> sh> Now there is no doubt whatsoever that this service
> sh> will force the established journals to restructure themselves
> sh> in certain ways. (My own prediction would be that it will
> sh> make journals scale down to providing only the service of
> sh> peer review and authentication, . . . .
>
> The editor has just disappeared, it seems.
Nothing of the sort. The Editor's role is PRECISELY the same as it
always was in classical peer review, which is not changed by a single
epsilon -- at least according to my own recommendations...
This ends my comment. The references promised above follow.
Harnad, S. (1979) Creative disagreement. The Sciences 19: 18 - 20.
Harnad, S. (ed.) (1982d) Peer commentary on peer review: A case
study in scientific quality control, New York: Cambridge University
Press.
Harnad, S. (1984d) Commentaries, opinions and the growth of
scientific knowledge. American Psychologist 39: 1497 - 1498.
Harnad, S. (1985f) Rational disagreement in peer review. Science,
Technology and Human Values 10: 55 - 62.
Harnad, S. (1986) Policing the Paper Chase. (Review of S. Lock, A
difficult balance: Peer review in biomedical publication.) Nature
322: 24 - 5.
Harnad, S. (1990d) Scholarly Skywriting and the Prepublication
Continuum of Scientific Inquiry. Psychological Science 1: 342 - 343
(reprinted in Current Contents 45: 9-13, November 11 1991).
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad90.skywriting.html
Harnad, S. (1996a) Implementing Peer Review on the Net: Scientific
Quality Control in Scholarly Electronic Journals. In: Peek, R. &
Newby, G. (Eds.) Scholarly Publishing: The Electronic Frontier.
Cambridge MA: MIT Press. Pp. 103-118.
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html
Harnad, S. (1998) Learned Inquiry and the Net: The Role of Peer
Review, Peer Commentary and Copyright. Learned Publishing 4(11):
283-292 Shorter version in 1997: Antiquity 71: 1042-1048 Excerpts
also appeared in the University of Toronto Bulletin: 51(6) P. 12.
http://citd.scar.utoronto.ca/EPub/talks/Harnad_Snider.html
Harnad, S. (1998h) The invisible hand of peer review. Nature
[online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible.html
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 2380 592-582
Computer Science fax: +44 2380 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
ftp://ftp.princeton.edu/pub/harnad/
Received on Wed Feb 10 1999 - 19:17:43 GMT