The responses to the E-biomed proposal are preponderantly affirmative
and strongly enough so that if Varmus has been testing the waters for
support he would seem to have no reason to hesitate in implementing a
revised version whenever he thinks enough time has passed to do so.
Hopefully, it will be modified in light of the flaws Stevan pointed out
in his critique of it, chiefly by a correction of its mistaken aim of
undertaking journal reform, as distinct from providing facilities
supporting journals in going on-line properly. But supposing that
happens, will the journals actually take advantage of such facilities?
Perhaps some will, but I doubt that this will happen to anything like
the extent wanted if nothing more is done than to provide archival
support for that as well as for self-archiving by authors.
More generally, in spite of increasing evidence of popular support
across an impressive range of interested parties for what I will call
"The Harnad Initiative" to free the professional literature, there seems
to me to be little reason to think that the attempts to implement it by
providing the archives for it will have the success hoped for. The
archives should be built, in any case, as they will have some use and
significant change will eventually come about, no doubt, but I wouldn't
bet on much immediate use of it that betokens a change in publication
practices. Why? Because the migration on-line is conceived thus far as
depending on self-archiving, and there is no reason to think that people
are presently motivated to do that, nor has anything been suggested or
planned that might provide some incentive.
The idea that if an archive is provided then it will be used has no
evidence in its favor, as far as I know, and if there really was some
general propensity for people to self-archive whenever the opportunity
presented itself that would surely have shown itself by now. There are
too many special considerations in connection with the LANL archive to
make the continuing increase in use there evidential for some general
trend toward going on-line across the board in academia, and I don't see
any real indications of this happening elsewhere, with possibly some
spotty exceptions here and there. I, personally, have a great deal of
interest in this happening and am usually optimistic to a fault, but I
don't perceive anything to make me feel optimistic about this though I
am certainly on the lookout for it.
Of course, it doesn't help the cause of self-archiving for the chief
proponent of the practice to label it as resorting to the "vanity
press", but although Stevan keeps shooting himself in the foot with
that, I don't think that is at the root of the problem. What is at the
root of it is, I think, a failure to understand the role of EDITORS in
the publication process, which has been obscured by the mistaken
conflation of the editorial function with the function of peer review.
In reading back through the discussion in the AMSCI forum since its
beginning nearly a year ago, I could find hardly any mention anywhere of
the function of editors. Perhaps it is assumed that editors are somehow
functionally the same as publishers, but they certainly are not, and
they are not functionally identical with peer reviewers either.
Editors tend to be self-effacing, and there seems to be a common
(mis)understanding that because editors only have a "service" function
as mediators they are not important. But it would be much closer to the
truth to say that editors are the true rulers of academic life because
they are found everywhere, at all of the gates of communication, opening
or closing them according to judgments which hardly anyone ever thinks
to question. Not important? Hey, think again! But why, in all of the
discussion of "decoupling" of functions are editors not discussed?
I suggest that this oversight stems from the way peer review is so often
talked about here, which involves a kind of mystification wherein the
editor's role disappears in the pseudo-glorification of the peer
reviewer. What I have especially in mind is the idea, frequently
expressed here, that peer review processes are used to "validate"
research claims and that refereed journals "certify" research results by
accepting the paper for publication or place upon the research report
the equivalent of a "stamp of approval" in virtue of this
"validation".
Now wait a minute! This is surely just rhetorical exaggeration that
results in something false if taken literally. Validating and
certifying and putting stamps of approval on documents is the sort of
thing they used to do at the Vatican -- or maybe they still do, since
the Pope is still officially infallible -- and in Moscow, too, up to a
decade or so ago. But in the secular sciences of the free world? No, I
don't think so, or at least this does not occur as a part of the
research process.
Some of the confusion may be a result of the widespread acceptance of
the administrative view that the research universities are knowledge
factories, producing and selling knowledge, with the faculty regarded as
workers on the production line. From this perspective, there is some
reason for putting stamps and certifications on its products -- provided
it is understand that something thus validated or certified can turn out
to be false, that what is labeled knowledge is not necessarily
knowledge, and that the stamp can be more questionable than what is
stamped. Caveat emptor! (But it is usually worth it, no doubt, or
they wouldn't be buying it.) But looking at research activity from the
point of view of the researcher, for whom research results are of value
not as they function in applications but rather as they are fed back
into the process of inquiry itself in such a way as to augment the basis
for further inquiry -- which is surely the view that should be taken by
the faculty of a research university -- the function of peer review is
not to validate, certify, or approve of results.
Do people in the hard sciences really believe that this sort of
"certification" of scientific findings occurs and that this is what
validates science as an activity and regulates it critically? I hope
not. A peer reviewer in any field is a presumptive equal of the person
whose work is being reviewed, and that means that there is no
presumptive superiority in status that makes the peer reviewer's view
right and the reviewee's view wrong when they disagree. If the
disagreement is a simple contradiction one is perhaps right and the
other wrong; but there is nothing in the conception of a peer or of a
reviewer that can justify regarding the peer reviewer as being in a
favored position when disagreement occurs or which would turn an
agreement in opinion of the two into a validation of the one by the
other. The conclusion of a peer review is just a second opinion, that's
all.
This is not a radical anarchist doctrine but something well understood
in practice by any competent editor who draws upon the report of a peer
reviewer in deciding what to do about the paper under consideration for
publication. Frequently, the editor has chosen the peer reviewer to
begin with, but the editor is not, in any case, required to accept the
reviewer's conclusion. Indeed, I am sure that in the sciences, as
elsewhere, there will be times when the editor regards the reviewer as
less credible than the reviewed.
In any case, it is not the peer reviewer who stamps approval on the
document, or who certifies it or validates it. No peer in a field has
the right or the power to do that. That is what is meant in calling it
PEER review. Let that understanding lapse and you are just a worker in
the knowledge factory, inferior to anybody your boss chooses to put over
you as your superior. That's Animal Farm, where some peers are more
equal than others.
So what about the editor, then? Well, the editor just decides what to
do about the paper in view of the various opinions from peer reviewers
(and perhaps other sorts of reviewers as well) taken together with a
number of other considerations which it would be useless to try to
itemize in the abstract because they are matters of judgment. Is the
editor's decision to publish the piece a "validation" or "certification"
or "stamping of a seal of approval" on it? Of course not. Editors are
not invested with powers that peer reviewers lack that would give them
such an office of authority. They have a different and much more
complex job to do.
Journals have missions, more or less vaguely understood and sometimes
(though not always) these are written down somewhere in a description of
the role the journal is thought of as playing in the profession. The
conscientious editor -- who may be the author of the mission statement
-- tries to make decisions that maximize the many values involved in
light of the overall aim. No "validation" or "certification" or "stamp
of approval" can be derived from such a procedure.
So why do we keep talking as if it were? More generally, why do we
keep talking as if formal peer review is the key to legitimization in
the sciences and elsewhere? Willingness of peers to criticize and
openness and responsiveness to peer criticism is what provides the
critical self-control of the process of inquiry through corrective
negative feedback -- it is this process itself, not individual persons
and judgments, that regulates inquiry overall and legitimates it as
science or scholarship -- but there are many different ways in which
corrective feedback loops can and do occur in the course of inquiry.
Formal peer review, set up for certain special purposes, including
journal publication, is one of them but should not be fixed upon so
exclusively as to blind us to the other ways critical self-control
functions in the professional communication of scientists, and should
not be allowed to mislead us into thinking that the sciences depend for
their validity on anybody wielding stamps of approval.
Let me quote something from Arthur Smith in support of this. It is in
response to the following remark by Stevan Harnad:
sh> Its not the prestige of the _journal_ that is important, it is
sh> the seal of approval which is given by the referees to the
sh> _authors_ work. The journal is the tail not the dog.
Smith says:
as> Those are nice theories, but I doubt them in practice.
as> Editors do not just apply some mathematical algorithm
as> to referee reports to determine what to do with a
as> paper - there is a difficult intellectual problem (just as
as> with this discussion here) to decide who is actually correct
as> and who is not, if there has been mis-communication, all
as> based on the editor's knowledge of the referees, of the
as> field, and a reading of the paper. The editor's decisions are
as> shaped by the journal policies, the editors themselves are
as> chosen by the publisher following various criteria. It is
as> important to remember that most journals do not just
as> select papers based on their "correctness" but on a
as> number of other qualities that determine the
as> characteristics of the journal as a whole, and may request
as> revisions of the work based on those criteria. And the referees
as> themselves apply different criteria depending on the journal.
as> "Refereed" and "accepted by Nature" are two quite different
as> things. That's not to say the referees are unimportant - their
as> is central. But prestige is not just the referees - I
as> refereed for Phys Rev Letters as a graduate student and would
as> have laughed hysterically if you'd told me PRL was the tail and
as> I was the dog...
Message of Sept 1, 1999 AMSCI SEPTEMBER FORUM
HTTP://AMSCI-FORUM.AMSCI.ORG/scripts/wa.exe?A2=ind98&L=september-forum&D=1&O=A&F=Plf&P=18987
There are different stories to be told for different disciplines, I am
sure, but what Arthur Smith says above is all the more true in
philosophy, where the journals are not chosen by the publishers or by
the discipline wide professional society but typically started up when
individuals or small groups of people feel that some area or focus of
interest is not adequately represented in the existing literature. The
idea that the prestige of a journal is a simple matter of higher or
lesser degrees of quality, and that the job of the peer reviewer is to
ascertain that degree and thus provide the content for an editor's act
of accepting or rejecting the paper by publishing or not publishing it,
fits the reality here so poorly as to be a falsification of it.
To the extent that quality control is involved, as of course it is, that
is an editorial function, not the task of the peer reviewer, who is
rarely in position to make overall judgments like that and is never
expected to do so by the editor, anyway, because the editor cannot
transfer his decisional powers to the peer reviewer in that way; and
even the editorial decision is rarely made apart from a number of other
considerations having to do with the understood purpose of the journal
rather than the quality of the paper. In fact, it is even possible
that an editor could at times choose to publish a paper which the editor
as well as the reviewer regards as being seriously flawed, but flawed at
a level not amenable to editorial repair, and do so because of the
mission of the journal.
Let me give an example. The philosopher/psychologist William James was
one of the sloppiest reasoners you are likely to find anywhere in print,
and even his most ardent devotees will usually admit this, but will also
point out, quite correctly and cogently, that his work has a
extraordinary ability to get at what is important partly in spite of and
partly because of his disdain for and rather feeble grasp of logical
form. A good editor would understand that and, in spite of strong
disagreement with James' view and with full awareness of the
subprofessional logical character of some of his work, publish it as it
stood anyway, and do so without feeling any need to insert churlish
explanatory editorial comments about its weaknesses. Why? Because the
aim of the editor is to promote the pursuit of truth in the field the
journal serves, and this is not done by providing editorially certified
documents or editorially sanitized ones either, but by providing
documents which, in the editor's judgment, will be likely to have a
salutary function in the process of inquiry in the field the editor is
in the service of. Yes, of course, the editor will clean things up and
improve them when feasible, but if the editor is good he or she will
always finally rely on balanced judgment in deciding when to publish and
when not.
The importance of understanding the editorial role, in addition to the
peer reviewer's role, becomes especially clear when the question of
decoupling of functions arises. If you re-read the Phelps document
which is ancestral to the Caltech proposal, for example, you will find
that Phelps' understanding of what is wanted in promoting on-line
publication practices supposes that the "certification" function of
publication (which he also regards as the "credentialing" of the author)
can be cleanly decoupled from the distribution function, so that one
need merely set up a pool of peer reviewers -- the first generation of
which are Unreviewed Reviewers whose superior quality is guaranteed by
The Self-chosen Chooser, apparently -- who can be called upon to perform
the operation of quality control on documents submitted to them,
assessing them as fit or not fit to print. (And, one is led to assume
in context, grading them in respect to degree of excellence so that one
could in time compute the exact productive value of a researcher by use
of a simple algorithm that averaged paper grades and quantity thereof
across years of service.) Separate arrangements would then be made for
electronically archiving the passing papers, and thus for distributing
the documents in that sense. The lengths to which this sort of thinking
can take one are suggested by Phelps' response to the following imagined
objection about the effect of charging a submission fee for having one's
paper reviewed for purposes of certification:
> -- The submission fee will discourage the publication of some
> work now reaching print, particularly if the process (as
> envisioned) evolves into a world with fewer paper print outlets.
Phelps' response:
> This concern confuses the processes of publication and
> certification, which have heretofore been linked, but can now
> be decoupled. The process of distributing manuscripts will be
> greatly enhanced by using a computer-based system.
> Distribution of scholarly ideas will be enhanced. What will
> possibly diminish is the volume of certified articles.
>
> While few would probably acknowledge the thought in person,
> it would be fair to report that many participants of the
> conference (and possibly many readers of this report) privately
> feel that there is TOO MUCH published material, much of it with
> little or no sustaining value, manuscripts published because of
> the pressures of the academic promotion and tenure process.
> The demise of lower-tier journals, if this is true, would be a
> welcome change in the academic landscape. Of course, one
> must ask why such journals continue to flourish in a free
> market economy such as that of the United States. The answer
> potentially lies in the resource allocation process of
> universities and colleges, wherein the journal subscription
> costs channel through a library staff who receive professional
> prestige and acclaim by supervising ever-larger collections.
> Indeed, the prestigious Association of Research Libraries
> heavily relies on collection size and additions to collections
> as a measure of a library's value and standing.
> From Charles Phelps, The Future of Scholarly Communication (June 1997)
http://www.econ.rochester.edu/Faculty/PhelpsPapers/Phelps_paper.html
That there is too much mindless pressure to publish is indisputable, but
note that it does not occur to Phelps -- or whoever it is whom his
thinking represents -- that in his scheme the pressure to write for
certification simply replaces the pressure to write for paper
publication, and is subject to being ratcheted all the higher by the
competition with others who will be attempting to take advantage of the
speed of electronic composition to raise their rate of productivity
higher than their competitors in order to keep their jobs. But what is
more disturbing than this unnoticed and unwanted nightmare of escalation
is the casual assumption that by eliminating "lower-tier" journals some
benefit is ipso facto achieved, as if it were just obvious that journals
are classifiable into the better and the worse, corresponding to the
more and the less prestigious, with what is worthless at the lower
levels so that nothing important would be lost by the elimination of
what is regarded by the peer reviewers as inferior. And what is even
more disturbing is the belief that the only way to account for the
obviously unwanted persistence of marginal thinking is in terms of
resource mismanagement, without so much as considering the possibility
that people with unfashionable ideas may actually believe in them enough
to try to communicate them to others, if any public facilities are
available for that purpose, much less considering the possibility that
there should be steps taken to insure that the unfashionable is not
simply silenced.
I am confident that Phelps didn't think about these implications not
because he has no appreciation of the values involved but because he was
blinded by his initial enthusiasm for this highly schematic vision of
functional decoupling and the benefits he thought he could see in that,
and didn't notice what had been overlooked in the imagined decoupling,
namely, the human factor of professional concern and judgment that is
located in the editorial function. This is what accounts for the eerily
inhuman quality of his account. The editorial function was forgotten
just as it has been forgotten in the ongoing discussion here by being
reduced, in effect, to a stamp of authority wielded by the peer
reviewers, who have been assigned the bogus task of being the official
validators of the entire scientific process.
This is unreal. Editors are typically far more important as
functionaries in any field than peer reviewers are. In fact, all
academic disciplines are governed at the most basic level by editorial
control of one kind and another, and this is even more true in the
humanities, where not only research publication but access to the basic
texts for the fields are sometimes mediated through editors who can hold
whole disciplines hostage by refusing to make the texts available until
they have put their imprimatur on them. Perhaps editors go unnoticed
because it is often all but impossible to put them into question, and
dangerous to try to do so. Nothing to be done about it, as Eeyore might
say, so no reason to think about it. Peer reviewers, by contrast, are
just ordinary researchers whose opinions carry no special weight when
they act as such.
I suggest, then, that the reason nothing is happening in academia as
regards the migration on-line is that editors see no future for
themselves in it and therefore are not about to change the
communicational arrangements researchers live by; and nothing will be
happening until the editors do see something worth running the risk they
run in going on-line, which may very well result in a diminishment of
their importance if attention is not paid to it. That their fear of
being diminished or even eliminated is a reasonable one is evident from
the following remark of Stevan's in his response of June 27th to the
immunologists:
> Now there is no doubt whatsoever that this service
> will force the established journals to restructure themselves
> in certain ways. (My own prediction would be that it will
> make journals scale down to providing only the service of
> peer review and authentication, . . . .
The editor has just disappeared, it seems. I think maybe we should try
to think of something useful for these people to do in the brave new
world because if they don't go there, we won't be going either. We can
begin by trying to get a better understanding of the role they are
actually playing now.
--
Joseph Ransdell <ransdell_at_door.net> or <bnjmr_at_ttu.edu>
Dept of Philosophy Texas Tech Univ. Lubbock TX 79409
(806) 742-3158 office 797-2592 home 742-0730 fax
ARISBE:Peirce Telecommunity http://www.door.net/arisbe
http://www.door.net/arisbe/homepage/ransdell.htm
Received on Wed Feb 10 1999 - 19:17:43 GMT