On Wed, 4 Aug 1999, Ransdell, Joseph M. wrote:
> SH> Now just what proposition of substance have I missed?
>
> The main point of substance is a claim on my part that your analysis of
> the publication process is faulty in not taking due account of the role
> of the editor in the process, owing to your mistaken understanding of
> the role of peer review.
One might think it odd, prima facie, that anyone who has edited a major
journal for 20 years has somehow failed to take due account of his own
role in the review process, but let us go on. One will be especially
interested to hear on what empirical basis this faulty understanding
will be corrected:
> The result of this is that your self-archiving
> initiative is based on an inadequate understanding of the problems that
> must be addressed if self-archiving is to be successful.
Again, creating a self-archive, collaborating with other self-archives,
and thinking and writing a good deal about the subject apparently
still makes for inadequate understanding; but one is all the more eager
to hear the empirical basis of the remedy that will no doubt follow:
> The error in your analysis is apparent in the message to which I am
> responding, where you say "Editors are a part of classical peer review.
> . .", whereas in fact peer review is normally a part of the editorial
> process, possibly performed by the editor him/herself acting as peer,
> possibly by one or more other persons specially commissioned for the
> task. The same mistake is implicit in the following passage from your
> response to the immunologists:
>
> sh> Journals will continue to be the publishers -- but their
> sh> role may well shrink to that of providing and then
> sh> certifying the quality control. The rest of publishing
> sh> will vanish (for refereed journals).
>
> What has also vanished here is the editor, who is reduced to providing
> for and then certifying the quality control exercised by the peer
> reviewer.
I regret again to have to say that this is nonsense, and that it
attests (one must be careful about not being ad hominem) to a mistaken
understanding of the how peer review is actually implemented by
journals.
I can only repeat: The editor is PART of the classical peer review
process, on which I regret I have not the time to write a primer here,
but it includes a preliminary review of the paper by the Editor, the
selection of qualified referees, the review of their referee reports and
the formulation of the disposition letter indicating what (if anything)
needs to be done to make the draft acceptable (if it is potentially
acceptable). [In some variants, there are editorial boards, action
editors, and collective meetings to select referees and decide on how to
act on the referee reports.] This refereeing process may have several
iterations with revised drafts.
ALL of this is PART of peer review, and would continue to be part of it
even if/when journals scaled down to providing only the service of
quality control/certification [QC/C]: That would still include editing
and refereeing.
To repeat. Joseph is making no substantive point here whatsoever; he is
simply making a category error. When I speak about FRUIT (peer review)
he says that's not enough, I'm forgetting about APPLES (editing).
I am afraid this is not advancing anything at issue here, be it (1) peer
review reform (which we have agreed to set aside) or (2) freeing the peer
reviewed literature through self-archiving (which is the focus of this
discussion) or even (3) the value of self-archiving unrefereed preprints
(an interesting, important side-issue).
> Well, not completely vanished perhaps: there is still the puff of smoke
> left by the mysterious operation of certification, here assigned to the
> editor rather than to the peer reviewer. However, this leaves it
> unclear whether you mean that the editor certifies the peer reviewer,
> who certifies the paper, or whether the editor certifies it directly,
> drawing upon the information about the paper provided by the peer
> reviewer. There is more than a quibble here since you are referring to
> a supposed office of certification that occurs somewhere in the
> publication process, and since the power of such an office and the
> conduct of its officer has serious consequences for authors -- their
> very jobs may be at stake -- one wants to be able to locate it
> precisely.
Would it not be a better use of the time of those following these
discussions if Joseph informed himself on the nature of peer review,
rather than asking me these questions here?
The process of certification may be mysterious to an inexperienced
author, but it is not mysterious to editors and referees, or even to
authors who have taken the trouble to inform themselves about the
(non-secret, non-occult) mechanism of peer review.
> Also, one wants to know how editors acquire such an office to begin
> with. Are there tests that editors must pass that qualifies them for
> wielding such power? Who composes the tests and certifies that the
> editors have passed them, and why are those persons given such power
> over the selection of editors? What are their qualifications? Surely
> power like this is not distributed on the basis of "old boy" preferences
> and the like. But then who prepares the tests for THESE people and thus
> validates them? You understand the logic of the argument here.
There is no argument here. These questions are worth raising in the
context of empirical inquiry on peer review (which this Forum is not).
They have answers, and the answers are not always optimal. Peer review,
in other words, could stand some reform. But this is irrelevant here; a
complete red herring. The focus here is on freeing the peer reviewed
journal literature SUCH AS IT IS through self-archiving. Peer review
reform is not part of the agenda, nor should the freeing of the
literature be made in any way contingent on peer review reform.
Joseph thinks the self-archiving initiative will fail. It is not clear
to me why he thinks it will fail (so far it seems to be doing remarkably
well), but worse, it is not clear to me whether he even knows what the
self-archiving initiative MEANS, for he seems to have a different
agenda, which I suspect is more like freeing the literature from the
tyranny of editors (= peer review) rather than freeing it simpliciter,
which is the agenda here.
> The answer is not "Oh, well, the old problem of who guards the guardians
> has been around for millennia and we can't be expected to solve that
> here." The answer is that there is no solution to that problem and the
> idea of certification is nonsense in this context for precisely that
> reason. There is no function of certification that occurs as part of
> the peer review or editorial process. There are no guardians of the
> type you think must exist because there can't be.
Q.E.D.
I regret (and now I am speaking ex officio as editor/moderator of this
American Scientist September-Forum) that I will not be able to accept
further postings that systematically attempt to co-opt the agenda of
this Forum in favour of a different agenda.
[Joseph: If you want to discuss the advantages and disadvantages of
"certification," please start your own list. This list is devoted to
freeing the "certified" literature, and it is not a contribution to this
discussion to express doubts about this initiative simply because
you have doubts about the value of "certification." Let us go about the
business of freeing the certified literature such as it is first; then
we'll worry about the short-comings of certification. To underscore how
orthogonal your endlessly repeated point is to this Forum, you might
just as well have addressed it to the proprietary paper medium in which
this certified literature appears currently: Tell the PUBLISHERS that
certification is of no value; they stand to save a lot of money if they
are persuaded. But please stop injecting it into this forum, where it is
irrelevant, and where it has in any case already been repeatedly aired.]
> Your conception of peer review is defective, involving an illicit
> introduction of anachronistic university administration functions into
> the inquiry process, where they have no business being.
This is utter nonsense. It is Joseph who is preoccupied with university
administrators. They are NOT part of peer review; editors ARE.
University administrators are, however, the payers of S/L/P access
tolls; and they also have interests vested in enhancing the impact of
their authors' research. So they definitely have, and should have, an
interest in the self-archiving initiative. But this does not make them
a part of the peer review mechanism, nor do they wish to become a part
of it. The conflation is entirely Joseph's, and has been so ever since
his very first posting about Steve Koonin's initiative in the Chronicle
of Higher Education Colloquy -- a discussion that, in my opinion,
Joseph's constant redirection onto his own agenda ultimately derailed and
brought to an end, whereas it should naturally have continued and segued
into the Scholar's Forum discussion.
I regret to have to say that I cannot and will not allow this to happen
with this Forum.
See:
http://www.chronicle.com/colloquy/98/copyright/re.htm
> Research
> science is not a creation of the universities, and is still only
> partially integrated within them, where it sits uneasily because its
> inveterately egalitarian ethic -- which is built into its
> communicational practices -- is at odds with the rank hierarchical
> structure of the universities. To see how much at odds they are you
> might consider the fact that both the LANL archives and the world wide
> web were created by science where it was not under control of the
> universities and ask yourself whether either of them could have been
> created in the universities.
All these initiatives are collaborative and involve both Universities
and Research institutions. Joseph's is accordingly a false opposition,
again driven by factors that one cannot delve into without inquiring
into motivation, which would be ad hominem. So let as just say this
Manichaean view of the dynamical factors underlying the remarkable
collaborative creativity that has emerged on the Net is off the mark,
and individuals like Paul Ginsparg and Tim Berners-Lee are far more
pertinent, whatever their affiliations, if one is bent on assigning
Aristotelean causality, than are their institutions.
> It may be impossible for universities to be other than they are. The
> staying power of the authoritarian structures of rank and privilege are
> certainly impressive, and I have no recommendations for reform along
> those lines myself because, after 30 years or more of trying to imagine
> what such a reform would be like, studying the situation from the point
> of view of an insider, I have been unable to come up with any viable
> alternative. (No, I have no grievance against universities: I have
> always been treated justly and respectfully and think it remarkable that
> people would actually be willing to pay me to do what I do.) Thus it
> may be that the tension between the implicit imperatives of science and
> those of the university are irreconcilable, which means that
> accommodation must continue to be sought rather than full agreement,
> assuming science is to continue to be supported by the universities.
> But then this has been the situation all along, as science has been
> partially institutionalized within the universities in this country
> beginning in the last century. But accommodation is one thing and
> corruption is another, and the process of research science is corrupted
> when authoritarian practices derived from the institution accommodating
> it, whether it be the university or the government or industry, is
> unwittingly allowed into it as part of its normal process. That is
> what is happening when peer review is misconstrued as a certification
> practice that occurs within the publication process.
Nolo contendere. But no more postings on this topic will appear on this
list.
> Sometimes a shift in perspective can dispel a confusion. I ask myself
> when a philosopher -- meaning a person in my discipline -- would have
> occasion to say something like "This paper has been certified
> (validated, legitimated) by peer review because it has been published by
> a reputable peer review refereed journal"? I can't imagine such a
> thing. One would snicker uneasily, wondering what could be meant.
> Certified to be what? True? Validated as officially valid? Stamped as
> authoritatively good? Nobody has given any editor of any journal, much
> less any peer reviewer, the authority to make such pronouncements in
> word or deed, implicitly or explicitly. And my field is almost
> completely controlled organizationally by the organizing focus of the
> refereed journals, to which nobody, including myself, makes any
> objection because that is the sea in which we swim. We all have
> opinions, quite various because our interests are quite various, about
> the worth of this journal and that, some are mainsteam and some more
> marginal, and so forth. But no one translates that into talk about
> certification or validation. All of the talk here in the September
> Forum about validation and certification as the function of peer review,
> and of refereed journals as functioning chiefly to provide the peer
> review validation function, is completely unreal, as regards philosophy
> at least, and in view of the fact that philosophy itself includes people
> with interests and connections in every discipline in the universities,
> I do not think this can be regarded as an idiosyncrasy of the field.
This is an armchair conjecture. The test would be to see what would
happen to the philosophical literature if the peer reviewed journals
were replaced by an online vanity press. Until some local experiments,
at least, are done to test any of this, it seems idle to speculate
about it.
http://www.ecs.soton.ac.uk/~harnad/nature2.html
> I stop at this point without having explained a number of things, one of
> which I think you should understand in order to put what I say into
> proper context, the lack of which causes you to look for explanations of
> what I am saying and even the meaning of what I am saying in the wrong
> place. I should have made clear to you before where I am coming from as
> regards what qualifications I have to be talking about these matters
> here. I am not speaking from a basis in a priori speculation but on
> the basis of experience comparable to your own in some respects, in
> connection with matters clearly relevant here, especially to
> self-archiving. But I don't want to overburden the present message.
Experience comparable to my own would be to have edited a peer reviewed
journal, or to have implemented a self-archive. But even that would not
under-write what you have offered in its place, which is armchair views
about what's what in the Academy in general, and in Philosophy in
particular -- armchair views that (one hopes) would not find their way
into the pages of a reputable peer-reviewed journal in Philosophy.
This is not a peer-reviewed journal; it is only a moderated online
discussion group. But I regret that I will have to use the moderator's
privilege to invoke cloture on Joseph's discussion of university
administrations and the shortcomings of the certification process.
(Vpiej-l is an unmoderated list; perhaps Joseph will have more luck
continuing it there, but I'm afraid it's not part of their mandate
either, as their interest is in electronic journals.)
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 2380 592-582
Computer Science fax: +44 2380 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
ftp://ftp.princeton.edu/pub/harnad/
Received on Wed Feb 10 1999 - 19:17:43 GMT