Re: UK Research Assessment Exercise (RAE) review

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Fri, 29 Nov 2002 14:01:57 +0000

On Thu, 28 Nov 2002, Jan Velterop wrote:

> The perception that I wanted to steer the discussion in the direction
> of peer-review reform is perhaps the reason why Stevan as moderator
> chose not to post my full contribution on the September98-list (fair
> enough, that's his prerogative) but only the bits to which he reacts

No, it was a mistake on my part! I think you posted it originally only to
BOAI, so I didn't re-post it to AmSci, but posted only my quote/comments.
I will go back now, and re-post it from the BOAI archives to AmSci.
I apologize..

> (I'll post the full contribution to the bmanifesto-list shortly, so
> that my open access friends can have a complete record of the
> discussion; the hiatuses are minor, but I just don't like censorship of
> any kind on discussion lists).

Jan, I take full responsibility for the times I invoke cloture, but I
always announce it openly. This was not such a case! It was an error.
(Did you post the original to AmSci too, or only to BOAI?)

> But my topic is not peer-review reform
> per se; the issue is and was the impediments that entrenched,
> traditional scientometric qualifyers are putting up for new open access
> journals. These impediments are presumably alright if one believes that
> open access to peer-reviewed literature is only ever realistically
> possible if articles published in entrenched, traditional journals are
> being mounted on open institutional or self-archives, but I don't, and
> I happen to know quite a few people who believe with me that there are
> other ways to the proverbial Rome as well, such as journals published
> with open access from the outset.

It is not my (undenied) preference for the BOAI-1 route (for the reasons
I have so often stated) that is responsible for the fact that any new
journal, regardless of medium or economic model, faces an uphill battle
until it manages to establish a track-record of some kind. "Track-record"
simply means reliable and persisting evidence (hence predictor)
of quality.

This fact about the need for a track-record is not (just) because of a
bias for "traditional" journals; nor is it because of the tyranny of the
ISI journal-impact factor (though it is certainly time for multivariate
scientometric flowers to be added to the orchard); nor is it in any way
because of my own support for BOAI-1!

The struggle to establish a credible track-record as soon as possible is
quite understandable for all new journals. They require that in order
to attract and sustain authors, readers, citations, and all the other
things a journal needs in order to have high quality and impact.

THe BMC journals are meeting this challenge admirably, and the Faculty
of 1000 reviews are a valuable factor in this.

But please, if BMC is still facing challenges, please don't attribute
them to my own efforts to promote a parallel path to open access (and
one that may eventually make things easier for open-access journals too,
if it succeeds).

>jv> Of course one can subsequently quantify such qualitative
>jv> information. But what a known and acknowledged authority thinks
>jv> of an article is to many more interesting than what anonymous
>jv> peer-reviewers think. Any research assessment exercise should
>jv> seriously look at resources such as offered by Faculty of 1000.
>
>sh> Let 1000 flowers bloom. But it's rather mis-stating the options to
>sh> describe them as open-review vs. anonymous-review! Classical peer
>sh> review is one thing. Then there is post-hoc open-review thereafter.
>
> Your mis-readings; not my mis-statements. Calling F1000 'secondary
> review', as I did, is clearly implying that it is complementary, and
> not an alternative to conventional journal peer-review. The reason why
> any research assessment exercise should look at such secondary
> resources is that they offer a) a second opinion by a known reviewer,
> and b) an opinion on individual papers rather than average track
> records of journals in which those papers were published.

I stand corrected. I was misled by the fact that you were advocating
nonanonymous primary peer review too (but we agree that that is
still just classical peer review, and essential).

>jv> "All BMC's medical have open peer review which works most
>jv> satisfactorily."

Stevan Harnad
Received on Fri Nov 29 2002 - 14:01:57 GMT