On Thu, 5 Sep 2002, David Goodman wrote:
> Well, Stevan, how do I know that your messages are usually worth
> reading?
But, David, my postings are not academic publications! Postings are a
new form of preserved, recorded, public online interaction. No one is
proposing that that should be replaced by peer-reviewed publication -- but
it would be equally erroneous to suggest that peer-reviewed publication
should be replaced by such self-postings on the Web (supplemented by
comments, solicited and unsolicited).
> Seriously, (and please excuse the liberty of using you as an example)
> how DO I know? I think that I know
> because I've found it valuable to read earlier messages. I initially
> discovered this because a colleague whom I trusted informed me about
> them. Had she not done so, I would have found out anyway because many
> people refer to them. Eventually, I heard you give a talk--which was a
> sort of certification by the organizers of the conference.
> These are all the informal channels of communication, and they are of
> course in the life of many or most of us more important than the formal.
Yes, but when we are discussing (1) the evaluation of research by
its employers and funders and (2) the navigation of research by its
potential users, it is formal communications (refereed publications)
and their formal validation and labelling (peer-review/journal-name)
that is not only important but necessary. And not just to filter and
tag the quality level reliably and recognizably for (1) and (2), but
also to generate the quality itself -- for, as I noted, peer review is
not merely a yes/no filter but a dynamic, corrective process helping to
get research reports to the quality level of the journal-tag in question
(those of them that have the possibility of reaching that level: some
journals, with rejection rates of 95%, are obviously markers of the very
top of the hierarchy, and most papers cannot reach that level even with
the help of feedback from referees).
> I will admit that for the purposes of serious administrative decisions
> affecting people's careers, informal channels may not be
> appropriate by themselves--though
> here too they do play a role. And, like all methods of
> formal and informal interpersonal interaction, they are subject to abuse.
All true, but neither here nor there, I am afraid, for the real
issue under discussion here, which is open access to the peer-reviewed
literature -- such as it is, leaving peer review 100% intact -- for the
formal interaction you note. Yes, there are informal supplements to this
formal role, but as they are in no wise substitutes for it, one wonders
why they would even be raised in this context: One might as well have
said in the paper era: "What do you need formal journals for? You can
use informal sources."
It would be very wrong to conclude that the reason the latter would
have been an obvious nonstarter in the paper era is merely because
paper did not allow the same volume of informal communication that the
Web does. On the contrary, the spectacular increase in the volume of
the informal communication makes the filtering, certification and clear
tagging of the formal, quality-controlled sector all the more important.
> Speaking of certification, by journals, how many journals are there whose
> certification we fully trust? In terms of quality, there are a few in
> each field; in terms of personal relevance, that's another matter.
That's another matter entirely -- and one that we can discuss once
the refereed literature is freely accessible online (as it should
already be!). Problems with (and reform of) journal quality-levels and
their quality-control system (peer review) are completely independent
of problems with journal accessibility, such as it is, now. Please, let
us not further retard or distract from the sure and optimal agenda of
the one (open access) with the much more uncertain and still-to-be-tested
agenda of the other (peer-review reform). Peer-review reform can proceed
apace; it has almost nothing to do with the online medium and the newfound
possibility of open access (apart from new efficiencies and economies in
implementing classical peer review online). There were the same quality
problems in the paper era.
"Peer Review Reform Hypothesis-Testing"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/0479.html
"A Note of Caution About 'Reforming the System'"
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/1169.html
> You are right, incidentally, that "certification"
> is a better word for what we're talking about than "labeling."
But Jan is also right, that this is merely semantics (or semiology,
rather). For the relevant fact is this:
Call them what we like (and why not just continue calling them by their
current, familiar journal-names?), the peer-reviewed research literature
will always need:
(1) peer-review
(2) peer-review service-providers (service-implementers, actually, as
the referees actually provide the service for free), with established
track-records for their quality-standards.
(3) certification of having met the quality-standards of peer review.
(4) reliable, recognizable "tags" to mark the quality-level (often
correlated with the rejection rate, sometimes the citation impact factor)
of the peer-review service-provider.
It seems to me more sensible to continue calling the providers of these
invariant functions that they have always provided for us "journals." What
is gained by calling them something else, just because the research
communication they have been certifying all along may at last become
freely accessible online? Among other things, the signal value of the tag,
as a reliable marker, depends on some name-continuity!
Cheers, Stevan
Received on Fri Sep 06 2002 - 14:02:14 BST