On Fri, 11 Aug 2000, David Goodman wrote:
> to divide the world into peer-reviewed journals vs non-peer reviewed is
> not quite the full story. It's the quality of the review that matters...
Correct. And there is, and always was, a quality-hierarchy of refereed
journals, all the way from the most rigorously refereed ones at the top,
to the vanity press at the bottom. The point of the second "C" in QC/C
(Quality-Control & Certification) is that this hierarchy is sign-posted,
so the user knows which "brands" he can trust, and how much he can trust
them.
The peer review "system" merely amounts to experts vetting the work of
fellow-experts for fellow-experts. The rest is just human judgement and
human nature.
This "system" can no doubt always stand some improvement, but
"improvements" need to be tried and tested first, without putting the
current QC/C literature at risk, and implemented only later, if the
"improvement" indeed improves, or at least does not worsen, current QC
levels.
This Forum is dedicated to freeing the QC/C literature, SUCH AS
IT IS, online, now. Improving on QC/C is another matter, and the fate
of the current literature need not and should not be tied to it in any
way. Freeing the current QC/C literature now is a face-valid end in
itself; altering or improving upon QC/C are, for now, merely speculative
matters.
> many... peer-reviewed journals... publish some material without
> review, or... lower standard of review Conferences are sometimes
> published [without] "review"
True. And the users of the literature know that already; those are the
vagaries of the current QC/C system, which is not perfect, but
track-recorded and sign-posted.
> There are also other forms of quality control than peer-review.
I would like to know what they are, and the evidence showing that they
work. I have replied many times in these pages to pre-emptive calls for
replacing peer review by, for example, open commentary (a non-starter on
which I have some expertise), or for abandoning peer review altogether.
Harnad, S. (1998) The invisible hand of peer review. Nature [online]
http://helix.nature.com/webmatters/invisible/invisible.html
(2000) Exploit Interactive 5
http://www.exploit-lib.org/issue5/peer-review/
> The quality of much of the material on the web is a disgrace from one
> viewpoint, and a tribute to free speech on the other.
We need not be concerned here with the overall quality of material on
the Web. We need only concern ourselves with the peer-reviewed sector,
which is at the moment almost 100% fire-walled by the access barriers
of Subscription/Site-License/Pay-Per-View [S/L/P]. Free it from those
barriers and sign-post it with its QC certification labels (via the
Santa Fe Convention of the Open Archive Initiative
http://www.openarchives.org) by self-archiving it in
Santa-Fe-compliant, interoperable institutional research archives
(
http://www.eprints.org), and you have ensured that at least the
QC-certified portions of the Web are no disgrace (or no moreso than the
current on-paper refereed literature).
> The presence on the web, and in the printed literature, of material
> that was once peer-reviewed but is now obsolete and dangerous is
> another problem--this has been discussed in particular about medical
> information.
Nolo contendere: As you say, this is a problem for both the on-line and
on-paper literature, and we cannot save the world here. Let us content
ourselves with freeing the current refereed corpus...
> the need for quality control does not imply that we need the entire
> scientific publishing industry. Now that the industry isn't needed for
> dissemination or archiving, it's the only ostensible justification they
> have, but it is not true that no other way of doing quality control (as
> peer review or otherwise) can be found than the maintenance of the
> entire structure.
Yes, implementing QC/C is the only remaining essential function of
refereed journal publishing; but don't discard it until/unless you have
a proven alternative. Fix what needs immediate fixing now, and worry
about the rest later; don't conflate the two agendas.
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 23-80 592-582
Computer Science fax: +44 23-80 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
NOTE: A complete archive of this ongoing discussion of providing free
access to the refereed journal literature is available at the American
Scientist September Forum (98 & 99 & 00):
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
You may join the list at the site above.
Discussion can be posted to:
american-scientist-open-access-forum_at_amsci.org
Received on Mon Jan 24 2000 - 19:17:43 GMT