PEER REVIEW IS AND ALWAYS WAS THE FILTER AND "AUTHORITY"
[Comment on: Umberto Eco's "Authors and Authority"
http://www.text-e.org/debats/ Further commnets invited]
Stevan Harnad
> UMBERTO ECO: "[filtering information] is the fundamental problem of
> the Web."
So many have said this, in one form or another. And, faced with the
heterogeneous hash on the Web today, most of it trash, even if
"filtered" by Google's link-count economy, it is understandable why
people might think that filtering it all is a new problem, unique to
the web, and THE problem.
But it is not; it is none of these things: the problem is old, predates
the web, and solved.
For science and scholarship, the filter is called PEER REVIEW, which is
nothing mysterious: it is just qualified specialists (referees),
assessing and improving the quality of the work of their
fellow-specialists, mediated by a meta-specialist (the editor), before
the work is published with a quality-control "tag" (the journal name),
certifying that it has met the established quality standards of that
particular journal.
The system is nothing unique or new; similar principles are involved in
certifying the quality of eggs.
What is new is that the economies of the on-line medium make it
possible for authors to bypass this filtration if they like,
"skywriting" their raw texts up into the PostGutenberg Galaxy directly.
What has made this vast vanity press -- this global graffiti board for
trivial pursuit -- possible is simply a new economic fact: It is now so
much cheaper to disseminate one's texts on-line than on-paper that it
is within reach of every living soul who can read and write. In the
Gutenberg Galaxy, the "filter" against this was simply the prohibitive
cost of publication.
But if, counterfactually, it had been as cheap and easy to print and
disseminate paper in those days as it is to stand up on a soapbox in
Hyde Park (or to hold forth on today's chat-radio and chat-TV), then
the "filtration" problem would have been there then too.
And the solution. Which is that if one is serious about one's inquiries,
one will restrict them to the texts of qualified experts, certified
by their qualified fellow-experts.
On the Web, we are simply assaulted by the new reality: Rather as if
the older media -- radio, TV -- had suddenly been augmented and
overwhelmed by billions of new channels, all purveying people's
personal (or corporate) phonograffiti. Except there, in the oral
tradition, we would not have had the established mechanism of peer
review and its quality-tagging to fall back on as our guide. Never
mind; we would still have figured out how to choose reliable
channels eventually. The Web has the advantage that, as a special case,
it inherits the peer-review filter gratis.
Gratis? Not quite. Peers review for free, and always did. And they also
write and distribute their papers for free. But someone always had to
pay for the IMPLEMENTATION of peer review (receiving the manuscripts,
circulating them to the qualified reviewers, tracking the process and
keeping it on schedule), and that cost some money (about $200 - $500
per accepted paper). But this relatively modest cost was bundled in
with many additional, purely Gutenberg costs (printing, distributing
and marketing the paper -- and lately also the paper images, PDF,
on-line). All those costs were wrapped into the journal, which was then
sold as everything else on-paper was sold: for a fee.
Unlike with books and magazines, though, the authors of these
peer-reviewed papers never sought or received royalties, salaries,
or fees in exchange for their texts. All they ever wanted was as many
of their fellow-researchers as possible to read, use, cite, and build
upon their work.
And here is the real key to solving the "filtration" non-problem: As
soon as this peer-reviewed (filtered) literature is as freely and fully
available online as the unfiltered literature, the "brand names"
(properly authenticated through OAI-compliant metadata tagging) will
perform exactly the same filtration function they always performed on
paper. (See OAI reference below.)
But this will alas only work for this give-away literature. For the
non-give-away literature, the "filter" will continue to be a financial
one.
> ECO: "With the Web, everyone is in the situation of having to filter
> information that is so vast, and so unsustainable, that if it isn't
> filtered it cannot be absorbed... [W]e'll end up with a
> civilization in which every person has his own system of filters,
> in other words where every person creates his own encyclopaedia.
> Now a society with five billion concurrent encyclopaedias is a
> society in which there is no more communication."
Needlessly apocalyptic, it seems to me. First of all, we have the wrong
tertium comparationis here. The information sources each person collects
and uses are not "his own encyclopedia." They are simply the sources he
chooses. And once the peer-reviewed texts, with their quality-control
labels, are freely available online too, the person who chooses to
rely on unfiltered cybergraffiti will have only himself to blame.
> ECO: "[Search engines are] not a filtering system. There are already
> polemics on the fact that search engines "filter" only information
> that has been paid for. I don't believe in the possibility of
> automating the filter's function. The only solution is that there
> appear authorities, external or internal to the Web, that
> constantly monitor what is found."
No need for new authorities. Peer review will do. All that is needed is
that the peer-reviewed literature be as freely available on the Web as
the rest of the give-away literature on it. (Sorry, no solution for
authoritative information for which the author wishes to be paid for
accessing the full text, but even there, at least the metadata --
title, date, source, author, summary, keywords -- can be available to
compete for authority with the free cybergraffiti.)
(Of course, for other forms of online give-away than peer reviewed
science and scholarship, such as teaching materials, new "authorities"
will no doubt evolve, along with a tagging system of their own, if
the authors are ever to get proper credit, and users proper use, out
of this work.)
> ECO: "There should be specialized monitoring groups, for example the
> International Society for Philosophy"
No need. The name of the journal, and its established and answerable
reputation will do, as it did before. (Sometimes the publisher is indeed
a Learned Society.)
> ECO: "[How] does the ingenuous visitor know that this site is the
> monitoring, expert one?"
Proper metadata tagging and authentication. (With the authenticated
tags as guide, even Google alone could do the rest, filtering out
only the tagged -- i.e. peer-reviewed -- literature.) See OAI reference
below.
> ECO: "We haven't resolved these problems. If I had the answer I'd
> probably become a billionaire, but I don't."
The filtering and tagging and authentication problems are solved. And
they will not make anyone a billionaire. If anything, they may
diminish the revenues of some rather prosperous journal publishers.
But before that can happen, scientists and scholars need to apply the
solutions (by ensuring that there is free online access to all their
peer-reviewed papers: See BOAI below).
> ECO: "A filtering authority is not a "censor" but a consultant."
Correct. Or, even more specifically, in the PostGutenberg age a
learned journal becomes a peer-review service that controls the quality
of scientific and scholarly research, and certifies the outcome. (Nor
is this merely passive green-light/red-light, filtering! Peer review
is a dynamic interaction, sometimes involving several rounds of
refereeing, revision, and re-refereeing, mediated and adjudicated by the
editor.
> ECO: "A publisher is a filter."
A (journal) publisher is a filter-implementer. The filtering itself is
done by the peer-reviewers (for free, for journals). Books, being
non-give-aways, are a different story entirely.
> ECO: "The other problem with Internet is the spread
> of the samizdat."
Samizdat was mainly artistic (normally a non-give-away sphere).
I don't know of much scientific or scholarly samizdat of the kind
normally destined for peer-reviewed journals. (Or if it was smuggled
out, it did appear in peer-reviewed journals!) Samizdat is not the
right homolog for cybergraffiti: Vanity press, Hyde Park, and Chat
Media are.
> ECO: "[The Web] could establish an "anything goes" of taste"
Haven't the pre-web mass media (radio, TV, movies, disks) already done that?
> ECO: "But... taste filters in literature have only ever interested
> 0,5 per cent of the population. If today 70 per cent of the
> population surfs the Web and considers good any poem or story it
> encounters, we can say that these people had been excluded from the
> enjoyment of the literary product and that they have finally come
> in contact with some form of literary expression."
Agreed. So there is in fact no particular new problem to speak of here
(though one might note that the mass media have already had this effect,
even before the Web).
> ECO: "Our species has gotten used to a certain sort of attention,
> which involves turning pages, lingering with attention; the kind of
> reading effected on screen inevitably is different, faster, one
> scrolls with greater speed... However, new generations [will adapt]
> to the screen."
And/or the more generalized "virtual book" will emulate more of the
features of on-paper reading -- those that really turn out to be
optimal, and not merely habitual.
ECO: "it has never happened in the history of humanity that the
introduction of a technological means killed off all the practices
of the previous means. Even the wheel did not entirely supplant the
sledge; photography did not destroy painting, if anything it gave
it a new direction; and all the statistics tell us that where
people watch a lot of television there tends to be a higher rate of
newspaper use: it isn't true that people who watch too much
television don't read the papers. I believe that the increase in
information, even if it is on screen, will not have an effect on
the use of books..."
Perhaps. (But this might be underestimating the potential simulational
and subsumptive power of virtual books. VR does not merely extend; in
some cases it can replace. Nor am I so sure that TV has not killed off
a lot of actual and potential literacy-- of which newspaper-reading is
hardly a sterling example...)
> ECO: "Then there are the emotional, aesthetic, tactile dimensions -
> so you can leaf through the beautiful paper of a book, although
> here too it could be that in three generations things might well be
> different."
Again, let us not underestimate the V-book in this respect, nor prejudge
what stimuli are sufficient to engage our emotions...
References
URLS:
http://www.openarchives.org
http://www.soros.org/openaccess
Harnad, S. (1990) Scholarly Skywriting and the Prepublication Continuum
of Scientific Inquiry. Psychological Science 1: 342 - 343 (reprinted in
Current Contents 45: 9-13, November 11 1991).
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad90.skywriting.html
Harnad, S. (1991) Post-Gutenberg Galaxy: The Fourth Revolution in the
Means of Production of Knowledge. Public-Access Computer Systems
Review 2 (1): 39 - 53 (also reprinted in PACS Annual Review Volume 2
1992; and in R. D. Mason (ed.) Computer Conferencing: The Last Word.
Beach Holme Publishers, 1992; and in: M. Strangelove & D. Kovacs:
Directory of Electronic Journals, Newsletters, and Academic Discussion
Lists (A. Okerson, ed), 2nd edition. Washington, DC, Association of
Research Libraries, Office of Scientific & Academic Publishing, 1992);
and in Hungarian translation in REPLIKA 1994; and in Japanese in
Research and Development of Scholarly Information Dissemination Systems
1994-1995.
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad91.postgutenberg.html
Harnad, S. (1998/2000) The invisible hand of peer review. Nature
[online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible/invisible.html
Longer version in Exploit Interactive 5 (2000):
http://www.exploit-lib.org/issue5/peer-review/
http://www.ecs.soton.ac.uk/~harnad/nature2.html
Harnad, S., Varian, H. & Parks, R. (2000) Academic publishing in the
online era: What Will Be For-Fee And What Will Be For-Free? Culture
Machine 2 (Online Journal)
http://www.ecs.soton.ac.uk/~harnad/Temp/Varian/new1.htm
http://culturemachine.tees.ac.uk/Cmach/Backissues/j002/Articles/art_harn.htm
Harnad, S. (2001) Skyreading and Skywriting for Researchers: A
Post-Gutenberg Anomaly and How to Resolve it.
http://www.text-e.org/conf/index.cfm?ConfText_ID=7
Received on Thu Feb 28 2002 - 20:51:33 GMT