On Thu, 16 Jan 2003, Andrew Odlyzko wrote [in part]:
[ao]> The recent postings to this list about rejection
[ao]> rates and costs of peer review point out yet
[ao]> another way that costs can be lowered: Elimination
[ao]> of the wasteful duplication in the peer review system.
Publishers of several journals can achieve economies of scale
by using the same staff to oversee multiple journals.
Economies of scale for the peer reviewers would require
centralized peer review for a particular field or
discipline. This approach has been tested in Canada by
the Canadian Breast Cancer Research Initiative (CBCRI):
<
http://www.breast.cancer.ca/>. The Canadian Institutes
of Health Research (CIHR), the National Cancer Institute of
Canada (NCIC), the Canadian Cancer Society, Health Canada,
and the Canadian Breast Cancer Research Foundation (CBCF),
all use the same peer review system (that of the NCIC)
for the evaluation of research proposals submitted directly
to the CBCRI.
However, this hasn't really achieved much economy of scale,
because some of these agencies (NCIC, CIHR, CBCF) also, for
what I think are good reasons, also peer-review those
breast-cancer applications that are sent directly to them,
rather than to the CBCRI. The individual research teams
make the decision (and some choose, again for what I think
are good reasons) to submit essentially the same application
to more than one of these various agencies.
Different peer-review committees judge "quality" according
to somewhat different criteria, and involve committee
members who may be true "peers" in relation to one aspect of
as research field or discipline, but not in relation to
another. The mix of expertise matters.
So, many research teams prefer to have an opportunity to
"take more than one kick at the can". If peer-review is
regarded as a process of weighted randomization, then, from
the point of view of an individual research team, the
probability of successfully obtaining support is increased
if multiple applications are submitted.
The situation isn't very different for peer-review of
research reports, except that the number of "peers"
involved in the review process is usually much smaller
(e.g. 2 or 3 people, instead of about 10). The smaller
the number of reviewers, the greater the variance in the
score or rating of perceived quality.
Jim Till
University of Toronto
Received on Thu Jan 16 2003 - 16:54:26 GMT