Perils of Press-Release Journalism: NSF and Chronicle of Higher Education
In response to my critique of his Chronicle of Higher Education
posting on Evans and Reimer's (2009) Science article (which I
likewise critiqued, though much more mildly), I got an email from
Paul Basken asking me to explain what, if anything he had got wrong,
since his posting was based entirely on a press release from NSF.
Sure enough, the silly spin originated from the NSF Press release
(though the buck stops with E & R's vague and somewhat tendentious
description and interpretation of some of their findings). Here is
the NSF Press Release, enhanced with my comments, for your
delectation and verdict:
____________________________________________________________________________
If you offer something of value to people for free while
someone else charges a hefty sum of money for the same
type of product, one would logically assume that most
people would choose the free option. According to new
research in today's edition of the journal Science, if
the product in question is access to scholarly papers and
research, that logic might just be wrong. These findings
provide new insight into the nature of scholarly
discourse and the future of the open source publication
movement[sic, emphasis added].
(1) If you offer something valuable for free, people will choose the
free option unless they've already paid for the paid option
(especially if they needed -- and could afford -- it earlier).
(2) Free access after an embargo of a year is not the same
"something" as immediate free access. Its "value" for a potential
user is lower. (That's one of the reasons institutions keep paying
for subscription/license access to journals.)
(3) Hence it is not in the least surprising that immediate
print-on-paper access + (paid) online access (IP + IO) generates more
citations than immediate (paid) print-on-paper access (IP) alone.
(4) Nor is it surprising that immediate (paid) print-on-paper access
+ online access + delayed free online access (IP +IO + DF) generates
more citations than just immediate (paid) print-on-paper + online
access (IO + IP) alone -- even if the free access is provided a year
later than the paid access.
(5) Why on earth would anyone conclude that the fact that the
increase in citations from IP to IP + IO is 12% and the increase in
citations from IP + IO to IP + IO + DF is a further 8% implies
anything whatsoever about people's preference for paid access over
free access? Especially when the free access is not even immediate
(IF) but delayed (DF)?
Most research is published in scientific journals and
reviews, and subscriptions to these outlets have
traditionally cost money--in some cases a great deal of
money. Publishers must cover the costs of producing
peer-reviewed publications and in most cases also try to
turn a profit. To access these publications, other
scholars and researchers must either be able to afford
subscriptions or work at institutions that can provide
access.
In recent years, as the Internet has helped lower the
cost of publishing, more and more scientists have begun
publishing their research in open source outlets online.
Since these publications are free to anyone with an
Internet connection, the belief has been that more
interested readers will find them and potentially cite
them. Earlier studies had postulated that being in an
open source format could more than double the number of
times a journal article is used by other researchers.
What on earth is an "open source outlet"? ("Open source" is a
software matter.) Let's assume what's meant is "open access"; but
then is this referring to (i) publishing in an open access journal,
to (ii) publishing in a subscription journal but also self-archiving
the published article to make it open access, or to (iii)
self-archiving an unpublished paper?
What (many) previous studies had measured (not "postulated") was that
(ii) publishing in a subscription journal (IP + IO) and also
self-archiving the published article to make it Open Access (IP + IO
+ OA) could more than double the citations, compared to IP + IO
alone.
To test this theory, James A. Evans, an assistant
professor of sociology at the University of Chicago, and
Jacob Reimer, a student of neurobiology also at the
University of Chicago, analyzed millions of articles
available online, including those from open source
publications and those that required payment to access.
No, they did nothing of the sort; and no "theory" was tested.
Evans & Reimer (E & R) only analyzed articles from subscription
access journals before and after they became accessible online (to
paid subscribers only) (i.e., IP vs IP + IO) as well as before and
after the online version was made accessible free for all (after a
paid-access-only embargo of up to a year or more: i.e., IP +IO vs IP
+ IO + DF). Their methodology was based on comparing citation counts
for articles within the same journals before and after being made
free online at various intervals.
The results were surprising. On average, when a given
publication was made available online after being in
print for a year, being published in an open source
format increased the use of that article by about 8
percent. When articles are made available online in a
commercial format a year after publication, however,
usage increases by about 12 percent.
In other words, the citation count increase from just (paid) IP to
(paid) IP + IO was 12% and the citation count increase from just
(paid) IP + IO to (paid) IP + IO + DF was 8%. Not in the least
surprising: Making paid-access articles accessible online increases
their citations, and making them free online (even if only after a
delay of a year) increases them still more.
What is surprising is the rather absurd spin that this press release
appears to be trying to put on this unsurprising finding.
"Across the scientific community," Evans said in an
interview, "it turns out that open access does have a
positive impact on the attention that's given to the
journal articles, but it's a small impact."
We already knew that OA increased citations, as the many prior
published studies have shown. Most of those studies, however, were
based on immediate OA (i.e., IF), not embargoed OA. What E & R do
show, interestingly, is that even delaying OA for a year still
increases citations, though not nearly as much as immediate OA (IF).
Yet Evans and Reimer's research also points to one very
positive impact of the open source movement that is
sometimes overlooked in the debate about scholarly
publications. Researchers in the developing world, where
research funding and libraries are not as robust as they
are in wealthier countries, were far more likely to read
and cite open source articles.
A large portion of the citation increase from (delayed) OA turns out
to come from Developing Countries (refuting Frandsen's recent report
to the contrary). (A similar comparison, within the US, of citations
from the Have-Not Universities (with the smaller journal subscription
budgets) compared to the Harvards may well reveal the same effect
closer to home, though probably at a smaller scale.)
The University of Chicago team concludes that outside the
developed world, the open source movement "widens the
global circle of those who can participate in science and
benefit from it."
And it will be interesting to test for the same effect comparing the
Harvards and the Have-Nots in the US -- but a more realistic estimate
might come from looking at immediate OA (IF) rather than just
embargoed OA (DF).
So while some scientists and scholars may chose to pay
for scientific publications even when free publications
are available, their colleagues in other parts of the
world may find that going with open source works is the
only choice they have.
It would be interesting to hear the authors of this NSF press release
-- or E & R, for that matter -- explain how this paradoxical
"preference" for paid access over free access was tested during the
access embargo period...
Stevan Harnad
American Scientist Open Access Forum
Received on Wed Feb 25 2009 - 23:39:01 GMT
This archive was generated by hypermail 2.3.0
: Fri Dec 10 2010 - 19:49:43 GMT