On 2010-11-23, at 9:46 AM, Philip Davis wrote (in the Sigmetrics Forum):
Critics of our open access publishing experiment (read: Stevan
Harnad) have expressed skepticism that we were too eager to report
our findings and should have waited between 2 and 3 years. All of
the articles in our study have now aged 3-years and we report [1]
that our initial findings [2] were robust: articles receiving the
open access treatment received more article downloads but no more
citations.
ARTICLE DOWNLOADS
During the first year of publication, open access articles received
more than double the number of full-text downloads (119%, 95% C.I.
100% - 140%) and 61% more PDF downloads (95% C.I. 48% - 74%) from a
third more unique visitors (32%, 95% C.I. 24% - 41%). Abstract views
were reduced by nearly a third (-29%, 95% C.I. -34% - -24%)
signaling a reader preference for the full article when available.
ARTICLE CITATIONS
Thirty-six months after publication, open access treatment articles
were cited no more frequently than articles in the control group
(Figure 2). Open access articles received, on average, 10.6
citations (95% C.I. 9.2 -12.0) compared to 10.7 (95% C.I. 9.6 -
11.8) for the control group. No significant citation differences
were detected at 12, 18, 24 and 30 months after publication.
1. Davis, P. M. 2010. Does Open Access Lead to Increased Readership
and Citations? A Randomized Controlled Trial of Articles Published
in APS Journals. The Physiologist 53: 197-201.
http://www.the-aps.org/publications/tphys/2010html/December/open_access.htm
2. Davis, P. M., Lewenstein, B. V., Simon, D. H., Booth, J. G., &
Connolly, M. J. L. 2008. Open access publishing, article downloads
and citations: randomised trial. BMJ 337: a568.
http://dx.doi.org/10.1136/bmj.a568
See: Correlation, Causation, and the Weight of
Evidence
http://openaccess.eprints.org/index.php?/archives/772-guid.html
Phil Davis's dissertation results are welcome and interesting, and include some
good theoretical insights, but insofar as the OA Citation Advantage is
concerned, the empirical findings turn out to be just a failure to replicate the
OA Citation Advantage in that particular sample and time-span. Phil's original
2008 sample of 247 OA and 1372 non-OA articles in 11 journals one year after
publication has now been extended to 712 OA and 2533 non-OA articles in 36
journals 2-3 years after publication. The result is a significant download
advantage for OA articles but no significant citation advantage. (Brody et al
(2006) reported -- for physics article in Arxiv -- that a download advantage in
the first 6 months after publication is correlated with a citation advantage 1.5
years or more after that; see also Gentil-Beccot et al's (2009) data, below).
Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage
Statistics as Predictors of Later Citation Impact. Journal of the
American Association for Information Science and Technology (JASIST)
57(8) pp. 1060-1072.
Gentil-Beccot, A, Mele, S & Brooks TC (2009) Citing and reading
behaviours in high-energy physics. Scientometrics
10.1007/s11192-009-0111-1
The only way to describe this outcome is as a non-replication of the OA Citation
Advantage on this particular sample; it is most definitely not a demonstration
that the OA Advantage is an artifact of self-selection, since there is no
control group demonstrating the presence of the citation advantage with
self-selected OA and the absence of the citation advantage with randomized OA
across the same sample and time-span: There is simply the failure to detect any
citation advantage at all.
This failure to replicate is almost certainly due to the small sample size as
well as the short time-span. (Phil's a-priori estimates of the sample size
required to detect a 20% difference took no account of the fact that citations
grow with time; and the a-priori criterion fails even to be met for the
self-selected subsample of 65.)
"I could not detect the effect in a much smaller and briefer sample than
others" is hardly news! Compare the sample size of Phil's negative results with
the sample-sizes and time-spans of some of the studies that found positive
results:
Slide0001a.gif
And here is how the OA citation advantage builds up with time (read the curves
on the left from bottom to top to see the effect of a longer and longer time
interval: [the topmost curve should read "1998-2008" rather than "1998-2001"]):
citegrowth.gif
Gargouri, Y., Hajjem, C., Lariviere, V., Gingras, Y., Brody, T., Carr, L. and
Harnad, S. (2010) Self-Selected or Mandated, Open Access Increases Citation
Impact for Higher Quality Research. PLOS ONE 10(5) e13636
Harnad, S. (2008) Davis et al's 1-year Study of Self-Selection Bias: No
Self-Archiving Control, No OA Effect, No Conclusion. Open Access
Archivangelism July 31 2008
Stevan Harnad
[ Part 2, Image/GIF (Name: "Slide0001a.gif") 26 KB. ]
[ Unable to print this part. ]
[ Part 3, Image/GIF (Name: "citegrowth.gif") 47 KB. ]
[ Unable to print this part. ]
Received on Tue Nov 23 2010 - 16:30:22 GMT